2024-04-26T04:35:18.759 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T04:35:18.759 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi --verbose --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/orig.config.yaml 2024-04-26T04:35:18.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T04:35:18.898 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581 branch: squid description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7674581' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8320 sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a sleep_before_teardown: 0 subset: 10/32 suite: upgrade suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a targets: smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHokJW6KKOphAlUAn1Gx1/UP8XseML1VMycrSBv1nQQ1/dIpgnqyI3TpBiwQYhDdalWfw3FYL81jXKDCE99hi8g= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOp+Y3P28sQr8wXSrFkYhaXy48KVGlV8+JsgWS2VxVwXXWKbBMW84m8U+YDunFdWQ6WzgqHsTJJZvZojMKivmzA= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/upgrade/cephfs/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_02:08:13 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-26T04:35:18.898 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa; will attempt to use it 2024-04-26T04:35:18.899 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks 2024-04-26T04:35:18.899 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T04:35:18.900 INFO:teuthology.task.internal:Checking packages... 2024-04-26T04:35:18.927 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a' 2024-04-26T04:35:18.927 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:35:18.927 INFO:teuthology.packaging:ref: None 2024-04-26T04:35:18.927 INFO:teuthology.packaging:tag: None 2024-04-26T04:35:18.927 INFO:teuthology.packaging:branch: squid 2024-04-26T04:35:18.927 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:35:18.927 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-26T04:35:19.129 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2447.gb22e2ebd 2024-04-26T04:35:19.131 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T04:35:19.140 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T04:35:19.140 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T04:35:19.222 INFO:teuthology.task.internal:Saving configuration 2024-04-26T04:35:19.242 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T04:35:19.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T04:35:19.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:30:16.524234', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHokJW6KKOphAlUAn1Gx1/UP8XseML1VMycrSBv1nQQ1/dIpgnqyI3TpBiwQYhDdalWfw3FYL81jXKDCE99hi8g='} 2024-04-26T04:35:19.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:30:16.525782', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOp+Y3P28sQr8wXSrFkYhaXy48KVGlV8+JsgWS2VxVwXXWKbBMW84m8U+YDunFdWQ6WzgqHsTJJZvZojMKivmzA='} 2024-04-26T04:35:19.340 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T04:35:19.354 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-26T04:35:19.354 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-26T04:35:19.355 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T04:35:19.496 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff718be10d0>, signals=[15]) 2024-04-26T04:35:19.497 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T04:35:19.506 INFO:teuthology.task.internal:Opening connections... 2024-04-26T04:35:19.507 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:35:19.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:35:19.591 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-04-26T04:35:19.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:35:19.669 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T04:35:19.677 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-04-26T04:35:19.695 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-04-26T04:35:19.696 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-04-26T04:35:19.753 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T04:35:19.754 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T04:35:19.755 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-04-26T04:35:19.777 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-26T04:35:19.794 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-26T04:35:19.795 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-26T04:35:19.851 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-04-26T04:35:19.851 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-04-26T04:35:19.851 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-04-26T04:35:19.851 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T04:35:19.852 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T04:35:19.852 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-26T04:35:19.876 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T04:35:19.888 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T04:35:19.975 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T04:35:19.975 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:35:19.978 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:35:19.994 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T04:35:20.025 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T04:35:20.026 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:35:20.035 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:35:20.147 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T04:35:20.200 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T04:35:20.223 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-26T04:35:20.224 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-26T04:35:20.224 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:35:20.224 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:35:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:35:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:35:20.241 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-04-26T04:35:20.242 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-04-26T04:35:20.299 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-437.el9.x86_64 2024-04-26T04:35:20.299 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.14.0-437.el9.x86_64 2024-04-26T04:35:20.299 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2024-04-26T04:35:20.300 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-437.el9.x86_64 2024-04-26T04:35:20.300 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-437.el9.x86_64 2024-04-26T04:35:20.301 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-04-26T04:35:22.952 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-04-26T04:35:23.154 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T04:35:25.478 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-26T04:35:25.957 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-26T04:35:30.271 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.2 MB 00:01 2024-04-26T04:35:30.601 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-26T04:35:31.829 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-26T04:35:32.248 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-26T04:35:33.881 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T04:35:34.177 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T04:35:38.912 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-26T04:35:39.191 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T04:35:40.818 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T04:35:40.818 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T04:35:40.898 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:35:40.899 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:35:40.899 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:35:41.010 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2024-04-26T04:35:41.064 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T04:35:41.064 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T04:35:41.155 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:35:41.156 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:35:41.157 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:35:41.275 DEBUG:teuthology.orchestra.run.smithi077:> echo no | sudo yum reinstall kernel || true 2024-04-26T04:35:41.505 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 04:35:38 AM UTC. 2024-04-26T04:35:41.624 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:35:41.645 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-26T04:35:41.645 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:41.645 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-26T04:35:41.646 INFO:teuthology.orchestra.run.smithi078.stderr:Operation aborted. 2024-04-26T04:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:Is this ok [y/N]: 2024-04-26T04:35:41.706 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2024-04-26T04:35:41.769 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 04:35:39 AM UTC. 2024-04-26T04:35:41.909 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:35:41.926 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:41.926 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:Reinstalling: 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 255 k 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 0 2024-04-26T04:35:41.927 INFO:teuthology.orchestra.run.smithi077.stderr:Operation aborted. 2024-04-26T04:35:41.983 INFO:teuthology.orchestra.run.smithi077.stdout:Is this ok [y/N]: 2024-04-26T04:35:41.983 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum reinstall -y kernel || true 2024-04-26T04:35:42.199 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 04:35:38 AM UTC. 2024-04-26T04:35:42.314 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:35:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:42.316 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-26T04:35:42.316 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-26T04:35:42.316 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:35:42.471 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 04:35:39 AM UTC. 2024-04-26T04:35:42.586 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:Reinstalling: 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:35:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:42.588 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 255 k 2024-04-26T04:35:42.588 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 0 2024-04-26T04:35:42.588 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:35:42.589 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-26T04:35:42.589 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:35:42.590 INFO:teuthology.orchestra.run.smithi078.stdout:Total 930 kB/s | 255 kB 00:00 2024-04-26T04:35:42.612 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:35:42.781 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:35:42.782 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:35:42.784 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-26T04:35:42.785 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:35:42.785 INFO:teuthology.orchestra.run.smithi077.stdout:Total 1.3 MB/s | 255 kB 00:00 2024-04-26T04:35:42.786 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:35:42.787 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:35:42.803 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:35:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:35:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:35:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:42.976 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:35:42.976 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:35:42.981 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:35:42.982 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:35:43.126 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:35:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:35:43.236 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:35:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:43.874 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:35:44.302 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:44.302 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:44.303 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalled: 2024-04-26T04:35:44.303 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T04:35:44.303 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:35:44.303 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:35:44.459 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T04:35:44.506 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T04:35:44.506 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T04:35:44.507 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T04:35:44.507 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-26T04:35:44.507 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:35:44.540 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2024-04-26T04:35:44.569 DEBUG:teuthology.parallel:result is None 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:Reinstalled: 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:35:44.581 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:35:44.765 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T04:35:44.812 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T04:35:44.812 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T04:35:44.813 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T04:35:44.813 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-26T04:35:44.813 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:35:44.847 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2024-04-26T04:35:44.869 DEBUG:teuthology.parallel:result is None 2024-04-26T04:35:44.869 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T04:35:44.876 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T04:35:44.876 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:35:44.912 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:35:44.926 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T04:35:44.954 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T04:35:44.999 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T04:35:45.000 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:35:45.002 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:35:45.030 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T04:35:45.073 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T04:35:45.073 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:35:45.075 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:35:45.105 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:35:45.111 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:35:45.113 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:35:45.119 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:35:45.121 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T04:35:45.156 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T04:35:45.156 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:35:45.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:35:45.187 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T04:35:45.397 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T04:35:45.397 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:35:45.400 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:35:45.416 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:35:45.500 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:35:45.550 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:35:45.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:35:45.685 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:35:45.685 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:35:45.753 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:35:45.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:35:45.850 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:35:45.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:35:45.983 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:35:45.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:35:46.048 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-04-26T04:35:46.050 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-04-26T04:35:46.082 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:35:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:35:46.547 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T04:35:46.672 INFO:teuthology.task.internal:Starting timer... 2024-04-26T04:35:46.672 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T04:35:46.741 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T04:35:46.872 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T04:35:46.872 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-04-26T04:35:46.923 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-04-26T04:35:46.925 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-04-26T04:35:46.965 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-04-26T04:35:46.967 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T04:35:46.967 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-04-26T04:35:46.989 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-04-26T04:35:46.990 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-04-26T04:35:47.030 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-04-26T04:35:47.031 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive'} 2024-04-26T04:35:47.031 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:35:47.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:35:47.055 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:35:47.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:35:47.094 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T04:35:47.095 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-04-26T04:35:47.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-04-26T04:35:47.167 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T04:35:47.177 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T04:35:47.182 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T04:35:47.312 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T04:35:47.328 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T04:35:47.330 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi078.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T04:39:16.218 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com')] 2024-04-26T04:39:16.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-26T04:39:16.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:39:16.296 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-04-26T04:39:16.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-04-26T04:39:16.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-26T04:39:16.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:39:16.458 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-26T04:39:16.541 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-26T04:39:16.542 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T04:39:16.552 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T04:39:16.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:39:16.553 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:39:16.555 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:39:16.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:39:16.581 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T04:39:16.594 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T04:39:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T04:39:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-04-26T04:39:16.623 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T04:39:16.630 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-04-26T04:39:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T04:39:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-04-26T04:39:16.656 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-04-26T04:39:16.656 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T04:39:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T04:39:16.681 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T04:39:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.717 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.748 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:39:16.751 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:39:16.751 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-26T04:39:16.751 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.751 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.751 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.752 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:39:16.752 INFO:teuthology.run_tasks:Running task install... 2024-04-26T04:39:16.761 DEBUG:teuthology.task.install:project ceph 2024-04-26T04:39:16.761 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}} 2024-04-26T04:39:16.761 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'} 2024-04-26T04:39:16.761 INFO:teuthology.task.install:Using flavor: default 2024-04-26T04:39:16.767 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T04:39:16.767 INFO:teuthology.task.install:extra packages: [] 2024-04-26T04:39:16.767 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': None, 'wait_for_package': False} 2024-04-26T04:39:16.767 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:39:16.768 INFO:teuthology.packaging:ref: None 2024-04-26T04:39:16.768 INFO:teuthology.packaging:tag: None 2024-04-26T04:39:16.768 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:39:16.768 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:39:16.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-26T04:39:16.770 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': None, 'wait_for_package': False} 2024-04-26T04:39:16.770 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:39:16.770 INFO:teuthology.packaging:ref: None 2024-04-26T04:39:16.770 INFO:teuthology.packaging:tag: None 2024-04-26T04:39:16.770 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:39:16.770 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:39:16.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-26T04:39:17.092 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/ 2024-04-26T04:39:17.093 INFO:teuthology.task.install.rpm:Package version is 17.2.7-904.gd8c6b0a3 2024-04-26T04:39:17.107 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/ 2024-04-26T04:39:17.108 INFO:teuthology.task.install.rpm:Package version is 17.2.7-904.gd8c6b0a3 2024-04-26T04:39:17.202 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T04:39:17.202 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:39:17.202 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T04:39:17.227 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T04:39:17.227 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:39:17.228 INFO:teuthology.packaging:ref: None 2024-04-26T04:39:17.228 INFO:teuthology.packaging:tag: None 2024-04-26T04:39:17.228 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:39:17.228 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:39:17.228 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T04:39:17.281 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T04:39:17.282 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:39:17.282 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T04:39:17.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T04:39:17.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T04:39:17.305 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:39:17.305 INFO:teuthology.packaging:ref: None 2024-04-26T04:39:17.305 INFO:teuthology.packaging:tag: None 2024-04-26T04:39:17.305 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:39:17.305 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:39:17.306 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T04:39:17.366 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T04:39:17.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T04:39:17.429 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-04-26T04:39:17.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-04-26T04:39:17.443 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T04:39:17.506 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-04-26T04:39:17.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-04-26T04:39:17.656 INFO:teuthology.orchestra.run.smithi078.stdout:48 files removed 2024-04-26T04:39:17.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw 2024-04-26T04:39:17.729 INFO:teuthology.orchestra.run.smithi077.stdout:48 files removed 2024-04-26T04:39:17.755 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw 2024-04-26T04:39:18.375 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 179 kB/s | 77 kB 00:00 2024-04-26T04:39:18.437 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 181 kB/s | 77 kB 00:00 2024-04-26T04:39:18.862 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-26T04:39:19.000 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-04-26T04:39:19.609 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 3.1 kB/s | 1.8 kB 00:00 2024-04-26T04:39:19.621 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 2.5 kB/s | 1.8 kB 00:00 2024-04-26T04:39:20.236 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T04:39:20.242 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T04:39:22.067 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T04:39:22.597 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-26T04:39:26.718 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-26T04:39:26.973 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-26T04:39:28.192 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-26T04:39:28.438 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-26T04:39:29.346 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T04:39:29.489 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-26T04:39:34.360 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-04-26T04:39:34.631 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T04:39:36.480 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 14 M 2024-04-26T04:39:36.482 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 3.1 M 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.9 M 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 5.0 M 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 22 M 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 25 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 637 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 457 k 2024-04-26T04:39:36.483 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 4.9 M 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 45 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 117 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 156 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 320 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 293 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 100 k 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:39:36.484 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:39:36.485 INFO:teuthology.orchestra.run.smithi077.stdout:Install 22 Packages 2024-04-26T04:39:36.485 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 2 Packages 2024-04-26T04:39:36.485 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:39:36.485 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 58 M 2024-04-26T04:39:36.485 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:39:36.783 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 14 M 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 3.1 M 2024-04-26T04:39:36.785 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.9 M 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 5.0 M 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 22 M 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 25 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 637 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 457 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T04:39:36.786 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 4.9 M 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 45 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 117 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 156 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 320 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 293 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 100 k 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:39:36.787 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:39:36.788 INFO:teuthology.orchestra.run.smithi078.stdout:Install 22 Packages 2024-04-26T04:39:36.788 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 2 Packages 2024-04-26T04:39:36.788 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:39:36.788 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 58 M 2024-04-26T04:39:36.788 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:39:37.861 INFO:teuthology.orchestra.run.smithi077.stdout:(1/24): ceph-base-17.2.7-904.gd8c6b0a3.el9.x86_ 5.7 MB/s | 5.0 MB 00:00 2024-04-26T04:39:37.928 INFO:teuthology.orchestra.run.smithi077.stdout:(2/24): ceph-selinux-17.2.7-904.gd8c6b0a3.el9.x 378 kB/s | 25 kB 00:00 2024-04-26T04:39:38.078 INFO:teuthology.orchestra.run.smithi077.stdout:(3/24): libcephfs2-17.2.7-904.gd8c6b0a3.el9.x86 4.1 MB/s | 637 kB 00:00 2024-04-26T04:39:38.178 INFO:teuthology.orchestra.run.smithi077.stdout:(4/24): libradosstriper1-17.2.7-904.gd8c6b0a3.e 4.5 MB/s | 457 kB 00:00 2024-04-26T04:39:39.154 INFO:teuthology.orchestra.run.smithi077.stdout:(5/24): librgw2-17.2.7-904.gd8c6b0a3.el9.x86_64 5.0 MB/s | 4.9 MB 00:00 2024-04-26T04:39:39.237 INFO:teuthology.orchestra.run.smithi077.stdout:(6/24): python3-ceph-argparse-17.2.7-904.gd8c6b 542 kB/s | 45 kB 00:00 2024-04-26T04:39:39.337 INFO:teuthology.orchestra.run.smithi077.stdout:(7/24): python3-ceph-common-17.2.7-904.gd8c6b0a 1.2 MB/s | 117 kB 00:00 2024-04-26T04:39:39.429 INFO:teuthology.orchestra.run.smithi077.stdout:(8/24): python3-cephfs-17.2.7-904.gd8c6b0a3.el9 1.7 MB/s | 156 kB 00:00 2024-04-26T04:39:39.488 INFO:teuthology.orchestra.run.smithi077.stdout:(9/24): python3-rados-17.2.7-904.gd8c6b0a3.el9. 5.3 MB/s | 320 kB 00:00 2024-04-26T04:39:39.564 INFO:teuthology.orchestra.run.smithi077.stdout:(10/24): python3-rbd-17.2.7-904.gd8c6b0a3.el9.x 3.8 MB/s | 293 kB 00:00 2024-04-26T04:39:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:(1/24): ceph-common-17.2.7-904.gd8c6b0a3.el9.x8 8.8 MB/s | 22 MB 00:02 2024-04-26T04:39:39.630 INFO:teuthology.orchestra.run.smithi077.stdout:(11/24): python3-rgw-17.2.7-904.gd8c6b0a3.el9.x 1.5 MB/s | 100 kB 00:00 2024-04-26T04:39:39.658 INFO:teuthology.orchestra.run.smithi078.stdout:(2/24): ceph-selinux-17.2.7-904.gd8c6b0a3.el9.x 432 kB/s | 25 kB 00:00 2024-04-26T04:39:39.725 INFO:teuthology.orchestra.run.smithi078.stdout:(3/24): ceph-base-17.2.7-904.gd8c6b0a3.el9.x86_ 1.8 MB/s | 5.0 MB 00:02 2024-04-26T04:39:39.747 INFO:teuthology.orchestra.run.smithi077.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T04:39:39.781 INFO:teuthology.orchestra.run.smithi077.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-26T04:39:39.791 INFO:teuthology.orchestra.run.smithi078.stdout:(4/24): libcephfs2-17.2.7-904.gd8c6b0a3.el9.x86 4.7 MB/s | 637 kB 00:00 2024-04-26T04:39:39.856 INFO:teuthology.orchestra.run.smithi077.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-26T04:39:39.967 INFO:teuthology.orchestra.run.smithi078.stdout:(5/24): libradosstriper1-17.2.7-904.gd8c6b0a3.e 1.8 MB/s | 457 kB 00:00 2024-04-26T04:39:39.998 INFO:teuthology.orchestra.run.smithi077.stdout:(15/24): ceph-radosgw-17.2.7-904.gd8c6b0a3.el9. 4.7 MB/s | 14 MB 00:03 2024-04-26T04:39:40.023 INFO:teuthology.orchestra.run.smithi077.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-26T04:39:40.049 INFO:teuthology.orchestra.run.smithi077.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 855 kB/s | 42 kB 00:00 2024-04-26T04:39:40.076 INFO:teuthology.orchestra.run.smithi078.stdout:(6/24): python3-ceph-argparse-17.2.7-904.gd8c6b 416 kB/s | 45 kB 00:00 2024-04-26T04:39:40.218 INFO:teuthology.orchestra.run.smithi078.stdout:(7/24): librgw2-17.2.7-904.gd8c6b0a3.el9.x86_64 12 MB/s | 4.9 MB 00:00 2024-04-26T04:39:40.224 INFO:teuthology.orchestra.run.smithi077.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-04-26T04:39:40.234 INFO:teuthology.orchestra.run.smithi078.stdout:(8/24): python3-ceph-common-17.2.7-904.gd8c6b0a 742 kB/s | 117 kB 00:00 2024-04-26T04:39:40.249 INFO:teuthology.orchestra.run.smithi077.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.3 MB/s | 309 kB 00:00 2024-04-26T04:39:40.275 INFO:teuthology.orchestra.run.smithi077.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-26T04:39:40.276 INFO:teuthology.orchestra.run.smithi078.stdout:(9/24): python3-cephfs-17.2.7-904.gd8c6b0a3.el9 2.6 MB/s | 156 kB 00:00 2024-04-26T04:39:40.350 INFO:teuthology.orchestra.run.smithi077.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-26T04:39:40.360 INFO:teuthology.orchestra.run.smithi078.stdout:(10/24): python3-rbd-17.2.7-904.gd8c6b0a3.el9.x 3.4 MB/s | 293 kB 00:00 2024-04-26T04:39:40.385 INFO:teuthology.orchestra.run.smithi078.stdout:(11/24): python3-rados-17.2.7-904.gd8c6b0a3.el9 2.1 MB/s | 320 kB 00:00 2024-04-26T04:39:40.427 INFO:teuthology.orchestra.run.smithi078.stdout:(12/24): python3-rgw-17.2.7-904.gd8c6b0a3.el9.x 1.5 MB/s | 100 kB 00:00 2024-04-26T04:39:40.653 INFO:teuthology.orchestra.run.smithi078.stdout:(13/24): ceph-radosgw-17.2.7-904.gd8c6b0a3.el9. 3.9 MB/s | 14 MB 00:03 2024-04-26T04:39:40.703 INFO:teuthology.orchestra.run.smithi078.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 181 kB/s | 45 kB 00:00 2024-04-26T04:39:40.728 INFO:teuthology.orchestra.run.smithi078.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 468 kB/s | 160 kB 00:00 2024-04-26T04:39:40.783 INFO:teuthology.orchestra.run.smithi077.stdout:(22/24): librados2-17.2.7-904.gd8c6b0a3.el9.x86 6.1 MB/s | 3.1 MB 00:00 2024-04-26T04:39:40.795 INFO:teuthology.orchestra.run.smithi078.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-26T04:39:40.834 INFO:teuthology.orchestra.run.smithi077.stdout:(23/24): librbd1-17.2.7-904.gd8c6b0a3.el9.x86_6 6.1 MB/s | 2.9 MB 00:00 2024-04-26T04:39:40.837 INFO:teuthology.orchestra.run.smithi078.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-26T04:39:40.862 INFO:teuthology.orchestra.run.smithi078.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 316 kB/s | 42 kB 00:00 2024-04-26T04:39:41.146 INFO:teuthology.orchestra.run.smithi078.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 157 kB/s | 49 kB 00:00 2024-04-26T04:39:41.171 INFO:teuthology.orchestra.run.smithi078.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 218 kB/s | 67 kB 00:00 2024-04-26T04:39:41.305 INFO:teuthology.orchestra.run.smithi078.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 607 kB/s | 309 kB 00:00 2024-04-26T04:39:41.459 INFO:teuthology.orchestra.run.smithi077.stdout:(24/24): ceph-common-17.2.7-904.gd8c6b0a3.el9.x 5.0 MB/s | 22 MB 00:04 2024-04-26T04:39:41.464 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:39:41.464 INFO:teuthology.orchestra.run.smithi077.stdout:Total 12 MB/s | 58 MB 00:04 2024-04-26T04:39:41.522 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:39:41.553 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:39:41.553 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:39:41.605 INFO:teuthology.orchestra.run.smithi078.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-26T04:39:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:(23/24): librbd1-17.2.7-904.gd8c6b0a3.el9.x86_6 7.3 MB/s | 2.9 MB 00:00 2024-04-26T04:39:41.991 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:39:41.991 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:39:42.397 INFO:teuthology.orchestra.run.smithi078.stdout:(24/24): librados2-17.2.7-904.gd8c6b0a3.el9.x86 2.5 MB/s | 3.1 MB 00:01 2024-04-26T04:39:42.401 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:39:42.402 INFO:teuthology.orchestra.run.smithi078.stdout:Total 10 MB/s | 58 MB 00:05 2024-04-26T04:39:42.461 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:39:42.496 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:39:42.496 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:39:42.931 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:39:42.931 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:39:43.238 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:39:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-26T04:39:43.670 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:39:43.761 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:39:43.840 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/26 2024-04-26T04:39:43.871 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:39:43.959 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:39:44.006 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-26T04:39:44.035 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-26T04:39:44.183 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:39:44.469 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-26T04:39:44.500 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 8/26 2024-04-26T04:39:44.517 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-26T04:39:44.563 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 8/26 2024-04-26T04:39:44.600 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-26T04:39:44.617 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:39:44.642 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x 10/26 2024-04-26T04:39:44.680 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 11/26 2024-04-26T04:39:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:39:44.741 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 12/26 2024-04-26T04:39:44.769 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:39:44.788 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/26 2024-04-26T04:39:44.818 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:39:44.818 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:39:44.857 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-26T04:39:44.895 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-26T04:39:44.905 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:39:44.927 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-26T04:39:44.951 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-26T04:39:44.981 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-26T04:39:45.220 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-26T04:39:45.257 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 18/26 2024-04-26T04:39:45.339 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 18/26 2024-04-26T04:39:45.413 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 19/26 2024-04-26T04:39:45.413 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-26T04:39:45.443 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86 20/26 2024-04-26T04:39:45.446 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 8/26 2024-04-26T04:39:45.509 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 8/26 2024-04-26T04:39:45.538 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-26T04:39:45.581 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x 10/26 2024-04-26T04:39:45.617 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 11/26 2024-04-26T04:39:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 12/26 2024-04-26T04:39:45.707 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:39:45.772 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:39:45.812 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-26T04:39:45.841 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-26T04:39:45.873 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-26T04:39:46.165 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-26T04:39:46.203 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 18/26 2024-04-26T04:39:46.276 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 18/26 2024-04-26T04:39:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 19/26 2024-04-26T04:39:46.381 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86 20/26 2024-04-26T04:39:47.791 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:47.831 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:48.337 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:48.376 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 22/26 2024-04-26T04:39:48.466 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 22/26 2024-04-26T04:39:48.466 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T04:39:48.466 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T04:39:48.467 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:39:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:39:48.805 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:49.404 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 21/26 2024-04-26T04:39:49.466 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 22/26 2024-04-26T04:39:49.563 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 22/26 2024-04-26T04:39:49.563 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T04:39:49.564 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T04:39:49.564 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:39:49.592 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:39:57.964 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:39:57.964 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-04-26T04:39:57.964 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-04-26T04:39:57.965 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:39:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-04-26T04:39:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:39:59.196 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 24/26 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 24/26 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:39:59.274 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:39:59.278 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-26T04:39:59.357 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-26T04:39:59.357 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 24/26 2024-04-26T04:40:00.303 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 24/26 2024-04-26T04:40:00.303 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:00.303 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T04:40:00.303 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:40:00.304 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:40:00.304 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-26T04:40:00.389 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-26T04:40:00.389 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:00.670 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:00.670 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/26 2024-04-26T04:40:00.670 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:40:00.670 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 5/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 6/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 7/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x 8/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86 9/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 10/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 11/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 12/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-26T04:40:00.671 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-26T04:40:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 25/26 2024-04-26T04:40:01.374 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:01.374 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T04:40:01.375 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T04:40:01.376 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:01.377 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:01.627 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-test 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 5/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 6/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 7/26 2024-04-26T04:40:01.722 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x 8/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86 9/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 10/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 11/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 12/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 13/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-26T04:40:01.723 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-26T04:40:01.724 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-26T04:40:01.724 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 23/26 2024-04-26T04:40:01.724 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-26T04:40:01.724 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 25/26 2024-04-26T04:40:02.227 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:28 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:02.350 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 48 M 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 163 k 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:02.351 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:02.352 INFO:teuthology.orchestra.run.smithi077.stdout:Install 4 Packages 2024-04-26T04:40:02.352 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:02.352 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 48 M 2024-04-26T04:40:02.352 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 192 M 2024-04-26T04:40:02.352 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:02.429 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-26T04:40:02.429 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-04-26T04:40:02.429 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.429 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T04:40:02.430 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T04:40:02.431 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:02.590 INFO:teuthology.orchestra.run.smithi077.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-04-26T04:40:02.633 INFO:teuthology.orchestra.run.smithi077.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-26T04:40:02.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-test 2024-04-26T04:40:02.801 INFO:teuthology.orchestra.run.smithi077.stdout:(3/4): libcephsqlite-17.2.7-904.gd8c6b0a3.el9.x 433 kB/s | 163 kB 00:00 2024-04-26T04:40:03.234 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:29 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:03.355 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:03.355 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 48 M 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 163 k 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-04-26T04:40:03.356 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:03.357 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 48 M 2024-04-26T04:40:03.357 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 192 M 2024-04-26T04:40:03.357 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:03.642 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-26T04:40:03.707 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.6 MB/s | 64 kB 00:00 2024-04-26T04:40:03.987 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): libcephsqlite-17.2.7-904.gd8c6b0a3.el9.x 328 kB/s | 163 kB 00:00 2024-04-26T04:40:05.046 INFO:teuthology.orchestra.run.smithi077.stdout:(4/4): ceph-test-17.2.7-904.gd8c6b0a3.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-26T04:40:05.049 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:05.049 INFO:teuthology.orchestra.run.smithi077.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-26T04:40:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:05.095 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:05.095 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:05.460 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:05.460 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:05.972 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:06.038 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T04:40:06.078 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T04:40:06.106 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/4 2024-04-26T04:40:06.935 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): ceph-test-17.2.7-904.gd8c6b0a3.el9.x86_6 14 MB/s | 48 MB 00:03 2024-04-26T04:40:06.938 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:06.939 INFO:teuthology.orchestra.run.smithi078.stdout:Total 14 MB/s | 48 MB 00:03 2024-04-26T04:40:06.963 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:06.982 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:06.982 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:07.343 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:07.344 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:07.855 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:07.928 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T04:40:07.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T04:40:07.996 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/4 2024-04-26T04:40:10.465 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/4 2024-04-26T04:40:10.536 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/4 2024-04-26T04:40:12.369 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/4 2024-04-26T04:40:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/4 2024-04-26T04:40:12.720 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/4 2024-04-26T04:40:12.720 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/4 2024-04-26T04:40:12.720 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/4 2024-04-26T04:40:12.720 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T04:40:13.014 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:13.015 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:13.295 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph 2024-04-26T04:40:13.804 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:39 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:13.926 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 6.5 k 2024-04-26T04:40:13.928 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.1 M 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 1.4 M 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 240 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 3.9 M 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 15 M 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T04:40:13.929 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T04:40:13.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout:Install 33 Packages 2024-04-26T04:40:13.931 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:13.932 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 27 M 2024-04-26T04:40:13.932 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 96 M 2024-04-26T04:40:13.932 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:14.674 INFO:teuthology.orchestra.run.smithi077.stdout:(1/33): ceph-17.2.7-904.gd8c6b0a3.el9.x86_64.rp 20 kB/s | 6.5 kB 00:00 2024-04-26T04:40:14.742 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/4 2024-04-26T04:40:14.742 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/4 2024-04-26T04:40:14.742 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/4 2024-04-26T04:40:14.742 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T04:40:15.016 INFO:teuthology.orchestra.run.smithi077.stdout:(2/33): ceph-mgr-17.2.7-904.gd8c6b0a3.el9.x86_6 2.1 MB/s | 1.4 MB 00:00 2024-04-26T04:40:15.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T04:40:15.030 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:15.030 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:15.030 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:15.030 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:15.031 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T04:40:15.031 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T04:40:15.031 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:15.031 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:15.257 INFO:teuthology.orchestra.run.smithi077.stdout:(3/33): ceph-mds-17.2.7-904.gd8c6b0a3.el9.x86_6 2.3 MB/s | 2.1 MB 00:00 2024-04-26T04:40:15.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph 2024-04-26T04:40:15.358 INFO:teuthology.orchestra.run.smithi077.stdout:(4/33): ceph-mgr-modules-core-17.2.7-904.gd8c6b 2.3 MB/s | 240 kB 00:00 2024-04-26T04:40:15.507 INFO:teuthology.orchestra.run.smithi077.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-26T04:40:15.591 INFO:teuthology.orchestra.run.smithi077.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-04-26T04:40:15.641 INFO:teuthology.orchestra.run.smithi077.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-26T04:40:15.675 INFO:teuthology.orchestra.run.smithi077.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-04-26T04:40:15.709 INFO:teuthology.orchestra.run.smithi077.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 3.7 MB/s | 125 kB 00:00 2024-04-26T04:40:15.742 INFO:teuthology.orchestra.run.smithi077.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 6.3 MB/s | 215 kB 00:00 2024-04-26T04:40:15.792 INFO:teuthology.orchestra.run.smithi077.stdout:(11/33): ceph-mon-17.2.7-904.gd8c6b0a3.el9.x86_ 3.5 MB/s | 3.9 MB 00:01 2024-04-26T04:40:15.816 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:41 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:(12/33): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T04:40:15.843 INFO:teuthology.orchestra.run.smithi077.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 694 kB/s | 35 kB 00:00 2024-04-26T04:40:15.868 INFO:teuthology.orchestra.run.smithi077.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T04:40:15.894 INFO:teuthology.orchestra.run.smithi077.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-26T04:40:15.943 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:15.946 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:15.946 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:15.946 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:15.946 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:15.946 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 6.5 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.1 M 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 1.4 M 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 240 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 3.9 M 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 15 M 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T04:40:15.947 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T04:40:15.948 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout:Install 33 Packages 2024-04-26T04:40:15.949 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:15.950 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 27 M 2024-04-26T04:40:15.950 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 96 M 2024-04-26T04:40:15.950 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:16.052 INFO:teuthology.orchestra.run.smithi077.stdout:(16/33): ceph-osd-17.2.7-904.gd8c6b0a3.el9.x86_ 15 MB/s | 15 MB 00:01 2024-04-26T04:40:16.345 INFO:teuthology.orchestra.run.smithi077.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 91 kB/s | 43 kB 00:00 2024-04-26T04:40:16.461 INFO:teuthology.orchestra.run.smithi077.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 92 kB/s | 11 kB 00:00 2024-04-26T04:40:16.578 INFO:teuthology.orchestra.run.smithi077.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-04-26T04:40:16.620 INFO:teuthology.orchestra.run.smithi077.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 236 kB/s | 172 kB 00:00 2024-04-26T04:40:16.695 INFO:teuthology.orchestra.run.smithi077.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2024-04-26T04:40:16.729 INFO:teuthology.orchestra.run.smithi077.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-04-26T04:40:16.787 INFO:teuthology.orchestra.run.smithi078.stdout:(1/33): ceph-17.2.7-904.gd8c6b0a3.el9.x86_64.rp 19 kB/s | 6.5 kB 00:00 2024-04-26T04:40:16.804 INFO:teuthology.orchestra.run.smithi077.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-26T04:40:16.838 INFO:teuthology.orchestra.run.smithi077.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2024-04-26T04:40:17.063 INFO:teuthology.orchestra.run.smithi077.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 305 kB/s | 79 kB 00:00 2024-04-26T04:40:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:(2/33): ceph-mds-17.2.7-904.gd8c6b0a3.el9.x86_6 3.2 MB/s | 2.1 MB 00:00 2024-04-26T04:40:17.097 INFO:teuthology.orchestra.run.smithi077.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 343 kB/s | 358 kB 00:01 2024-04-26T04:40:17.180 INFO:teuthology.orchestra.run.smithi077.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-04-26T04:40:17.263 INFO:teuthology.orchestra.run.smithi078.stdout:(3/33): ceph-mgr-17.2.7-904.gd8c6b0a3.el9.x86_6 1.7 MB/s | 1.4 MB 00:00 2024-04-26T04:40:17.306 INFO:teuthology.orchestra.run.smithi077.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-04-26T04:40:17.331 INFO:teuthology.orchestra.run.smithi077.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 384 kB/s | 90 kB 00:00 2024-04-26T04:40:17.388 INFO:teuthology.orchestra.run.smithi078.stdout:(4/33): ceph-mgr-modules-core-17.2.7-904.gd8c6b 1.9 MB/s | 240 kB 00:00 2024-04-26T04:40:17.456 INFO:teuthology.orchestra.run.smithi077.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 440 kB/s | 272 kB 00:00 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi078.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-26T04:40:17.580 INFO:teuthology.orchestra.run.smithi078.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 16 MB/s | 1.2 MB 00:00 2024-04-26T04:40:17.598 INFO:teuthology.orchestra.run.smithi077.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 141 kB/s | 20 kB 00:00 2024-04-26T04:40:17.614 INFO:teuthology.orchestra.run.smithi078.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-26T04:40:17.648 INFO:teuthology.orchestra.run.smithi078.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-04-26T04:40:17.681 INFO:teuthology.orchestra.run.smithi078.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 3.6 MB/s | 125 kB 00:00 2024-04-26T04:40:17.715 INFO:teuthology.orchestra.run.smithi078.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 6.3 MB/s | 215 kB 00:00 2024-04-26T04:40:17.748 INFO:teuthology.orchestra.run.smithi078.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-04-26T04:40:17.782 INFO:teuthology.orchestra.run.smithi078.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 1.0 MB/s | 35 kB 00:00 2024-04-26T04:40:17.807 INFO:teuthology.orchestra.run.smithi077.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 459 kB/s | 230 kB 00:00 2024-04-26T04:40:17.941 INFO:teuthology.orchestra.run.smithi078.stdout:(13/33): ceph-osd-17.2.7-904.gd8c6b0a3.el9.x86_ 18 MB/s | 15 MB 00:00 2024-04-26T04:40:17.966 INFO:teuthology.orchestra.run.smithi078.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 278 kB/s | 51 kB 00:00 2024-04-26T04:40:17.991 INFO:teuthology.orchestra.run.smithi078.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-26T04:40:18.050 INFO:teuthology.orchestra.run.smithi078.stdout:(16/33): ceph-mon-17.2.7-904.gd8c6b0a3.el9.x86_ 3.1 MB/s | 3.9 MB 00:01 2024-04-26T04:40:18.207 INFO:teuthology.orchestra.run.smithi077.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 488 kB/s | 427 kB 00:00 2024-04-26T04:40:18.211 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:18.211 INFO:teuthology.orchestra.run.smithi077.stdout:Total 6.3 MB/s | 27 MB 00:04 2024-04-26T04:40:18.328 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:18.365 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:18.365 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:18.525 INFO:teuthology.orchestra.run.smithi078.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 78 kB/s | 43 kB 00:00 2024-04-26T04:40:18.632 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:18.633 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:18.642 INFO:teuthology.orchestra.run.smithi078.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 264 kB/s | 172 kB 00:00 2024-04-26T04:40:18.668 INFO:teuthology.orchestra.run.smithi078.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 75 kB/s | 11 kB 00:00 2024-04-26T04:40:18.768 INFO:teuthology.orchestra.run.smithi078.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-04-26T04:40:18.793 INFO:teuthology.orchestra.run.smithi078.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 185 kB/s | 23 kB 00:00 2024-04-26T04:40:18.877 INFO:teuthology.orchestra.run.smithi078.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-04-26T04:40:18.902 INFO:teuthology.orchestra.run.smithi078.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-26T04:40:18.986 INFO:teuthology.orchestra.run.smithi078.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 383 kB/s | 358 kB 00:00 2024-04-26T04:40:19.011 INFO:teuthology.orchestra.run.smithi078.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 345 kB/s | 46 kB 00:00 2024-04-26T04:40:19.086 INFO:teuthology.orchestra.run.smithi078.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 428 kB/s | 79 kB 00:00 2024-04-26T04:40:19.112 INFO:teuthology.orchestra.run.smithi078.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-04-26T04:40:19.229 INFO:teuthology.orchestra.run.smithi078.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 306 kB/s | 36 kB 00:00 2024-04-26T04:40:19.279 INFO:teuthology.orchestra.run.smithi078.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 468 kB/s | 90 kB 00:00 2024-04-26T04:40:19.429 INFO:teuthology.orchestra.run.smithi078.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 614 kB/s | 272 kB 00:00 2024-04-26T04:40:19.538 INFO:teuthology.orchestra.run.smithi078.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-04-26T04:40:19.580 INFO:teuthology.orchestra.run.smithi078.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 654 kB/s | 230 kB 00:00 2024-04-26T04:40:19.774 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:19.843 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-26T04:40:19.875 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-26T04:40:19.888 INFO:teuthology.orchestra.run.smithi078.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 701 kB/s | 427 kB 00:00 2024-04-26T04:40:19.895 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:19.895 INFO:teuthology.orchestra.run.smithi078.stdout:Total 6.9 MB/s | 27 MB 00:03 2024-04-26T04:40:19.908 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-26T04:40:19.999 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-26T04:40:20.010 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:20.046 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:20.120 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:20.121 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-26T04:40:20.121 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-26T04:40:20.156 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-26T04:40:20.184 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-26T04:40:20.219 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-26T04:40:20.252 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-26T04:40:20.293 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-26T04:40:20.314 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:20.314 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:20.320 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-26T04:40:20.354 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-26T04:40:20.392 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-26T04:40:20.465 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-26T04:40:20.602 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-26T04:40:20.683 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-26T04:40:20.736 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-26T04:40:20.775 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-26T04:40:20.823 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-26T04:40:20.879 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-26T04:40:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-26T04:40:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-26T04:40:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-26T04:40:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-26T04:40:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-26T04:40:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-26T04:40:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-26T04:40:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-26T04:40:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-26T04:40:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.n 28/33 2024-04-26T04:40:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 29/33 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 29/33 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:40:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-26T04:40:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-26T04:40:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-26T04:40:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-26T04:40:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-26T04:40:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-26T04:40:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-26T04:40:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-26T04:40:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-26T04:40:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-26T04:40:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-26T04:40:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-26T04:40:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-26T04:40:22.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-26T04:40:22.452 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-26T04:40:22.490 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-26T04:40:22.548 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-26T04:40:22.602 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-26T04:40:22.799 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-26T04:40:22.856 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-26T04:40:22.931 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-26T04:40:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 30/33 2024-04-26T04:40:23.019 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-26T04:40:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 30/33 2024-04-26T04:40:23.045 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:23.045 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T04:40:23.045 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:40:23.045 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:40:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-26T04:40:23.182 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-26T04:40:23.332 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.n 28/33 2024-04-26T04:40:23.362 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 29/33 2024-04-26T04:40:23.386 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 29/33 2024-04-26T04:40:23.386 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:23.386 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T04:40:23.387 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:40:23.387 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:40:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 31/33 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 31/33 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:40:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:23.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 32/33 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 32/33 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:40:23.676 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:23.740 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 33/33 2024-04-26T04:40:24.697 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 30/33 2024-04-26T04:40:24.722 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 30/33 2024-04-26T04:40:24.723 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:24.723 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T04:40:24.723 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:40:24.723 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:40:24.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:25.097 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 31/33 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 31/33 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:40:25.120 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:25.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 32/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 33/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 5/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.n 6/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-26T04:40:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-26T04:40:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-26T04:40:25.340 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 32/33 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:40:25.345 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:25.410 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 33/33 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T04:40:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T04:40:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T04:40:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T04:40:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:26.282 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:26.628 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-base 2024-04-26T04:40:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 33/33 2024-04-26T04:40:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 3/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 4/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 5/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.n 6/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-26T04:40:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-26T04:40:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-26T04:40:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-26T04:40:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-26T04:40:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-26T04:40:27.136 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:53 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:27.214 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:40:27.254 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:27.255 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:40:27.256 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:27.323 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install cephadm 2024-04-26T04:40:27.826 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:53 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T04:40:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T04:40:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T04:40:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T04:40:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T04:40:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T04:40:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T04:40:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:27.847 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:27.944 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 82 k 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:40:27.945 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:27.946 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 82 k 2024-04-26T04:40:27.946 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 372 k 2024-04-26T04:40:27.946 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:28.203 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-base 2024-04-26T04:40:28.486 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm-17.2.7-904.gd8c6b0a3.el9.noarch.rpm 152 kB/s | 82 kB 00:00 2024-04-26T04:40:28.486 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:28.487 INFO:teuthology.orchestra.run.smithi077.stdout:Total 151 kB/s | 82 kB 00:00 2024-04-26T04:40:28.487 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:28.491 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:28.492 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:28.494 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:28.494 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:28.570 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:28.715 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:54 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:28.793 INFO:teuthology.orchestra.run.smithi078.stdout:Package ceph-base-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:40:28.834 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:28.834 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:40:28.835 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:28.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install cephadm 2024-04-26T04:40:29.000 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:29.088 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:29.410 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:55 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:29.413 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:29.529 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:29.530 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:29.530 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:29.530 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 82 k 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 82 k 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 372 k 2024-04-26T04:40:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:29.649 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:29.867 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T04:40:30.047 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm-17.2.7-904.gd8c6b0a3.el9.noarch.rpm 159 kB/s | 82 kB 00:00 2024-04-26T04:40:30.048 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:30.048 INFO:teuthology.orchestra.run.smithi078.stdout:Total 158 kB/s | 82 kB 00:00 2024-04-26T04:40:30.048 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:30.053 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:30.053 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:30.055 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:30.055 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:30.135 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:30.385 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:56 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:30.507 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:30.508 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:30.508 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2024-04-26T04:40:30.508 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:30.508 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:30.508 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 137 k 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 137 k 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 379 k 2024-04-26T04:40:30.509 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:30.623 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:30.992 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-immutable-object-cache-17.2.7-904.gd8c6b0a 284 kB/s | 137 kB 00:00 2024-04-26T04:40:30.993 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:30.993 INFO:teuthology.orchestra.run.smithi077.stdout:Total 283 kB/s | 137 kB 00:00 2024-04-26T04:40:30.994 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:31.001 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:31.001 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:31.057 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:31.058 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/1 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:31.276 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:31.285 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:31.310 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:31.311 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:31.311 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T04:40:31.311 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:31.484 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T04:40:31.999 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:57 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:32.121 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:32.122 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:32.122 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 137 k 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 137 k 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 379 k 2024-04-26T04:40:32.123 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:32.180 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:32.181 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:32.181 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:32.181 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:32.181 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:32.181 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:32.368 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr 2024-04-26T04:40:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-immutable-object-cache-17.2.7-904.gd8c6b0a 243 kB/s | 137 kB 00:00 2024-04-26T04:40:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:Total 242 kB/s | 137 kB 00:00 2024-04-26T04:40:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:32.697 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:32.697 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:32.753 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:32.753 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:32.887 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:58 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:32.967 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:40:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:33.008 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:33.009 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:40:33.010 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:33.022 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:40:33.022 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T04:40:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:33.100 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T04:40:33.618 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:59 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:33.737 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 3.4 M 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 24 k 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 15 k 2024-04-26T04:40:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout:Install 7 Packages 2024-04-26T04:40:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:33.740 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 3.7 M 2024-04-26T04:40:33.740 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 80 M 2024-04-26T04:40:33.740 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.e 1/1 2024-04-26T04:40:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:33.843 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:40:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:33.843 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:34.056 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr 2024-04-26T04:40:34.272 INFO:teuthology.orchestra.run.smithi077.stdout:(1/7): ceph-prometheus-alerts-17.2.7-904.gd8c6b 39 kB/s | 15 kB 00:00 2024-04-26T04:40:34.372 INFO:teuthology.orchestra.run.smithi077.stdout:(2/7): ceph-grafana-dashboards-17.2.7-904.gd8c6 51 kB/s | 24 kB 00:00 2024-04-26T04:40:34.489 INFO:teuthology.orchestra.run.smithi077.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 42 kB/s | 9.0 kB 00:00 2024-04-26T04:40:34.567 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:00 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:34.622 INFO:teuthology.orchestra.run.smithi077.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 232 kB/s | 31 kB 00:00 2024-04-26T04:40:34.646 INFO:teuthology.orchestra.run.smithi078.stdout:Package ceph-mgr-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:40:34.648 INFO:teuthology.orchestra.run.smithi077.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 148 kB/s | 41 kB 00:00 2024-04-26T04:40:34.687 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:34.688 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:40:34.688 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:34.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T04:40:34.815 INFO:teuthology.orchestra.run.smithi077.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 982 kB/s | 188 kB 00:00 2024-04-26T04:40:35.015 INFO:teuthology.orchestra.run.smithi077.stdout:(7/7): ceph-mgr-dashboard-17.2.7-904.gd8c6b0a3. 3.0 MB/s | 3.4 MB 00:01 2024-04-26T04:40:35.019 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:35.019 INFO:teuthology.orchestra.run.smithi077.stdout:Total 2.9 MB/s | 3.7 MB 00:01 2024-04-26T04:40:35.019 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:35.027 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:35.027 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:35.284 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:01 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:35.384 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:35.405 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:35.405 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:35.405 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:35.405 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 3.4 M 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 24 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 15 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:35.406 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:35.407 INFO:teuthology.orchestra.run.smithi078.stdout:Install 7 Packages 2024-04-26T04:40:35.407 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:35.407 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 3.7 M 2024-04-26T04:40:35.407 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 80 M 2024-04-26T04:40:35.407 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:35.450 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-26T04:40:35.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-26T04:40:35.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-26T04:40:35.562 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-26T04:40:35.591 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.no 5/7 2024-04-26T04:40:36.002 INFO:teuthology.orchestra.run.smithi078.stdout:(1/7): ceph-prometheus-alerts-17.2.7-904.gd8c6b 33 kB/s | 15 kB 00:00 2024-04-26T04:40:36.060 INFO:teuthology.orchestra.run.smithi078.stdout:(2/7): ceph-grafana-dashboards-17.2.7-904.gd8c6 49 kB/s | 24 kB 00:00 2024-04-26T04:40:36.302 INFO:teuthology.orchestra.run.smithi078.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 30 kB/s | 9.0 kB 00:00 2024-04-26T04:40:36.477 INFO:teuthology.orchestra.run.smithi078.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 98 kB/s | 41 kB 00:00 2024-04-26T04:40:36.503 INFO:teuthology.orchestra.run.smithi078.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 153 kB/s | 31 kB 00:00 2024-04-26T04:40:36.583 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.n 6/7 2024-04-26T04:40:36.620 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 7/7 2024-04-26T04:40:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 628 kB/s | 188 kB 00:00 2024-04-26T04:40:36.836 INFO:teuthology.orchestra.run.smithi078.stdout:(7/7): ceph-mgr-dashboard-17.2.7-904.gd8c6b0a3. 2.6 MB/s | 3.4 MB 00:01 2024-04-26T04:40:36.840 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:36.840 INFO:teuthology.orchestra.run.smithi078.stdout:Total 2.5 MB/s | 3.7 MB 00:01 2024-04-26T04:40:36.840 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:36.848 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:36.849 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:36.910 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:36.912 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:37.209 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:37.296 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-26T04:40:37.330 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-26T04:40:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-26T04:40:37.426 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-26T04:40:37.454 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.no 5/7 2024-04-26T04:40:37.676 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 7/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.n 1/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 2/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.no 3/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-26T04:40:37.677 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:37.930 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-26T04:40:37.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-26T04:40:37.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T04:40:37.931 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T04:40:37.931 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:37.931 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:38.116 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T04:40:38.451 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.n 6/7 2024-04-26T04:40:38.491 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 7/7 2024-04-26T04:40:38.625 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:04 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:38.747 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 7.4 M 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T04:40:38.748 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout:Install 12 Packages 2024-04-26T04:40:38.749 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:38.750 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 38 M 2024-04-26T04:40:38.750 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 209 M 2024-04-26T04:40:38.750 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:(1/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.8 MB/s | 803 kB 00:00 2024-04-26T04:40:39.332 INFO:teuthology.orchestra.run.smithi077.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-26T04:40:39.357 INFO:teuthology.orchestra.run.smithi077.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-26T04:40:39.424 INFO:teuthology.orchestra.run.smithi077.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2024-04-26T04:40:39.449 INFO:teuthology.orchestra.run.smithi077.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 204 kB/s | 18 kB 00:00 2024-04-26T04:40:39.475 INFO:teuthology.orchestra.run.smithi077.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-26T04:40:39.559 INFO:teuthology.orchestra.run.smithi077.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 44 MB/s | 4.7 MB 00:00 2024-04-26T04:40:39.584 INFO:teuthology.orchestra.run.smithi077.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-26T04:40:39.602 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 7/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.n 1/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 2/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.no 3/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-26T04:40:39.603 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-26T04:40:39.684 INFO:teuthology.orchestra.run.smithi077.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-04-26T04:40:39.709 INFO:teuthology.orchestra.run.smithi077.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T04:40:39.849 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T04:40:39.850 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:39.850 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:40.001 INFO:teuthology.orchestra.run.smithi077.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-04-26T04:40:40.082 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T04:40:40.510 INFO:teuthology.orchestra.run.smithi077.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-9 5.4 MB/s | 7.4 MB 00:01 2024-04-26T04:40:40.512 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:40.512 INFO:teuthology.orchestra.run.smithi077.stdout:Total 22 MB/s | 38 MB 00:01 2024-04-26T04:40:40.594 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:06 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:40.715 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:40.716 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-04-26T04:40:40.716 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:40.716 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-04-26T04:40:40.716 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 7.4 M 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T04:40:40.717 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout:Install 12 Packages 2024-04-26T04:40:40.718 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:40.719 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 38 M 2024-04-26T04:40:40.719 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 209 M 2024-04-26T04:40:40.719 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:40.862 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:40.889 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:40.889 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:41.153 INFO:teuthology.orchestra.run.smithi078.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-26T04:40:41.187 INFO:teuthology.orchestra.run.smithi078.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-26T04:40:41.212 INFO:teuthology.orchestra.run.smithi078.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 33 kB 00:00 2024-04-26T04:40:41.246 INFO:teuthology.orchestra.run.smithi078.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 569 kB/s | 18 kB 00:00 2024-04-26T04:40:41.296 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:41.296 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:41.314 INFO:teuthology.orchestra.run.smithi078.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-04-26T04:40:41.338 INFO:teuthology.orchestra.run.smithi078.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 389 kB/s | 35 kB 00:00 2024-04-26T04:40:41.397 INFO:teuthology.orchestra.run.smithi078.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-04-26T04:40:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-04-26T04:40:41.598 INFO:teuthology.orchestra.run.smithi078.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-04-26T04:40:41.622 INFO:teuthology.orchestra.run.smithi078.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-04-26T04:40:42.015 INFO:teuthology.orchestra.run.smithi078.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-26T04:40:42.041 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:42.114 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T04:40:42.131 INFO:teuthology.orchestra.run.smithi078.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-9 6.5 MB/s | 7.4 MB 00:01 2024-04-26T04:40:42.134 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:42.135 INFO:teuthology.orchestra.run.smithi078.stdout:Total 27 MB/s | 38 MB 00:01 2024-04-26T04:40:42.158 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:40:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:40:42.292 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T04:40:42.470 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T04:40:42.479 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:42.505 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:42.505 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:42.516 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T04:40:42.602 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T04:40:42.908 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:42.908 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:43.246 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T04:40:43.355 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T04:40:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:43.728 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T04:40:43.771 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:40:43.872 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:40:43.923 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T04:40:44.100 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T04:40:44.130 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T04:40:44.207 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T04:40:44.583 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:40:44.860 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T04:40:44.969 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T04:40:45.613 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T04:40:45.642 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 12/12 2024-04-26T04:40:46.197 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:40:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T04:40:47.264 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 12/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 12/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 1/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T04:40:48.183 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T04:40:48.184 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:40:48.184 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T04:40:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T04:40:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T04:40:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T04:40:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T04:40:48.566 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:48.566 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:48.917 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-rook 2024-04-26T04:40:49.439 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:15 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:49.560 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 47 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T04:40:49.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout:Install 13 Packages 2024-04-26T04:40:49.563 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:49.564 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 2.2 M 2024-04-26T04:40:49.564 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 26 M 2024-04-26T04:40:49.564 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:49.797 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 12/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0 1/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T04:40:49.798 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:40:49.799 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T04:40:49.918 INFO:teuthology.orchestra.run.smithi077.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 211 kB/s | 26 kB 00:00 2024-04-26T04:40:49.943 INFO:teuthology.orchestra.run.smithi077.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 131 kB/s | 19 kB 00:00 2024-04-26T04:40:49.993 INFO:teuthology.orchestra.run.smithi077.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-04-26T04:40:50.019 INFO:teuthology.orchestra.run.smithi077.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T04:40:50.044 INFO:teuthology.orchestra.run.smithi077.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.4 MB/s | 279 kB 00:00 2024-04-26T04:40:50.069 INFO:teuthology.orchestra.run.smithi077.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-26T04:40:50.094 INFO:teuthology.orchestra.run.smithi077.stdout:(7/13): ceph-mgr-rook-17.2.7-904.gd8c6b0a3.el9. 159 kB/s | 47 kB 00:00 2024-04-26T04:40:50.162 INFO:teuthology.orchestra.run.smithi077.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 163 kB/s | 15 kB 00:00 2024-04-26T04:40:50.162 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T04:40:50.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T04:40:50.163 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T04:40:50.164 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:50.164 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:50.178 INFO:teuthology.orchestra.run.smithi077.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-04-26T04:40:50.262 INFO:teuthology.orchestra.run.smithi077.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-04-26T04:40:50.287 INFO:teuthology.orchestra.run.smithi077.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 1.0 MB/s | 206 kB 00:00 2024-04-26T04:40:50.330 INFO:teuthology.orchestra.run.smithi077.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-26T04:40:50.438 INFO:teuthology.orchestra.run.smithi077.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.7 MB/s | 1.0 MB 00:00 2024-04-26T04:40:50.440 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:50.440 INFO:teuthology.orchestra.run.smithi077.stdout:Total 2.5 MB/s | 2.2 MB 00:00 2024-04-26T04:40:50.502 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:50.517 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:50.517 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:50.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-rook 2024-04-26T04:40:50.599 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:50.600 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:51.076 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:17 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:51.193 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:51.198 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:51.199 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:51.200 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:51.200 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:51.200 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:51.200 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 47 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T04:40:51.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:51.202 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:51.203 INFO:teuthology.orchestra.run.smithi078.stdout:Install 13 Packages 2024-04-26T04:40:51.205 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:51.206 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 2.2 M 2024-04-26T04:40:51.206 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 26 M 2024-04-26T04:40:51.206 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:51.347 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T04:40:51.400 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T04:40:51.462 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T04:40:51.499 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T04:40:51.541 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T04:40:51.636 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T04:40:51.746 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T04:40:51.756 INFO:teuthology.orchestra.run.smithi078.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 211 kB/s | 26 kB 00:00 2024-04-26T04:40:51.773 INFO:teuthology.orchestra.run.smithi078.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 138 kB/s | 19 kB 00:00 2024-04-26T04:40:51.809 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T04:40:51.824 INFO:teuthology.orchestra.run.smithi078.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.3 MB/s | 222 kB 00:00 2024-04-26T04:40:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T04:40:51.874 INFO:teuthology.orchestra.run.smithi078.stdout:(5/13): ceph-mgr-rook-17.2.7-904.gd8c6b0a3.el9. 198 kB/s | 47 kB 00:00 2024-04-26T04:40:51.907 INFO:teuthology.orchestra.run.smithi078.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-04-26T04:40:51.932 INFO:teuthology.orchestra.run.smithi078.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 641 kB/s | 54 kB 00:00 2024-04-26T04:40:52.375 INFO:teuthology.orchestra.run.smithi078.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 64 kB/s | 32 kB 00:00 2024-04-26T04:40:52.399 INFO:teuthology.orchestra.run.smithi078.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 32 kB/s | 15 kB 00:00 2024-04-26T04:40:52.544 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T04:40:52.590 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T04:40:52.624 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T04:40:52.658 INFO:teuthology.orchestra.run.smithi078.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-04-26T04:40:52.669 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T04:40:52.694 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 13/13 2024-04-26T04:40:52.742 INFO:teuthology.orchestra.run.smithi078.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 254 kB/s | 206 kB 00:00 2024-04-26T04:40:52.850 INFO:teuthology.orchestra.run.smithi078.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 467 kB/s | 90 kB 00:00 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 13/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T04:40:53.431 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T04:40:53.432 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T04:40:53.432 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T04:40:53.432 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T04:40:53.754 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T04:40:53.754 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T04:40:53.755 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T04:40:53.756 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:53.756 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:53.784 INFO:teuthology.orchestra.run.smithi078.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 742 kB/s | 1.0 MB 00:01 2024-04-26T04:40:53.790 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:53.790 INFO:teuthology.orchestra.run.smithi078.stdout:Total 870 kB/s | 2.2 MB 00:02 2024-04-26T04:40:53.856 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:53.872 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:53.872 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:53.957 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:53.959 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:54.006 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T04:40:54.533 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:20 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:54.556 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:40:54.659 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 123 k 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T04:40:54.660 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout:Install 6 Packages 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 7.0 M 2024-04-26T04:40:54.661 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 33 M 2024-04-26T04:40:54.662 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:54.766 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T04:40:54.819 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T04:40:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T04:40:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T04:40:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T04:40:55.047 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T04:40:55.117 INFO:teuthology.orchestra.run.smithi077.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-26T04:40:55.156 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T04:40:55.193 INFO:teuthology.orchestra.run.smithi077.stdout:(2/6): ceph-mgr-cephadm-17.2.7-904.gd8c6b0a3.el 488 kB/s | 123 kB 00:00 2024-04-26T04:40:55.205 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T04:40:55.276 INFO:teuthology.orchestra.run.smithi077.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-26T04:40:55.410 INFO:teuthology.orchestra.run.smithi077.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 268 kB/s | 58 kB 00:00 2024-04-26T04:40:55.435 INFO:teuthology.orchestra.run.smithi077.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 322 kB/s | 51 kB 00:00 2024-04-26T04:40:55.460 INFO:teuthology.orchestra.run.smithi077.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2024-04-26T04:40:55.462 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:55.462 INFO:teuthology.orchestra.run.smithi077.stdout:Total 8.7 MB/s | 7.0 MB 00:00 2024-04-26T04:40:55.532 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:55.543 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:55.543 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:55.639 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:55.639 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:55.923 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T04:40:55.940 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:55.977 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T04:40:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T04:40:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T04:40:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 13/13 2024-04-26T04:40:56.108 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T04:40:56.153 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T04:40:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T04:40:56.748 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T04:40:56.805 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T04:40:56.833 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 6/6 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 13/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T04:40:56.879 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T04:40:56.880 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T04:40:56.880 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T04:40:56.880 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T04:40:56.880 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T04:40:57.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:57.198 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:40:57.433 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T04:40:57.586 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 6/6 2024-04-26T04:40:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/6 2024-04-26T04:40:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T04:40:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T04:40:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T04:40:57.587 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T04:40:57.853 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T04:40:57.853 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:57.853 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:57.854 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:40:57.940 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:23 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:58.059 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:17.2.7-904.gd8c6b0a3.el9 ceph-noarch 123 k 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T04:40:58.060 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout:Install 6 Packages 2024-04-26T04:40:58.061 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:40:58.062 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 7.0 M 2024-04-26T04:40:58.062 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 33 M 2024-04-26T04:40:58.062 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:40:58.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-fuse 2024-04-26T04:40:58.611 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:24 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:40:58.729 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 812 k 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout:Install 2 Packages 2024-04-26T04:40:58.730 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:40:58.731 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 892 k 2024-04-26T04:40:58.731 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 2.6 M 2024-04-26T04:40:58.731 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:40:58.837 INFO:teuthology.orchestra.run.smithi078.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-04-26T04:40:59.000 INFO:teuthology.orchestra.run.smithi077.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 521 kB/s | 80 kB 00:00 2024-04-26T04:40:59.020 INFO:teuthology.orchestra.run.smithi078.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-26T04:40:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:(3/6): ceph-mgr-cephadm-17.2.7-904.gd8c6b0a3.el 282 kB/s | 123 kB 00:00 2024-04-26T04:40:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 188 kB/s | 58 kB 00:00 2024-04-26T04:40:59.354 INFO:teuthology.orchestra.run.smithi077.stdout:(2/2): ceph-fuse-17.2.7-904.gd8c6b0a3.el9.x86_6 1.6 MB/s | 812 kB 00:00 2024-04-26T04:40:59.355 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:59.356 INFO:teuthology.orchestra.run.smithi077.stdout:Total 1.4 MB/s | 892 kB 00:00 2024-04-26T04:40:59.367 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:40:59.371 INFO:teuthology.orchestra.run.smithi078.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-04-26T04:40:59.389 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:40:59.389 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:40:59.421 INFO:teuthology.orchestra.run.smithi078.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 170 kB/s | 51 kB 00:00 2024-04-26T04:40:59.425 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:40:59.425 INFO:teuthology.orchestra.run.smithi078.stdout:Total 5.1 MB/s | 7.0 MB 00:01 2024-04-26T04:40:59.466 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:40:59.467 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:40:59.495 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:40:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:40:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:40:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:40:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:40:59.647 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:40:59.783 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T04:40:59.843 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/2 2024-04-26T04:40:59.902 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:00.077 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T04:41:00.140 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T04:41:00.476 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/2 2024-04-26T04:41:00.476 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/2 2024-04-26T04:41:00.653 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T04:41:00.734 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:00.746 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T04:41:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 6/6 2024-04-26T04:41:00.990 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados-devel 2024-04-26T04:41:01.509 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:27 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 6/6 2024-04-26T04:41:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 1/6 2024-04-26T04:41:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T04:41:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T04:41:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T04:41:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T04:41:01.629 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 126 k 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:41:01.630 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:01.631 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 126 k 2024-04-26T04:41:01.631 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 448 k 2024-04-26T04:41:01.631 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:41:01.805 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T04:41:01.805 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:01.805 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:01.805 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:17.2.7-904.gd8c6b0a3.el9.noarch 2024-04-26T04:41:01.805 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:01.806 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:01.843 INFO:teuthology.orchestra.run.smithi077.stdout:librados-devel-17.2.7-904.gd8c6b0a3.el9.x86_64. 595 kB/s | 126 kB 00:00 2024-04-26T04:41:01.844 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:01.844 INFO:teuthology.orchestra.run.smithi077.stdout:Total 591 kB/s | 126 kB 00:00 2024-04-26T04:41:01.844 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:41:01.850 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:41:01.850 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:41:01.912 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:41:01.912 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:41:02.069 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:41:02.078 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-fuse 2024-04-26T04:41:02.206 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:02.581 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:02.606 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:28 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:02.726 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 812 k 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout:Install 2 Packages 2024-04-26T04:41:02.727 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:02.728 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 892 k 2024-04-26T04:41:02.728 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 2.6 M 2024-04-26T04:41:02.728 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:02.845 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:03.103 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs2 2024-04-26T04:41:03.241 INFO:teuthology.orchestra.run.smithi078.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 550 kB/s | 80 kB 00:00 2024-04-26T04:41:03.591 INFO:teuthology.orchestra.run.smithi078.stdout:(2/2): ceph-fuse-17.2.7-904.gd8c6b0a3.el9.x86_6 1.6 MB/s | 812 kB 00:00 2024-04-26T04:41:03.592 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:03.593 INFO:teuthology.orchestra.run.smithi078.stdout:Total 1.0 MB/s | 892 kB 00:00 2024-04-26T04:41:03.604 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:03.620 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:29 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:03.626 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:03.627 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:03.700 INFO:teuthology.orchestra.run.smithi077.stdout:Package libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:03.702 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:03.702 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:03.741 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:03.742 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:03.742 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:03.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs-devel 2024-04-26T04:41:03.880 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:04.012 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T04:41:04.080 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/2 2024-04-26T04:41:04.345 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:30 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:04.463 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:04.463 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:04.463 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 25 k 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 25 k 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 110 k 2024-04-26T04:41:04.464 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:41:04.622 INFO:teuthology.orchestra.run.smithi077.stdout:libcephfs-devel-17.2.7-904.gd8c6b0a3.el9.x86_64 158 kB/s | 25 kB 00:00 2024-04-26T04:41:04.623 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:04.623 INFO:teuthology.orchestra.run.smithi077.stdout:Total 157 kB/s | 25 kB 00:00 2024-04-26T04:41:04.623 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:41:04.626 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:41:04.626 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:41:04.628 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:41:04.628 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:41:04.715 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:41:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2/2 2024-04-26T04:41:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/2 2024-04-26T04:41:04.827 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:04.979 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:05.149 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:05.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librados-devel 2024-04-26T04:41:05.368 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:05.369 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:05.369 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:05.369 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:05.369 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:05.369 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:05.547 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados2 2024-04-26T04:41:05.760 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:31 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:05.881 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 126 k 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:05.882 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:05.883 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:41:05.883 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:05.883 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 126 k 2024-04-26T04:41:05.883 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 448 k 2024-04-26T04:41:05.883 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:06.062 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:06.107 INFO:teuthology.orchestra.run.smithi078.stdout:librados-devel-17.2.7-904.gd8c6b0a3.el9.x86_64. 565 kB/s | 126 kB 00:00 2024-04-26T04:41:06.107 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:06.108 INFO:teuthology.orchestra.run.smithi078.stdout:Total 561 kB/s | 126 kB 00:00 2024-04-26T04:41:06.108 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:06.114 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:06.114 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:06.140 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:06.176 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:06.176 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:06.179 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:06.180 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:06.180 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:06.245 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librbd1 2024-04-26T04:41:06.325 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:06.438 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:06.751 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:06.796 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:06.831 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:06.872 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:06.873 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:06.873 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:06.948 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rados 2024-04-26T04:41:07.039 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:07.039 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:07.039 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:07.040 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:07.040 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:07.040 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:07.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install libcephfs2 2024-04-26T04:41:07.450 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:33 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:07.529 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:07.570 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:07.571 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:07.571 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:07.636 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rgw 2024-04-26T04:41:07.815 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:33 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:07.895 INFO:teuthology.orchestra.run.smithi078.stdout:Package libcephfs2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:07.936 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:07.937 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:07.937 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:08.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install libcephfs-devel 2024-04-26T04:41:08.144 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:34 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:08.224 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:08.264 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:08.265 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:08.265 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:08.339 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-cephfs 2024-04-26T04:41:08.546 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:34 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:08.664 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 25 k 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 25 k 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 110 k 2024-04-26T04:41:08.665 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:08.845 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:34 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:08.921 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:08.963 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:08.964 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:08.964 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:09.002 INFO:teuthology.orchestra.run.smithi078.stdout:libcephfs-devel-17.2.7-904.gd8c6b0a3.el9.x86_64 74 kB/s | 25 kB 00:00 2024-04-26T04:41:09.002 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:09.002 INFO:teuthology.orchestra.run.smithi078.stdout:Total 74 kB/s | 25 kB 00:00 2024-04-26T04:41:09.002 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:09.006 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:09.006 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:09.007 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:09.008 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:09.038 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rbd 2024-04-26T04:41:09.094 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:09.205 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:09.547 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:35 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:09.550 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:09.626 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:09.667 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:09.668 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-04-26T04:41:09.668 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:09.735 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-fuse 2024-04-26T04:41:09.770 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:09.770 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:09.770 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:09.771 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:09.771 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:09.771 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:09.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librados2 2024-04-26T04:41:10.244 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:36 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:10.363 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 82 k 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 82 k 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 222 k 2024-04-26T04:41:10.364 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:41:10.450 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:36 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:10.529 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:10.569 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:10.570 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:10.570 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:10.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librbd1 2024-04-26T04:41:11.002 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-fuse-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 128 kB/s | 82 kB 00:00 2024-04-26T04:41:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:Total 128 kB/s | 82 kB 00:00 2024-04-26T04:41:11.003 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:41:11.010 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:41:11.010 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:41:11.073 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:41:11.073 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:41:11.154 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:37 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:11.219 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:41:11.234 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:11.274 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:11.275 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:11.275 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:11.306 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:11.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rados 2024-04-26T04:41:11.630 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:11.865 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:37 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:11.876 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:11.876 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:11.876 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:11.877 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:11.877 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:11.877 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:11.951 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rados-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:11.993 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:11.994 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:11.994 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:12.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rgw 2024-04-26T04:41:12.110 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-mirror 2024-04-26T04:41:12.579 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:38 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:12.623 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:38 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:12.657 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rgw-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:12.697 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:12.698 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:12.698 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:12.744 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:12.744 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.9 M 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:12.745 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:41:12.746 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:12.746 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 2.9 M 2024-04-26T04:41:12.746 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 12 M 2024-04-26T04:41:12.746 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:41:12.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-cephfs 2024-04-26T04:41:13.284 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:39 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:13.362 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-cephfs-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:13.402 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:13.403 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:13.403 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:13.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rbd 2024-04-26T04:41:13.481 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-mirror-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 4.0 MB/s | 2.9 MB 00:00 2024-04-26T04:41:13.481 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:13.481 INFO:teuthology.orchestra.run.smithi077.stdout:Total 4.0 MB/s | 2.9 MB 00:00 2024-04-26T04:41:13.481 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:41:13.490 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:41:13.490 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:41:13.570 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:41:13.570 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:41:13.978 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:41:13.985 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:39 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:14.039 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:41:14.063 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:14.067 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 is already installed. 2024-04-26T04:41:14.107 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:14.108 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-26T04:41:14.108 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:14.177 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-fuse 2024-04-26T04:41:14.681 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:40 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 82 k 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:14.803 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 82 k 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 222 k 2024-04-26T04:41:14.804 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:14.993 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:15.265 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-nbd 2024-04-26T04:41:15.296 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-fuse-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 165 kB/s | 82 kB 00:00 2024-04-26T04:41:15.297 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:15.297 INFO:teuthology.orchestra.run.smithi078.stdout:Total 165 kB/s | 82 kB 00:00 2024-04-26T04:41:15.297 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:15.304 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:15.304 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:15.366 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:15.366 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:15.517 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:15.626 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:15.786 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:41 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:15.909 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 165 k 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:15.910 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 165 k 2024-04-26T04:41:15.911 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 477 k 2024-04-26T04:41:15.911 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-04-26T04:41:15.942 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:16.191 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:16.191 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:16.191 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:16.191 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:16.192 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:16.192 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:16.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-mirror 2024-04-26T04:41:16.441 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-nbd-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 311 kB/s | 165 kB 00:00 2024-04-26T04:41:16.442 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:16.442 INFO:teuthology.orchestra.run.smithi077.stdout:Total 310 kB/s | 165 kB 00:00 2024-04-26T04:41:16.442 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-04-26T04:41:16.451 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-04-26T04:41:16.451 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-04-26T04:41:16.514 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-04-26T04:41:16.514 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-04-26T04:41:16.669 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-04-26T04:41:16.769 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:16.924 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:42 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:17.045 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 2.9 M 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 2.9 M 2024-04-26T04:41:17.046 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 12 M 2024-04-26T04:41:17.047 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:17.139 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:41:17.392 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-04-26T04:41:17.574 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-mirror-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 5.6 MB/s | 2.9 MB 00:00 2024-04-26T04:41:17.575 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:17.575 INFO:teuthology.orchestra.run.smithi078.stdout:Total 5.6 MB/s | 2.9 MB 00:00 2024-04-26T04:41:17.575 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:17.584 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:17.584 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:17.611 DEBUG:teuthology.parallel:result is None 2024-04-26T04:41:17.661 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:17.661 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:18.069 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:18.125 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:41:18.149 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:19.199 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:19.199 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:19.200 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:19.200 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:19.443 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-nbd 2024-04-26T04:41:19.965 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:45 ago on Fri 26 Apr 2024 04:39:34 AM UTC. 2024-04-26T04:41:20.088 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-26T04:41:20.089 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:17.2.7-904.gd8c6b0a3.el9 ceph 165 k 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-26T04:41:20.090 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:20.091 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 165 k 2024-04-26T04:41:20.091 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 477 k 2024-04-26T04:41:20.091 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-26T04:41:20.436 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-nbd-17.2.7-904.gd8c6b0a3.el9.x86_64.rpm 477 kB/s | 165 kB 00:00 2024-04-26T04:41:20.436 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:41:20.437 INFO:teuthology.orchestra.run.smithi078.stdout:Total 475 kB/s | 165 kB 00:00 2024-04-26T04:41:20.437 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-26T04:41:20.444 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-26T04:41:20.444 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-26T04:41:20.505 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-26T04:41:20.506 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-26T04:41:20.659 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-26T04:41:20.765 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 1/1 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:17.2.7-904.gd8c6b0a3.el9.x86_64 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:41:21.385 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-26T04:41:21.657 DEBUG:teuthology.parallel:result is None 2024-04-26T04:41:21.657 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:41:21.657 INFO:teuthology.packaging:ref: None 2024-04-26T04:41:21.657 INFO:teuthology.packaging:tag: None 2024-04-26T04:41:21.657 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:41:21.658 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:21.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-26T04:41:21.918 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T04:41:21.948 INFO:teuthology.orchestra.run.smithi077.stdout:17.2.7-904.gd8c6b0a3.el9 2024-04-26T04:41:21.948 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-904.gd8c6b0a3.el9 2024-04-26T04:41:21.949 INFO:teuthology.task.install:The correct ceph version 17.2.7-904.gd8c6b0a3 is installed. 2024-04-26T04:41:21.952 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:41:21.952 INFO:teuthology.packaging:ref: None 2024-04-26T04:41:21.952 INFO:teuthology.packaging:tag: None 2024-04-26T04:41:21.952 INFO:teuthology.packaging:branch: quincy 2024-04-26T04:41:21.952 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:21.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-26T04:41:22.235 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T04:41:22.256 INFO:teuthology.orchestra.run.smithi078.stdout:17.2.7-904.gd8c6b0a3.el9 2024-04-26T04:41:22.256 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-904.gd8c6b0a3.el9 2024-04-26T04:41:22.257 INFO:teuthology.task.install:The correct ceph version 17.2.7-904.gd8c6b0a3 is installed. 2024-04-26T04:41:22.258 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T04:41:22.259 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:41:22.259 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:41:22.296 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:41:22.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:41:22.326 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T04:41:22.326 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:41:22.327 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:41:22.376 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:41:22.446 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:41:22.446 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:41:22.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:41:22.549 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T04:41:22.549 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:41:22.550 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:41:22.593 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:41:22.667 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:41:22.668 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:41:22.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:41:22.758 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T04:41:22.759 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:41:22.762 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:41:22.785 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:41:22.853 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:41:22.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:41:22.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:41:22.942 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:41:23.350 INFO:teuthology.task.print:**** done install task... 2024-04-26T04:41:23.350 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T04:41:23.589 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'} 2024-04-26T04:41:23.589 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-26T04:41:23.590 INFO:tasks.cephadm:Cluster fsid is 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:41:23.590 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T04:41:23.590 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-26T04:41:23.590 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi077': '172.21.15.77', 'mon.smithi078': '172.21.15.78'} 2024-04-26T04:41:23.590 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T04:41:23.590 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-04-26T04:41:23.614 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-04-26T04:41:23.639 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-26T04:41:23.640 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:23.818 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 26 04:41 /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:23.818 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 26 04:41 /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:23.996 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:24.011 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T04:41:24.038 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-26T04:41:24.038 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-26T04:41:24.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-26T04:41:25.512 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-26T04:41:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-26T04:42:38.047 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-26T04:42:38.048 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-26T04:42:50.956 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-04-26T04:42:50.957 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:42:50.974 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-04-26T04:42:51.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-04-26T04:42:51.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-04-26T04:42:51.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-04-26T04:42:51.119 INFO:tasks.cephadm:Writing seed config... 2024-04-26T04:42:51.120 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T04:42:51.121 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T04:42:51.122 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T04:42:51.123 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T04:42:51.124 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:42:51.124 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T04:42:51.140 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3c1ee652-0387-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T04:42:51.140 DEBUG:teuthology.orchestra.run.smithi077:mon.smithi077> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service 2024-04-26T04:42:51.184 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T04:42:51.184 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.77 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:42:51.385 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:42:51.386 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '3c1ee652-0387-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.77', '--skip-admin-label'] 2024-04-26T04:42:51.411 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:42:51.411 INFO:teuthology.orchestra.run.smithi077.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T04:42:51.411 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying podman|docker is present... 2024-04-26T04:42:51.442 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:42:51.442 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying lvm2 is present... 2024-04-26T04:42:51.443 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying time synchronization is in place... 2024-04-26T04:42:51.449 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T04:42:51.449 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T04:42:51.455 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T04:42:51.455 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:42:51.462 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-04-26T04:42:51.468 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-04-26T04:42:51.468 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-04-26T04:42:51.468 INFO:teuthology.orchestra.run.smithi077.stdout:Repeating the final host check... 2024-04-26T04:42:51.492 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:42:51.492 INFO:teuthology.orchestra.run.smithi077.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T04:42:51.492 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl is present 2024-04-26T04:42:51.492 INFO:teuthology.orchestra.run.smithi077.stdout:lvcreate is present 2024-04-26T04:42:51.498 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T04:42:51.498 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T04:42:51.504 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T04:42:51.504 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:42:51.510 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:Host looks OK 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:Cluster fsid: 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:Acquiring lock 140586027840416 on /run/cephadm/3c1ee652-0387-11ef-bc93-c7b262605968.lock 2024-04-26T04:42:51.517 INFO:teuthology.orchestra.run.smithi077.stdout:Lock 140586027840416 acquired on /run/cephadm/3c1ee652-0387-11ef-bc93-c7b262605968.lock 2024-04-26T04:42:51.518 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 3300 ... 2024-04-26T04:42:51.518 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 6789 ... 2024-04-26T04:42:51.518 INFO:teuthology.orchestra.run.smithi077.stdout:Base mon IP(s) is [172.21.15.77:3300, 172.21.15.77:6789], mon addrv is [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-04-26T04:42:51.521 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.77 metric 100 2024-04-26T04:42:51.521 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.77 metric 100 2024-04-26T04:42:51.524 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T04:42:51.524 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-26T04:42:51.524 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-26T04:42:51.528 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T04:42:51.528 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T04:42:51.528 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T04:42:51.529 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T04:42:51.529 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 fe80::3e39:a3d5:549e:8707/64 scope link noprefixroute 2024-04-26T04:42:51.529 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T04:42:51.530 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-04-26T04:42:51.530 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-04-26T04:42:51.530 INFO:teuthology.orchestra.run.smithi077.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T04:42:51.530 INFO:teuthology.orchestra.run.smithi077.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T04:42:51.530 INFO:teuthology.orchestra.run.smithi077.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:0e2af62137028b073c6ccdb8d4d7e0ccccc475ce5f6e225cb983e90813813045 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying config sha256:0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-26T04:42:52.041 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T04:42:52.989 INFO:teuthology.orchestra.run.smithi077.stdout:ceph: stdout ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-26T04:42:52.990 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph version: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-26T04:42:52.990 INFO:teuthology.orchestra.run.smithi077.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T04:42:53.978 INFO:teuthology.orchestra.run.smithi077.stdout:stat: stdout 167 167 2024-04-26T04:42:53.978 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial keys... 2024-04-26T04:42:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDOMCtmuD3oERAAGCswMaH2cdrhlOBLCvAffQ== 2024-04-26T04:42:56.000 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDPMCtmp5WbEhAAI3jN1A/x68jzLOy4UEXGgw== 2024-04-26T04:42:57.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDQMCtmKzRYExAA+T+ttl2p8Qxe4yRFe3ObOQ== 2024-04-26T04:42:57.002 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial monmap... 2024-04-26T04:42:57.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T04:42:57.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-26T04:42:57.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:42:57.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T04:42:57.984 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool for smithi077 [v2:172.21.15.77:3300,v1:172.21.15.77:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T04:42:57.985 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = octopus 2024-04-26T04:42:57.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: set fsid to 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:42:57.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T04:42:57.985 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:42:57.985 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mon... 2024-04-26T04:42:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:create mon.smithi077 on 2024-04-26T04:42:59.226 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T04:42:59.360 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T04:42:59.499 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target. 2024-04-26T04:42:59.499 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target. 2024-04-26T04:42:59.657 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077 2024-04-26T04:42:59.657 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service: Unit ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service not loaded. 2024-04-26T04:42:59.820 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target.wants/ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service → /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968@.service. 2024-04-26T04:43:00.333 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:00 smithi077 ceph-mon[25925]: mon.smithi077 is new leader, mons smithi077 in quorum (ranks 0) 2024-04-26T04:43:00.333 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:00 smithi077 bash[25905]: d577db238a58cd53ffeb7a84049c4a75ec85de27ea434c5231e8af29c589e1b5 2024-04-26T04:43:00.346 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:43:00.346 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-04-26T04:43:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:43:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:43:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-04-26T04:43:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T04:43:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon to start... 2024-04-26T04:43:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon... 2024-04-26T04:43:00.592 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:00 smithi077 systemd[1]: Started Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: mon.smithi077 is new leader, mons smithi077 in quorum (ranks 0) 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: monmap e1: 1 mons at {smithi077=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: fsmap 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: mgrmap e1: no daemons active 2024-04-26T04:43:01.466 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:01 smithi077 ceph-mon[25925]: from='client.? 172.21.15.77:0/3309866689' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T04:43:01.468 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout id: 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout services: 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi077 (age 0.595021s) 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T04:43:01.479 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout data: 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.795+0000 7fb1e08f8700 1 Processor -- start 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1e08f8700 1 -- start start 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1e08f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1e08f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc105920 con 0x7fb1dc076c50 2024-04-26T04:43:01.480 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51492/0 (socket says 172.21.15.77:51492) 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.796+0000 7fb1da59c700 1 -- 172.21.15.77:0/1777938619 learned_addr learned my addr 172.21.15.77:0/1777938619 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1da59c700 1 -- 172.21.15.77:0/1777938619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc105a60 con 0x7fb1dc076c50 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1da59c700 1 --2- 172.21.15.77:0/1777938619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0098e0 tx=0x7fb1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fc9b27066b93a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1d959a700 1 -- 172.21.15.77:0/1777938619 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc00b860 con 0x7fb1dc076c50 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1d959a700 1 -- 172.21.15.77:0/1777938619 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc00b9c0 con 0x7fb1dc076c50 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1d959a700 1 -- 172.21.15.77:0/1777938619 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc00bc90 con 0x7fb1dc076c50 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1e08f8700 1 -- 172.21.15.77:0/1777938619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 msgr2=0x7fb1dc1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.797+0000 7fb1e08f8700 1 --2- 172.21.15.77:0/1777938619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0098e0 tx=0x7fb1cc009250 comp rx=0 tx=0).stop 2024-04-26T04:43:01.481 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- 172.21.15.77:0/1777938619 shutdown_connections 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 --2- 172.21.15.77:0/1777938619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc1053e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- 172.21.15.77:0/1777938619 >> 172.21.15.77:0/1777938619 conn(0x7fb1dc0fced0 msgr2=0x7fb1dc0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- 172.21.15.77:0/1777938619 shutdown_connections 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- 172.21.15.77:0/1777938619 wait complete. 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 Processor -- start 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- start start 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1e08f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc1113d0 con 0x7fb1dc076c50 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.798+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51508/0 (socket says 172.21.15.77:51508) 2024-04-26T04:43:01.482 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1da59c700 1 -- 172.21.15.77:0/3309866689 learned_addr learned my addr 172.21.15.77:0/3309866689 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1da59c700 1 -- 172.21.15.77:0/3309866689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0094f0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1da59c700 1 --2- 172.21.15.77:0/3309866689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009290 tx=0x7fb1cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc01c5c0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc01c740 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.799+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc1115d0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.800+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc01ca50 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.800+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc111fb0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.800+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb1cc01cbf0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.800+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb1cc025440 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.801+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc04bed0 con 0x7fb1dc076c50 2024-04-26T04:43:01.483 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.803+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb1cc01a450 con 0x7fb1dc076c50 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.874+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb1dc112a90 con 0x7fb1dc076c50 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.874+0000 7fb1caffd700 1 -- 172.21.15.77:0/3309866689 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fb1cc01a5f0 con 0x7fb1dc076c50 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 msgr2=0x7fb1dc110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 --2- 172.21.15.77:0/3309866689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009290 tx=0x7fb1cc016000 comp rx=0 tx=0).stop 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 shutdown_connections 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 --2- 172.21.15.77:0/3309866689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc076c50 0x7fb1dc110e90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 >> 172.21.15.77:0/3309866689 conn(0x7fb1dc0fced0 msgr2=0x7fb1dc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 shutdown_connections 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:00.875+0000 7fb1e08f8700 1 -- 172.21.15.77:0/3309866689 wait complete. 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:mon is available 2024-04-26T04:43:01.484 INFO:teuthology.orchestra.run.smithi077.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T04:43:02.557 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:02 smithi077 ceph-mon[25925]: from='client.? 172.21.15.77:0/2596524141' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T04:43:02.558 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:02 smithi077 ceph-mon[25925]: from='client.? 172.21.15.77:0/2596524141' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T04:43:02.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:02.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:02.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f62431d7700 1 Processor -- start 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f62431d7700 1 -- start start 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f62431d7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f62431d7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c103650 con 0x7f623c102c70 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f6240f73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:02.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f6240f73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51514/0 (socket says 172.21.15.77:51514) 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f6240f73700 1 -- 172.21.15.77:0/159478851 learned_addr learned my addr 172.21.15.77:0/159478851 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f6240f73700 1 -- 172.21.15.77:0/159478851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c103790 con 0x7f623c102c70 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.887+0000 7f6240f73700 1 --2- 172.21.15.77:0/159478851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f62300098e0 tx=0x7f6230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bcd54521ba4591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f623b7fe700 1 -- 172.21.15.77:0/159478851 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f623000b860 con 0x7f623c102c70 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f623b7fe700 1 -- 172.21.15.77:0/159478851 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623000b9c0 con 0x7f623c102c70 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f623b7fe700 1 -- 172.21.15.77:0/159478851 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f623000bc90 con 0x7f623c102c70 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 -- 172.21.15.77:0/159478851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 msgr2=0x7f623c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 --2- 172.21.15.77:0/159478851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f62300098e0 tx=0x7f6230009250 comp rx=0 tx=0).stop 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 -- 172.21.15.77:0/159478851 shutdown_connections 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 --2- 172.21.15.77:0/159478851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c103080 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 -- 172.21.15.77:0/159478851 >> 172.21.15.77:0/159478851 conn(0x7f623c0fc6f0 msgr2=0x7f623c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 -- 172.21.15.77:0/159478851 shutdown_connections 2024-04-26T04:43:02.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.888+0000 7f62431d7700 1 -- 172.21.15.77:0/159478851 wait complete. 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f62431d7700 1 Processor -- start 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f62431d7700 1 -- start start 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f62431d7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f62431d7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c113530 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f6240f73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f6240f73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51518/0 (socket says 172.21.15.77:51518) 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.889+0000 7f6240f73700 1 -- 172.21.15.77:0/2596524141 learned_addr learned my addr 172.21.15.77:0/2596524141 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f6240f73700 1 -- 172.21.15.77:0/2596524141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300094f0 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f6240f73700 1 --2- 172.21.15.77:0/2596524141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6230003080 tx=0x7f6230016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f623001c5c0 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623001c720 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c113730 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f623001ca30 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c114250 con 0x7f623c102c70 2024-04-26T04:43:02.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f623001cc60 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.890+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f623001ac80 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.891+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c05f5c0 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.893+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f623001ae40 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.959+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f623c04bed0 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.962+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f623001a450 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.962+0000 7f62397fa700 1 -- 172.21.15.77:0/2596524141 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f6230024520 con 0x7f623c102c70 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.963+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 msgr2=0x7f623c112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.963+0000 7f62431d7700 1 --2- 172.21.15.77:0/2596524141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6230003080 tx=0x7f6230016000 comp rx=0 tx=0).stop 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.963+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 shutdown_connections 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.963+0000 7f62431d7700 1 --2- 172.21.15.77:0/2596524141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f623c102c70 0x7f623c112ff0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.963+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 >> 172.21.15.77:0/2596524141 conn(0x7f623c0fc6f0 msgr2=0x7f623c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.964+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 shutdown_connections 2024-04-26T04:43:02.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:01.964+0000 7f62431d7700 1 -- 172.21.15.77:0/2596524141 wait complete. 2024-04-26T04:43:02.571 INFO:teuthology.orchestra.run.smithi077.stdout:Generating new minimal ceph.conf... 2024-04-26T04:43:03.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:03 smithi077 ceph-mon[25925]: from='client.? 172.21.15.77:0/2982974353' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:05.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.986+0000 7f1b38938700 1 Processor -- start 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.986+0000 7f1b38938700 1 -- start start 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b38938700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b38938700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b34105990 con 0x7f1b340712d0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51522/0 (socket says 172.21.15.77:51522) 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3259c700 1 -- 172.21.15.77:0/4123456100 learned_addr learned my addr 172.21.15.77:0/4123456100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3259c700 1 -- 172.21.15.77:0/4123456100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b34105ad0 con 0x7f1b340712d0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3259c700 1 --2- 172.21.15.77:0/4123456100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1b240098e0 tx=0x7f1b24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819d95c51b336a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3159a700 1 -- 172.21.15.77:0/4123456100 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b2400b860 con 0x7f1b340712d0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3159a700 1 -- 172.21.15.77:0/4123456100 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b2400be50 con 0x7f1b340712d0 2024-04-26T04:43:05.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.987+0000 7f1b3159a700 1 -- 172.21.15.77:0/4123456100 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b24010380 con 0x7f1b340712d0 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 -- 172.21.15.77:0/4123456100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 msgr2=0x7f1b34105450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 --2- 172.21.15.77:0/4123456100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1b240098e0 tx=0x7f1b24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 -- 172.21.15.77:0/4123456100 shutdown_connections 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 --2- 172.21.15.77:0/4123456100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34105450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 -- 172.21.15.77:0/4123456100 >> 172.21.15.77:0/4123456100 conn(0x7f1b340fcec0 msgr2=0x7f1b340ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:05.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 -- 172.21.15.77:0/4123456100 shutdown_connections 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 -- 172.21.15.77:0/4123456100 wait complete. 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.988+0000 7f1b38938700 1 Processor -- start 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b38938700 1 -- start start 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b38938700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b38938700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b341032e0 con 0x7f1b340712d0 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51538/0 (socket says 172.21.15.77:51538) 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b3259c700 1 -- 172.21.15.77:0/2982974353 learned_addr learned my addr 172.21.15.77:0/2982974353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b3259c700 1 -- 172.21.15.77:0/2982974353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240094f0 con 0x7f1b340712d0 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b3259c700 1 --2- 172.21.15.77:0/2982974353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006b00 tx=0x7f1b24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b240107c0 con 0x7f1b340712d0 2024-04-26T04:43:05.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b24010db0 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b34101350 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b2401acf0 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.989+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b34101840 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.990+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1b24024480 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.990+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1b24019710 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.990+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3404bed0 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:02.993+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f1b2401a530 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.058+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1b34101a80 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.058+0000 7f1b22ffd700 1 -- 172.21.15.77:0/2982974353 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1b24010920 con 0x7f1b340712d0 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 msgr2=0x7f1b34102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 --2- 172.21.15.77:0/2982974353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b24006b00 tx=0x7f1b24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:05.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 shutdown_connections 2024-04-26T04:43:05.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 --2- 172.21.15.77:0/2982974353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b340712d0 0x7f1b34102da0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:05.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 >> 172.21.15.77:0/2982974353 conn(0x7f1b340fcec0 msgr2=0x7f1b340fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:05.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 shutdown_connections 2024-04-26T04:43:05.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:03.059+0000 7f1b38938700 1 -- 172.21.15.77:0/2982974353 wait complete. 2024-04-26T04:43:05.506 INFO:teuthology.orchestra.run.smithi077.stdout:Restarting the monitor... 2024-04-26T04:43:05.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:05 smithi077 systemd[1]: Stopping Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:43:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:06 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[25920]: 2024-04-26T04:43:06.566+0000 7f58236b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi077 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:43:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:06 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[25920]: 2024-04-26T04:43:06.566+0000 7f58236b3700 -1 mon.smithi077@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T04:43:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:06 smithi077 podman[26180]: 2024-04-26 04:43:06.61880855 +0000 UTC m=+0.953447476 container died d577db238a58cd53ffeb7a84049c4a75ec85de27ea434c5231e8af29c589e1b5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:43:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:09 smithi077 podman[26180]: 2024-04-26 04:43:09.861245779 +0000 UTC m=+4.195884712 container cleanup d577db238a58cd53ffeb7a84049c4a75ec85de27ea434c5231e8af29c589e1b5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:43:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:09 smithi077 bash[26180]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077 2024-04-26T04:43:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:11 smithi077 podman[26194]: 2024-04-26 04:43:11.975079259 +0000 UTC m=+5.355133010 container remove d577db238a58cd53ffeb7a84049c4a75ec85de27ea434c5231e8af29c589e1b5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T04:43:12.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service: Deactivated successfully. 2024-04-26T04:43:12.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 systemd[1]: Stopped Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:43:12.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 systemd[1]: Starting Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:43:12.677 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 podman[26292]: 2024-04-26 04:43:12.28936933 +0000 UTC m=+0.018409474 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-26T04:43:12.677 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 podman[26292]: 2024-04-26 04:43:12.491687737 +0000 UTC m=+0.220727914 container create f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T04:43:12.677 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 podman[26292]: 2024-04-26 04:43:12.677120651 +0000 UTC m=+0.406160785 container init f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T04:43:12.765 INFO:teuthology.orchestra.run.smithi077.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 podman[26292]: 2024-04-26 04:43:12.679824498 +0000 UTC m=+0.408864634 container start f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: pidfile_write: ignore empty --pid-file 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: load: jerasure load: lrc 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: RocksDB version: 6.15.5 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Compile date Apr 24 2024 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: DB SUMMARY 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: DB Session ID: D11V9LGFNT0WSS5196OQ 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:43:12.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi077/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi077/store.db: 000010.log size: 98164 ; 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.env: 0x55c6b0273b00 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.fs: Posix File System 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.info_log: 0x55c6b1f99620 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.statistics: (nil) 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.use_fsync: 0 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:43:12.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.db_log_dir: 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi077/store.db 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:43:12.948 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.write_buffer_manager: 0x55c6b2c8ca20 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:43:12.949 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.unordered_write: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.row_cache: None 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.wal_filter: None 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.preserve_deletes: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:43:12.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:43:12.951 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:43:12.952 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_open_files: -1 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Compression algorithms supported: 2024-04-26T04:43:12.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kZSTD supported: 0 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi077/store.db/MANIFEST-000009 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.merge_operator: 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_filter: None 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:43:12.954 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c6b1f6cf58) 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_type: 0 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: data_block_index_type: 0 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_shortening: 1 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: hash_index_allow_collision: 1 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: checksum: 1 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: no_block_cache: 0 2024-04-26T04:43:12.955 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache: 0x55c6b1f9f090 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_options: 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: capacity : 536870912 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: num_shard_bits : 4 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: strict_capacity_limit : 0 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_compressed: (nil) 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: persistent_cache: (nil) 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_size: 4096 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_size_deviation: 10 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_restart_interval: 16 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_block_restart_interval: 1 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: metadata_block_size: 4096 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: partition_filters: 0 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: use_delta_encoding: 1 2024-04-26T04:43:12.956 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: whole_key_filtering: 1 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: verify_compression: 0 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: format_version: 4 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: enable_index_compression: 1 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_align: 0 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression: NoCompression 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.num_levels: 7 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:43:12.957 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:43:12.958 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:43:12.959 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.arena_block_size: 4194304 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:43:12.960 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:43:12.961 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.ttl: 2592000 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi077/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106592705737, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-26T04:43:12.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106592706660, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93370, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91482, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84046, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1714106592, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "d06d1a6d-30ac-41dd-aafb-0e5d6a18e0d8", "db_session_id": "D11V9LGFNT0WSS5196OQ"}} 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106592706974, "job": 1, "event": "recovery_finished"} 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi077/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55c6b1f33180 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: DB pointer 0x55c6b2008000 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** DB Stats ** 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:43:12.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:43:12.964 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative compaction: 0.00 GB write, 28.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval compaction: 0.00 GB write, 28.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.965 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.3 0.00 0.00 1 0.001 0 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative compaction: 0.00 GB write, 28.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:43:12.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: starting mon.smithi077 rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi077 fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???) e1 preinit fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).mds e1 new map 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).mds e1 print_map 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: e1 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: legacy client fscid: -1 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: No filesystems configured 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:43:12.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 bash[26292]: f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 systemd[1]: Started Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mon.smithi077 is new leader, mons smithi077 in quorum (ranks 0) 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: monmap e1: 1 mons at {smithi077=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: fsmap 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T04:43:12.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:12 smithi077 ceph-mon[26311]: mgrmap e1: no daemons active 2024-04-26T04:43:13.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb7279b700 1 Processor -- start 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb7279b700 1 -- start start 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb7279b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb7279b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1031d0 con 0x7ffb6c102880 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54776/0 (socket says 172.21.15.77:54776) 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.136+0000 7ffb6bfff700 1 -- 172.21.15.77:0/1217294085 learned_addr learned my addr 172.21.15.77:0/1217294085 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.137+0000 7ffb6bfff700 1 -- 172.21.15.77:0/1217294085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c103310 con 0x7ffb6c102880 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.137+0000 7ffb6bfff700 1 --2- 172.21.15.77:0/1217294085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0098e0 tx=0x7ffb5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fbcb1f9dba80e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.137+0000 7ffb6affd700 1 -- 172.21.15.77:0/1217294085 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb5c00b860 con 0x7ffb6c102880 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.137+0000 7ffb6affd700 1 -- 172.21.15.77:0/1217294085 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb5c00be50 con 0x7ffb6c102880 2024-04-26T04:43:13.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.137+0000 7ffb6affd700 1 -- 172.21.15.77:0/1217294085 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb5c010380 con 0x7ffb6c102880 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- 172.21.15.77:0/1217294085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 msgr2=0x7ffb6c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 --2- 172.21.15.77:0/1217294085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0098e0 tx=0x7ffb5c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- 172.21.15.77:0/1217294085 shutdown_connections 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 --2- 172.21.15.77:0/1217294085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c102c90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- 172.21.15.77:0/1217294085 >> 172.21.15.77:0/1217294085 conn(0x7ffb6c0fe060 msgr2=0x7ffb6c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- 172.21.15.77:0/1217294085 shutdown_connections 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- 172.21.15.77:0/1217294085 wait complete. 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 Processor -- start 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.138+0000 7ffb7279b700 1 -- start start 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb7279b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:13.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb7279b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c111b30 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54790/0 (socket says 172.21.15.77:54790) 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb6bfff700 1 -- 172.21.15.77:0/2545460875 learned_addr learned my addr 172.21.15.77:0/2545460875 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb6bfff700 1 -- 172.21.15.77:0/2545460875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0094f0 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb6bfff700 1 --2- 172.21.15.77:0/2545460875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00b440 tx=0x7ffb5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb5c010760 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb5c010d50 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c111d30 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb5c01ae20 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.139+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c1128a0 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.140+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ffb5c02e440 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.140+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ffb5c024740 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.140+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c05f5c0 con 0x7ffb6c102880 2024-04-26T04:43:13.660 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.142+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7ffb5c029070 con 0x7ffb6c102880 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.208+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ffb6c104650 con 0x7ffb6c102880 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.211+0000 7ffb68ff9700 1 -- 172.21.15.77:0/2545460875 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffb5c01a6a0 con 0x7ffb6c102880 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 msgr2=0x7ffb6c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 --2- 172.21.15.77:0/2545460875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00b440 tx=0x7ffb5c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 shutdown_connections 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 --2- 172.21.15.77:0/2545460875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb6c102880 0x7ffb6c1115f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:13.661 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 >> 172.21.15.77:0/2545460875 conn(0x7ffb6c0fe060 msgr2=0x7ffb6c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 shutdown_connections 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:13.212+0000 7ffb7279b700 1 -- 172.21.15.77:0/2545460875 wait complete. 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mgr... 2024-04-26T04:43:13.662 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 9283 ... 2024-04-26T04:43:13.819 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mgr.smithi077.hkvsor 2024-04-26T04:43:13.819 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mgr.smithi077.hkvsor.service: Unit ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mgr.smithi077.hkvsor.service not loaded. 2024-04-26T04:43:13.967 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968.target.wants/ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mgr.smithi077.hkvsor.service → /etc/systemd/system/ceph-3c1ee652-0387-11ef-bc93-c7b262605968@.service. 2024-04-26T04:43:14.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:14 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2545460875' entity='client.admin' 2024-04-26T04:43:14.553 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:43:14.553 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-04-26T04:43:14.561 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:43:14.561 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:43:14.561 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-04-26T04:43:14.561 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T04:43:14.571 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:43:14.572 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-04-26T04:43:14.581 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:43:14.582 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:43:14.582 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-04-26T04:43:14.582 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-26T04:43:14.582 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr to start... 2024-04-26T04:43:14.582 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr... 2024-04-26T04:43:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:15 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1068555235' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "3c1ee652-0387-11ef-bc93-c7b262605968", 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:15.817 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "smithi077" 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:43:15.818 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:43:15.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:43:15.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:43:00.282317+0000", 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc437fa700 1 Processor -- start 2024-04-26T04:43:15.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc437fa700 1 -- start start 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc437fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc437fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c1059b0 con 0x7fcc3c076c10 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc41596700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc41596700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54822/0 (socket says 172.21.15.77:54822) 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc41596700 1 -- 172.21.15.77:0/510629621 learned_addr learned my addr 172.21.15.77:0/510629621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.987+0000 7fcc41596700 1 -- 172.21.15.77:0/510629621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c105af0 con 0x7fcc3c076c10 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.988+0000 7fcc41596700 1 --2- 172.21.15.77:0/510629621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc380098e0 tx=0x7fcc38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e5c7de7db2fbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.988+0000 7fcc33fff700 1 -- 172.21.15.77:0/510629621 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc3800b860 con 0x7fcc3c076c10 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.988+0000 7fcc33fff700 1 -- 172.21.15.77:0/510629621 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcc3800be50 con 0x7fcc3c076c10 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.988+0000 7fcc33fff700 1 -- 172.21.15.77:0/510629621 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc38010380 con 0x7fcc3c076c10 2024-04-26T04:43:15.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 -- 172.21.15.77:0/510629621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 msgr2=0x7fcc3c105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 --2- 172.21.15.77:0/510629621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc380098e0 tx=0x7fcc38009250 comp rx=0 tx=0).stop 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 -- 172.21.15.77:0/510629621 shutdown_connections 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 --2- 172.21.15.77:0/510629621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c105470 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 -- 172.21.15.77:0/510629621 >> 172.21.15.77:0/510629621 conn(0x7fcc3c0fcf00 msgr2=0x7fcc3c0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 -- 172.21.15.77:0/510629621 shutdown_connections 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.989+0000 7fcc437fa700 1 -- 172.21.15.77:0/510629621 wait complete. 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc437fa700 1 Processor -- start 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc437fa700 1 -- start start 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc437fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc437fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c111ae0 con 0x7fcc3c076c10 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc41596700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc41596700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54826/0 (socket says 172.21.15.77:54826) 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc41596700 1 -- 172.21.15.77:0/1068555235 learned_addr learned my addr 172.21.15.77:0/1068555235 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:15.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc41596700 1 -- 172.21.15.77:0/1068555235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380094f0 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.990+0000 7fcc41596700 1 --2- 172.21.15.77:0/1068555235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc38003050 tx=0x7fcc38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc38010760 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcc38010d50 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3c111d40 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc3801adb0 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c112850 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcc3802e440 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.991+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcc38024740 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.992+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3c05f5c0 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:14.994+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fcc38029070 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.076+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcc3c106840 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.076+0000 7fcc31ffb700 1 -- 172.21.15.77:0/1068555235 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fcc38010ec0 con 0x7fcc3c076c10 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 msgr2=0x7fcc3c1115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:15.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 --2- 172.21.15.77:0/1068555235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc38003050 tx=0x7fcc38009250 comp rx=0 tx=0).stop 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 shutdown_connections 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 --2- 172.21.15.77:0/1068555235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc3c076c10 0x7fcc3c1115a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 >> 172.21.15.77:0/1068555235 conn(0x7fcc3c0fcf00 msgr2=0x7fcc3c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 shutdown_connections 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:15.078+0000 7fcc437fa700 1 -- 172.21.15.77:0/1068555235 wait complete. 2024-04-26T04:43:15.833 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (1/15)... 2024-04-26T04:43:20.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:20 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1734109836' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:43:21.204 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "3c1ee652-0387-11ef-bc93-c7b262605968", 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "smithi077" 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:21.205 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:43:21.206 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:43:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T04:43:21.215 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:43:00.282317+0000", 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:43:21.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.515+0000 7f2916ce8700 1 Processor -- start 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.515+0000 7f2916ce8700 1 -- start start 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.515+0000 7f2916ce8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.515+0000 7f2916ce8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101031c0 con 0x7f2910102870 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.515+0000 7f2914a84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f2914a84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53076/0 (socket says 172.21.15.77:53076) 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f2914a84700 1 -- 172.21.15.77:0/1656776110 learned_addr learned my addr 172.21.15.77:0/1656776110 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f2914a84700 1 -- 172.21.15.77:0/1656776110 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910103300 con 0x7f2910102870 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f2914a84700 1 --2- 172.21.15.77:0/1656776110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29040098e0 tx=0x7f2904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96340b000ce21f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f290f7fe700 1 -- 172.21.15.77:0/1656776110 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f290400b860 con 0x7f2910102870 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f290f7fe700 1 -- 172.21.15.77:0/1656776110 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f290400be50 con 0x7f2910102870 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.516+0000 7f290f7fe700 1 -- 172.21.15.77:0/1656776110 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2904010380 con 0x7f2910102870 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 -- 172.21.15.77:0/1656776110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 msgr2=0x7f2910102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 --2- 172.21.15.77:0/1656776110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29040098e0 tx=0x7f2904009250 comp rx=0 tx=0).stop 2024-04-26T04:43:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 -- 172.21.15.77:0/1656776110 shutdown_connections 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 --2- 172.21.15.77:0/1656776110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910102c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 -- 172.21.15.77:0/1656776110 >> 172.21.15.77:0/1656776110 conn(0x7f29100fe070 msgr2=0x7f2910100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 -- 172.21.15.77:0/1656776110 shutdown_connections 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.517+0000 7f2916ce8700 1 -- 172.21.15.77:0/1656776110 wait complete. 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2916ce8700 1 Processor -- start 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2916ce8700 1 -- start start 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2916ce8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2916ce8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2910111ad0 con 0x7f2910102870 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2914a84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2914a84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53078/0 (socket says 172.21.15.77:53078) 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2914a84700 1 -- 172.21.15.77:0/1734109836 learned_addr learned my addr 172.21.15.77:0/1734109836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.518+0000 7f2914a84700 1 -- 172.21.15.77:0/1734109836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040094f0 con 0x7f2910102870 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f2914a84700 1 --2- 172.21.15.77:0/1734109836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f290400b440 tx=0x7f2904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:21.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2904010760 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910111cd0 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2910112840 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2904010d50 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f290401ae20 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f290402e440 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.519+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2904024760 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.520+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f40052e0 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.522+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f2904029070 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.594+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f28f4005860 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.595+0000 7f290d7fa700 1 -- 172.21.15.77:0/1734109836 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f290401a6a0 con 0x7f2910102870 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 msgr2=0x7f2910111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 --2- 172.21.15.77:0/1734109836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f290400b440 tx=0x7f2904009250 comp rx=0 tx=0).stop 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 shutdown_connections 2024-04-26T04:43:21.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 --2- 172.21.15.77:0/1734109836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102870 0x7f2910111590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:21.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 >> 172.21.15.77:0/1734109836 conn(0x7f29100fe070 msgr2=0x7f29100feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:21.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 shutdown_connections 2024-04-26T04:43:21.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:20.596+0000 7f2916ce8700 1 -- 172.21.15.77:0/1734109836 wait complete. 2024-04-26T04:43:21.220 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (2/15)... 2024-04-26T04:43:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:43:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: mgrmap e2: smithi077.hkvsor(active, starting, since 0.00361475s) 2024-04-26T04:43:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:43:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:43:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:21.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:21.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:21.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:21 smithi077 ceph-mon[26311]: from='mgr.14100 172.21.15.77:0/138474202' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:22 smithi077 ceph-mon[26311]: mgrmap e3: smithi077.hkvsor(active, since 1.00576s) 2024-04-26T04:43:24.229 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:23 smithi077 ceph-mon[26311]: mgrmap e4: smithi077.hkvsor(active, since 2s) 2024-04-26T04:43:24.229 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:23 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/376728861' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "3c1ee652-0387-11ef-bc93-c7b262605968", 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:43:24.494 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "smithi077" 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:24.495 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:43:24.496 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:43:24.497 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:43:00.282317+0000", 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:24.498 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.655+0000 7f099d94c700 1 Processor -- start 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.655+0000 7f099d94c700 1 -- start start 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.655+0000 7f099d94c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.655+0000 7f099d94c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09981031d0 con 0x7f0998102880 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53158/0 (socket says 172.21.15.77:53158) 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0996ffd700 1 -- 172.21.15.77:0/4126371497 learned_addr learned my addr 172.21.15.77:0/4126371497 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0996ffd700 1 -- 172.21.15.77:0/4126371497 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0998103310 con 0x7f0998102880 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0996ffd700 1 --2- 172.21.15.77:0/4126371497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09900098e0 tx=0x7f0990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28868f58924065f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0995ffb700 1 -- 172.21.15.77:0/4126371497 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f099000b860 con 0x7f0998102880 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0995ffb700 1 -- 172.21.15.77:0/4126371497 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f099000be50 con 0x7f0998102880 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.656+0000 7f0995ffb700 1 -- 172.21.15.77:0/4126371497 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0990010380 con 0x7f0998102880 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 -- 172.21.15.77:0/4126371497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 msgr2=0x7f0998102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:24.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 --2- 172.21.15.77:0/4126371497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09900098e0 tx=0x7f0990009250 comp rx=0 tx=0).stop 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 -- 172.21.15.77:0/4126371497 shutdown_connections 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 --2- 172.21.15.77:0/4126371497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f0998102c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 -- 172.21.15.77:0/4126371497 >> 172.21.15.77:0/4126371497 conn(0x7f09980fe060 msgr2=0x7f0998100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 -- 172.21.15.77:0/4126371497 shutdown_connections 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.657+0000 7f099d94c700 1 -- 172.21.15.77:0/4126371497 wait complete. 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f099d94c700 1 Processor -- start 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f099d94c700 1 -- start start 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f099d94c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f099d94c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998111b30 con 0x7f0998102880 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53160/0 (socket says 172.21.15.77:53160) 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f0996ffd700 1 -- 172.21.15.77:0/376728861 learned_addr learned my addr 172.21.15.77:0/376728861 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f0996ffd700 1 -- 172.21.15.77:0/376728861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900094f0 con 0x7f0998102880 2024-04-26T04:43:24.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.658+0000 7f0996ffd700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f099000b440 tx=0x7f0990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0990010760 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0998111d30 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0990010d50 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f099001ae20 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981128a0 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.659+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f0990024470 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.660+0000 7f098f7fe700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 0x7f0984038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.660+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f099004a580 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.660+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099805f5c0 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.660+0000 7f09967fc700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 0x7f0984038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.660+0000 7f09967fc700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 0x7f0984038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0980009990 tx=0x7f0980006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.665+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0990018940 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.923+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0998112ae0 con 0x7f0998102880 2024-04-26T04:43:24.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.923+0000 7f098f7fe700 1 -- 172.21.15.77:0/376728861 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f0990018350 con 0x7f0998102880 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 msgr2=0x7f0984038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 0x7f0984038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0980009990 tx=0x7f0980006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 msgr2=0x7f09981115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f099000b440 tx=0x7f0990009250 comp rx=0 tx=0).stop 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 shutdown_connections 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f09840367e0 0x7f0984038c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 --2- 172.21.15.77:0/376728861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0998102880 0x7f09981115f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 >> 172.21.15.77:0/376728861 conn(0x7f09980fe060 msgr2=0x7f09980feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 shutdown_connections 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:23.925+0000 7f099d94c700 1 -- 172.21.15.77:0/376728861 wait complete. 2024-04-26T04:43:24.502 INFO:teuthology.orchestra.run.smithi077.stdout:mgr is available 2024-04-26T04:43:25.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:25 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1919698868' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T04:43:25.686 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc0623de700 1 Processor -- start 2024-04-26T04:43:25.687 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc0623de700 1 -- start start 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc0623de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc0623de700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c1031d0 con 0x7fc05c102880 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53172/0 (socket says 172.21.15.77:53172) 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.884+0000 7fc05bfff700 1 -- 172.21.15.77:0/2201181488 learned_addr learned my addr 172.21.15.77:0/2201181488 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.885+0000 7fc05bfff700 1 -- 172.21.15.77:0/2201181488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c103310 con 0x7fc05c102880 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.885+0000 7fc05bfff700 1 --2- 172.21.15.77:0/2201181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098e0 tx=0x7fc04c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9933bef7c3b08461 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.885+0000 7fc05affd700 1 -- 172.21.15.77:0/2201181488 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c00b860 con 0x7fc05c102880 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.885+0000 7fc05affd700 1 -- 172.21.15.77:0/2201181488 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc04c00be50 con 0x7fc05c102880 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.885+0000 7fc05affd700 1 -- 172.21.15.77:0/2201181488 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c010380 con 0x7fc05c102880 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- 172.21.15.77:0/2201181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 msgr2=0x7fc05c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 --2- 172.21.15.77:0/2201181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098e0 tx=0x7fc04c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- 172.21.15.77:0/2201181488 shutdown_connections 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 --2- 172.21.15.77:0/2201181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c102c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- 172.21.15.77:0/2201181488 >> 172.21.15.77:0/2201181488 conn(0x7fc05c0fe060 msgr2=0x7fc05c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- 172.21.15.77:0/2201181488 shutdown_connections 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- 172.21.15.77:0/2201181488 wait complete. 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 Processor -- start 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- start start 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.886+0000 7fc0623de700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c111b30 con 0x7fc05c102880 2024-04-26T04:43:25.689 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53180/0 (socket says 172.21.15.77:53180) 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc05bfff700 1 -- 172.21.15.77:0/1919698868 learned_addr learned my addr 172.21.15.77:0/1919698868 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc05bfff700 1 -- 172.21.15.77:0/1919698868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0094f0 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc05bfff700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b440 tx=0x7fc04c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c010760 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc04c010d50 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c111d30 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c01ae20 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.887+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c1128a0 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.888+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fc04c02e440 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.888+0000 7fc058ff9700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 0x7fc044038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.888+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc04c04a560 con 0x7fc05c102880 2024-04-26T04:43:25.690 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.888+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c05f5c0 con 0x7fc05c102880 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.888+0000 7fc05b7fe700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 0x7fc044038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.889+0000 7fc05b7fe700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 0x7fc044038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc050009990 tx=0x7fc050006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:24.893+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc04c018940 con 0x7fc05c102880 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.076+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc05c04bed0 con 0x7fc05c102880 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.077+0000 7fc058ff9700 1 -- 172.21.15.77:0/1919698868 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc04c02d3a0 con 0x7fc05c102880 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 msgr2=0x7fc044038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 0x7fc044038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc050009990 tx=0x7fc050006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 msgr2=0x7fc05c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b440 tx=0x7fc04c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 shutdown_connections 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7fc044036850 0x7fc044038d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.079+0000 7fc0623de700 1 --2- 172.21.15.77:0/1919698868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c102880 0x7fc05c1115f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.080+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 >> 172.21.15.77:0/1919698868 conn(0x7fc05c0fe060 msgr2=0x7fc05c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.080+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 shutdown_connections 2024-04-26T04:43:25.691 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:25.080+0000 7fc0623de700 1 -- 172.21.15.77:0/1919698868 wait complete. 2024-04-26T04:43:25.692 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling cephadm module... 2024-04-26T04:43:26.659 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:26 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1294233361' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a3d4a3700 1 Processor -- start 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a3d4a3700 1 -- start start 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a3d4a3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a3d4a3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38102080 con 0x7f9a38101730 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:26.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53194/0 (socket says 172.21.15.77:53194) 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a36ffd700 1 -- 172.21.15.77:0/324949049 learned_addr learned my addr 172.21.15.77:0/324949049 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a36ffd700 1 -- 172.21.15.77:0/324949049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a381021c0 con 0x7f9a38101730 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.050+0000 7f9a36ffd700 1 --2- 172.21.15.77:0/324949049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098e0 tx=0x7f9a28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940d9d07434c9e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a35ffb700 1 -- 172.21.15.77:0/324949049 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a2800b860 con 0x7f9a38101730 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a35ffb700 1 -- 172.21.15.77:0/324949049 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9a2800be50 con 0x7f9a38101730 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a35ffb700 1 -- 172.21.15.77:0/324949049 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a28010380 con 0x7f9a38101730 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/324949049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 msgr2=0x7f9a38101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/324949049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098e0 tx=0x7f9a28009250 comp rx=0 tx=0).stop 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/324949049 shutdown_connections 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/324949049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38101b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:26.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/324949049 >> 172.21.15.77:0/324949049 conn(0x7f9a380fcef0 msgr2=0x7f9a380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/324949049 shutdown_connections 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.051+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/324949049 wait complete. 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a3d4a3700 1 Processor -- start 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a3d4a3700 1 -- start start 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a3d4a3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a3d4a3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38111b60 con 0x7f9a38101730 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53198/0 (socket says 172.21.15.77:53198) 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a36ffd700 1 -- 172.21.15.77:0/1294233361 learned_addr learned my addr 172.21.15.77:0/1294233361 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.052+0000 7f9a36ffd700 1 -- 172.21.15.77:0/1294233361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280094f0 con 0x7f9a38101730 2024-04-26T04:43:26.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a36ffd700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a280107c0 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9a28010db0 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a38111d60 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a2801ad00 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a38112880 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.053+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f9a280193c0 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.054+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3805f5c0 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.054+0000 7f9a1f7fe700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 0x7f9a20038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.054+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9a2804a5c0 con 0x7f9a38101730 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.054+0000 7f9a367fc700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 0x7f9a20038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.054+0000 7f9a367fc700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 0x7f9a20038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009990 tx=0x7f9a2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:26.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.058+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9a28019670 con 0x7f9a38101730 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.297+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f9a3804bed0 con 0x7f9a38101730 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.349+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9a2801a530 con 0x7f9a38101730 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.350+0000 7f9a1f7fe700 1 -- 172.21.15.77:0/1294233361 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f9a2802b9c0 con 0x7f9a38101730 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.352+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 msgr2=0x7f9a20038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.352+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 0x7f9a20038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009990 tx=0x7f9a2c006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.352+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 msgr2=0x7f9a38111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a28009250 comp rx=0 tx=0).stop 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 shutdown_connections 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f9a20036850 0x7f9a20038d00 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009990 tx=0x7f9a2c006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 --2- 172.21.15.77:0/1294233361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a38101730 0x7f9a38111620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 >> 172.21.15.77:0/1294233361 conn(0x7f9a380fcef0 msgr2=0x7f9a380fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 shutdown_connections 2024-04-26T04:43:26.981 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:26.353+0000 7f9a3d4a3700 1 -- 172.21.15.77:0/1294233361 wait complete. 2024-04-26T04:43:27.600 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:27 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1294233361' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T04:43:27.600 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:27 smithi077 ceph-mon[26311]: mgrmap e5: smithi077.hkvsor(active, since 5s) 2024-04-26T04:43:28.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "smithi077.hkvsor", 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.393+0000 7f1db9856700 1 Processor -- start 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.393+0000 7f1db9856700 1 -- start start 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.393+0000 7f1db9856700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.393+0000 7f1db9856700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db41026b0 con 0x7f1db4101d60 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53224/0 (socket says 172.21.15.77:53224) 2024-04-26T04:43:28.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db2ffd700 1 -- 172.21.15.77:0/1316582162 learned_addr learned my addr 172.21.15.77:0/1316582162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db2ffd700 1 -- 172.21.15.77:0/1316582162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db41027f0 con 0x7f1db4101d60 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db2ffd700 1 --2- 172.21.15.77:0/1316582162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1da800bb80 tx=0x7f1da800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6721e5fe287f2774 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db1ffb700 1 -- 172.21.15.77:0/1316582162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da800dd00 con 0x7f1db4101d60 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db1ffb700 1 -- 172.21.15.77:0/1316582162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1da800de60 con 0x7f1db4101d60 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.394+0000 7f1db1ffb700 1 -- 172.21.15.77:0/1316582162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da80188a0 con 0x7f1db4101d60 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 -- 172.21.15.77:0/1316582162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 msgr2=0x7f1db4102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 --2- 172.21.15.77:0/1316582162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1da800bb80 tx=0x7f1da800b4f0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 -- 172.21.15.77:0/1316582162 shutdown_connections 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 --2- 172.21.15.77:0/1316582162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db4102170 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 -- 172.21.15.77:0/1316582162 >> 172.21.15.77:0/1316582162 conn(0x7f1db40fc6b0 msgr2=0x7f1db40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 -- 172.21.15.77:0/1316582162 shutdown_connections 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.395+0000 7f1db9856700 1 -- 172.21.15.77:0/1316582162 wait complete. 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db9856700 1 Processor -- start 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db9856700 1 -- start start 2024-04-26T04:43:28.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db9856700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db9856700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db410f870 con 0x7f1db4101d60 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53238/0 (socket says 172.21.15.77:53238) 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db2ffd700 1 -- 172.21.15.77:0/1914211622 learned_addr learned my addr 172.21.15.77:0/1914211622 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db2ffd700 1 -- 172.21.15.77:0/1914211622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da800b790 con 0x7f1db4101d60 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1db2ffd700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d650 tx=0x7f1da800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.396+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da8010b50 con 0x7f1db4101d60 2024-04-26T04:43:28.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.397+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db410fa70 con 0x7f1db4101d60 2024-04-26T04:43:28.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.397+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1da801d050 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.397+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da8022590 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.397+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db41105e0 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.397+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1da8022790 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.398+0000 7f1d9b7fe700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 0x7f1d9c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.398+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1da8014070 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.398+0000 7f1db27fc700 1 -- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 msgr2=0x7f1d9c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.398+0000 7f1db27fc700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 0x7f1d9c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.398+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db405f5c0 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.403+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1da8049020 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.599+0000 7f1db27fc700 1 -- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 msgr2=0x7f1d9c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.599+0000 7f1db27fc700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 0x7f1d9c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.696+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1db4110ff0 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.696+0000 7f1d9b7fe700 1 -- 172.21.15.77:0/1914211622 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f1da800e360 con 0x7f1db4101d60 2024-04-26T04:43:28.278 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 msgr2=0x7f1d9c038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 0x7f1d9c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 msgr2=0x7f1db410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1da800d650 tx=0x7f1da800b4f0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 shutdown_connections 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7f1d9c036830 0x7f1d9c038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 --2- 172.21.15.77:0/1914211622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4101d60 0x7f1db410f330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.698+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 >> 172.21.15.77:0/1914211622 conn(0x7f1db40fc6b0 msgr2=0x7f1db41060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.699+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 shutdown_connections 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:27.699+0000 7f1db9856700 1 -- 172.21.15.77:0/1914211622 wait complete. 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-04-26T04:43:28.279 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 5... 2024-04-26T04:43:28.537 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:28 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1914211622' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T04:43:32.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: Active manager daemon smithi077.hkvsor restarted 2024-04-26T04:43:32.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: mgrmap e6: smithi077.hkvsor(active, starting, since 0.00340942s) 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:43:32.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:32.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:32.996 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:32.996 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:32.996 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:32 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:34.112 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:33 smithi077 ceph-mon[26311]: Found migration_current of "None". Setting to last migration. 2024-04-26T04:43:34.112 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:33 smithi077 ceph-mon[26311]: mgrmap e7: smithi077.hkvsor(active, since 1.00511s) 2024-04-26T04:43:34.384 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:34.385 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.739+0000 7faf0b770700 1 Processor -- start 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0b770700 1 -- start start 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0b770700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0b770700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041059b0 con 0x7faf040712d0 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0950c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0950c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33610/0 (socket says 172.21.15.77:33610) 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0950c700 1 -- 172.21.15.77:0/4237720945 learned_addr learned my addr 172.21.15.77:0/4237720945 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0950c700 1 -- 172.21.15.77:0/4237720945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04105af0 con 0x7faf040712d0 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.740+0000 7faf0950c700 1 --2- 172.21.15.77:0/4237720945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad0fca6b1b5ee37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faefbfff700 1 -- 172.21.15.77:0/4237720945 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf0000b860 con 0x7faf040712d0 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faefbfff700 1 -- 172.21.15.77:0/4237720945 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faf0000be50 con 0x7faf040712d0 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faefbfff700 1 -- 172.21.15.77:0/4237720945 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf00010380 con 0x7faf040712d0 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 -- 172.21.15.77:0/4237720945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 msgr2=0x7faf04105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 --2- 172.21.15.77:0/4237720945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf00009250 comp rx=0 tx=0).stop 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 -- 172.21.15.77:0/4237720945 shutdown_connections 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 --2- 172.21.15.77:0/4237720945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04105470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 -- 172.21.15.77:0/4237720945 >> 172.21.15.77:0/4237720945 conn(0x7faf040fcea0 msgr2=0x7faf040ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 -- 172.21.15.77:0/4237720945 shutdown_connections 2024-04-26T04:43:34.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.741+0000 7faf0b770700 1 -- 172.21.15.77:0/4237720945 wait complete. 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0b770700 1 Processor -- start 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0b770700 1 -- start start 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0b770700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0b770700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf04103300 con 0x7faf040712d0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0950c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0950c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33620/0 (socket says 172.21.15.77:33620) 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.742+0000 7faf0950c700 1 -- 172.21.15.77:0/1616088220 learned_addr learned my addr 172.21.15.77:0/1616088220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faf0950c700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000094f0 con 0x7faf040712d0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faf0950c700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faf00006b00 tx=0x7faf00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf000107c0 con 0x7faf040712d0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf04101370 con 0x7faf040712d0 2024-04-26T04:43:34.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faf00010db0 con 0x7faf040712d0 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf0001acf0 con 0x7faf040712d0 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.743+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf04101860 con 0x7faf040712d0 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7faf000193c0 con 0x7faf040712d0 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faef9ffb700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7faef0039410 con 0x7faef0036850 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faf08d0b700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7faf0004a610 con 0x7faf040712d0 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.744+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.945+0000 7faf08d0b700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:28.945+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:29.346+0000 7faf08d0b700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:29.346+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T04:43:34.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:30.147+0000 7faf08d0b700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:30.147+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:31.748+0000 7faf08d0b700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/171166588 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:31.748+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:32.801+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7faf000198a0 con 0x7faf040712d0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:32.801+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 msgr2=0x7faef0038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:32.801+0000 7faef9ffb700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.802+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7faf00019c20 con 0x7faf040712d0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.802+0000 7faef9ffb700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 0x7faef003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.802+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7faef0039410 con 0x7faef0039ff0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.803+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 0x7faef003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.804+0000 7faf08d0b700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 0x7faef003c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faef40039d0 tx=0x7faef4009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.804+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7faef0039410 con 0x7faef0039ff0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.814+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7faf0404bed0 con 0x7faef0039ff0 2024-04-26T04:43:34.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.814+0000 7faef9ffb700 1 -- 172.21.15.77:0/1616088220 <== mgr.14118 v2:172.21.15.77:6800/3157890049 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7faf0404bed0 con 0x7faef0039ff0 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 msgr2=0x7faef003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 0x7faef003c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faef40039d0 tx=0x7faef4009350 comp rx=0 tx=0).stop 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 msgr2=0x7faf04102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faf00006b00 tx=0x7faf00009250 comp rx=0 tx=0).stop 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 shutdown_connections 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7faef0039ff0 0x7faef003c3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf040712d0 0x7faf04102dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 --2- 172.21.15.77:0/1616088220 >> [v2:172.21.15.77:6800/171166588,v1:172.21.15.77:6801/171166588] conn(0x7faef0036850 0x7faef0038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 >> 172.21.15.77:0/1616088220 conn(0x7faf040fcea0 msgr2=0x7faf040fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:34.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 shutdown_connections 2024-04-26T04:43:34.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:33.815+0000 7faf0b770700 1 -- 172.21.15.77:0/1616088220 wait complete. 2024-04-26T04:43:34.394 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 5 is available 2024-04-26T04:43:34.394 INFO:teuthology.orchestra.run.smithi077.stdout:Setting orchestrator backend to cephadm... 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: [26/Apr/2024:04:43:34] ENGINE Bus STARTING 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: [26/Apr/2024:04:43:34] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: [26/Apr/2024:04:43:34] ENGINE Bus STARTED 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:35.645 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:35 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff916457700 1 Processor -- start 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff916457700 1 -- start start 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff916457700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff916457700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff910102080 con 0x7ff910101730 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33716/0 (socket says 172.21.15.77:33716) 2024-04-26T04:43:35.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.787+0000 7ff90ffff700 1 -- 172.21.15.77:0/1753709086 learned_addr learned my addr 172.21.15.77:0/1753709086 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff90ffff700 1 -- 172.21.15.77:0/1753709086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9101021c0 con 0x7ff910101730 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff90ffff700 1 --2- 172.21.15.77:0/1753709086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9000098e0 tx=0x7ff900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d5a6e5502d7b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff90effd700 1 -- 172.21.15.77:0/1753709086 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90000b860 con 0x7ff910101730 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff90effd700 1 -- 172.21.15.77:0/1753709086 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff90000be50 con 0x7ff910101730 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff90effd700 1 -- 172.21.15.77:0/1753709086 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff900010380 con 0x7ff910101730 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff916457700 1 -- 172.21.15.77:0/1753709086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 msgr2=0x7ff910101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.788+0000 7ff916457700 1 --2- 172.21.15.77:0/1753709086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9000098e0 tx=0x7ff900009250 comp rx=0 tx=0).stop 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- 172.21.15.77:0/1753709086 shutdown_connections 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 --2- 172.21.15.77:0/1753709086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910101b40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- 172.21.15.77:0/1753709086 >> 172.21.15.77:0/1753709086 conn(0x7ff9100fcef0 msgr2=0x7ff9100ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- 172.21.15.77:0/1753709086 shutdown_connections 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- 172.21.15.77:0/1753709086 wait complete. 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 Processor -- start 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- start start 2024-04-26T04:43:35.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.789+0000 7ff916457700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff910114290 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33718/0 (socket says 172.21.15.77:33718) 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90ffff700 1 -- 172.21.15.77:0/3643359068 learned_addr learned my addr 172.21.15.77:0/3643359068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90ffff700 1 -- 172.21.15.77:0/3643359068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000094f0 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90ffff700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b440 tx=0x7ff900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff900010760 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff900010d50 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9101143d0 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90001ae20 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.790+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101148c0 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.791+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7ff900024450 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.791+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91004bed0 con 0x7ff910101730 2024-04-26T04:43:35.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.791+0000 7ff90cff9700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 0x7ff8f8038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.791+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff90004a5b0 con 0x7ff910101730 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.791+0000 7ff90f7fe700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 0x7ff8f8038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.792+0000 7ff90f7fe700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 0x7ff8f8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff904009990 tx=0x7ff904006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.796+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff900010aa0 con 0x7ff910101730 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:34.996+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7ff910114b00 con 0x7ff8f8036720 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.003+0000 7ff90cff9700 1 -- 172.21.15.77:0/3643359068 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff910114b00 con 0x7ff8f8036720 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 msgr2=0x7ff8f8038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 0x7ff8f8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff904009990 tx=0x7ff904006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 msgr2=0x7ff910076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b440 tx=0x7ff900009250 comp rx=0 tx=0).stop 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 shutdown_connections 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff8f8036720 0x7ff8f8038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 --2- 172.21.15.77:0/3643359068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff910101730 0x7ff910076b60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:35.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 >> 172.21.15.77:0/3643359068 conn(0x7ff9100fcef0 msgr2=0x7ff9100fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:35.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 shutdown_connections 2024-04-26T04:43:35.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:35.006+0000 7ff916457700 1 -- 172.21.15.77:0/3643359068 wait complete. 2024-04-26T04:43:36.600 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:36 smithi077 ceph-mon[26311]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:36.600 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:36 smithi077 ceph-mon[26311]: mgrmap e8: smithi077.hkvsor(active, since 2s) 2024-04-26T04:43:36.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T04:43:36.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6f2155700 1 Processor -- start 2024-04-26T04:43:36.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6f2155700 1 -- start start 2024-04-26T04:43:36.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6f2155700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:36.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6f2155700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec079130 con 0x7fa6ec078d20 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33722/0 (socket says 172.21.15.77:33722) 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.055+0000 7fa6eb7fe700 1 -- 172.21.15.77:0/3035721706 learned_addr learned my addr 172.21.15.77:0/3035721706 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6eb7fe700 1 -- 172.21.15.77:0/3035721706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec077770 con 0x7fa6ec078d20 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6eb7fe700 1 --2- 172.21.15.77:0/3035721706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009e50 tx=0x7fa6dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f7dc17db48acbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6ea7fc700 1 -- 172.21.15.77:0/3035721706 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6dc0092e0 con 0x7fa6ec078d20 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6ea7fc700 1 -- 172.21.15.77:0/3035721706 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa6dc012430 con 0x7fa6ec078d20 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6ea7fc700 1 -- 172.21.15.77:0/3035721706 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6dc0075c0 con 0x7fa6ec078d20 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6f2155700 1 -- 172.21.15.77:0/3035721706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 msgr2=0x7fa6ec077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.056+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3035721706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009e50 tx=0x7fa6dc009840 comp rx=0 tx=0).stop 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- 172.21.15.77:0/3035721706 shutdown_connections 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3035721706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec077230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:36.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- 172.21.15.77:0/3035721706 >> 172.21.15.77:0/3035721706 conn(0x7fa6ec0fdfd0 msgr2=0x7fa6ec1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:36.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- 172.21.15.77:0/3035721706 shutdown_connections 2024-04-26T04:43:36.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- 172.21.15.77:0/3035721706 wait complete. 2024-04-26T04:43:36.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 Processor -- start 2024-04-26T04:43:36.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- start start 2024-04-26T04:43:36.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.057+0000 7fa6f2155700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec113e70 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6eb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33734/0 (socket says 172.21.15.77:33734) 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6eb7fe700 1 -- 172.21.15.77:0/3567343901 learned_addr learned my addr 172.21.15.77:0/3567343901 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6eb7fe700 1 -- 172.21.15.77:0/3567343901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc009ae0 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6eb7fe700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc012da0 tx=0x7fa6dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6dc0092e0 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa6dc007d10 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec114070 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6dc01c880 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.058+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec114be0 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.059+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa6dc019070 con 0x7fa6ec078d20 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.059+0000 7fa6d3fff700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 0x7fa6d4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:36.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.059+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa6dc04ac00 con 0x7fa6ec078d20 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.059+0000 7fa6eaffd700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 0x7fa6d4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.059+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec05f5c0 con 0x7fa6ec078d20 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.060+0000 7fa6eaffd700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 0x7fa6d4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009990 tx=0x7fa6e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.064+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa6dc017080 con 0x7fa6ec078d20 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.247+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6ec0ff310 con 0x7fa6d4036850 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.248+0000 7fa6d3fff700 1 -- 172.21.15.77:0/3567343901 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa6ec0ff310 con 0x7fa6d4036850 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.249+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 msgr2=0x7fa6d4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.249+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 0x7fa6d4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009990 tx=0x7fa6e0006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.249+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 msgr2=0x7fa6ec113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.249+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc012da0 tx=0x7fa6dc009840 comp rx=0 tx=0).stop 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 shutdown_connections 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fa6d4036850 0x7fa6d4038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 --2- 172.21.15.77:0/3567343901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6ec078d20 0x7fa6ec113930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:36.921 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 >> 172.21.15.77:0/3567343901 conn(0x7fa6ec0fdfd0 msgr2=0x7fa6ec0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:36.922 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 shutdown_connections 2024-04-26T04:43:36.922 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:36.250+0000 7fa6f2155700 1 -- 172.21.15.77:0/3567343901 wait complete. 2024-04-26T04:43:36.922 INFO:teuthology.orchestra.run.smithi077.stdout:Generating ssh key... 2024-04-26T04:43:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:37 smithi077 ceph-mon[26311]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:38.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d9b14700 1 Processor -- start 2024-04-26T04:43:38.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d9b14700 1 -- start start 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d9b14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d9b14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d41031d0 con 0x7fc3d4102880 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33738/0 (socket says 172.21.15.77:33738) 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d37fe700 1 -- 172.21.15.77:0/3226684528 learned_addr learned my addr 172.21.15.77:0/3226684528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d37fe700 1 -- 172.21.15.77:0/3226684528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4103310 con 0x7fc3d4102880 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.295+0000 7fc3d37fe700 1 --2- 172.21.15.77:0/3226684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40098e0 tx=0x7fc3c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854dc6edd5495d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d27fc700 1 -- 172.21.15.77:0/3226684528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c400b860 con 0x7fc3d4102880 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d27fc700 1 -- 172.21.15.77:0/3226684528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3c400be50 con 0x7fc3d4102880 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d27fc700 1 -- 172.21.15.77:0/3226684528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c4010380 con 0x7fc3d4102880 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 -- 172.21.15.77:0/3226684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 msgr2=0x7fc3d4102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:38.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/3226684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40098e0 tx=0x7fc3c4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 -- 172.21.15.77:0/3226684528 shutdown_connections 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/3226684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d4102c90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 -- 172.21.15.77:0/3226684528 >> 172.21.15.77:0/3226684528 conn(0x7fc3d40fe060 msgr2=0x7fc3d4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 -- 172.21.15.77:0/3226684528 shutdown_connections 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.296+0000 7fc3d9b14700 1 -- 172.21.15.77:0/3226684528 wait complete. 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d9b14700 1 Processor -- start 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d9b14700 1 -- start start 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d9b14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d9b14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d4111b30 con 0x7fc3d4102880 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33746/0 (socket says 172.21.15.77:33746) 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.297+0000 7fc3d37fe700 1 -- 172.21.15.77:0/2816389499 learned_addr learned my addr 172.21.15.77:0/2816389499 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3d37fe700 1 -- 172.21.15.77:0/2816389499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40094f0 con 0x7fc3d4102880 2024-04-26T04:43:38.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3d37fe700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c40107c0 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc3c4010db0 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d4111d30 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c401ad00 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.298+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d41128a0 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.299+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc3c40193c0 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.299+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d405f5c0 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.299+0000 7fc3bbfff700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 0x7fc3bc038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.299+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc3c404a650 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.299+0000 7fc3d2ffd700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 0x7fc3bc038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.300+0000 7fc3d2ffd700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 0x7fc3bc038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.304+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc3c4022930 con 0x7fc3d4102880 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.492+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d40ff200 con 0x7fc3bc0368a0 2024-04-26T04:43:38.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.699+0000 7fc3bbfff700 1 -- 172.21.15.77:0/2816389499 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3d40ff200 con 0x7fc3bc0368a0 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 msgr2=0x7fc3bc038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 0x7fc3bc038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c8006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 msgr2=0x7fc3d41115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 shutdown_connections 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc3bc0368a0 0x7fc3bc038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 --2- 172.21.15.77:0/2816389499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3d4102880 0x7fc3d41115f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 >> 172.21.15.77:0/2816389499 conn(0x7fc3d40fe060 msgr2=0x7fc3d40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 shutdown_connections 2024-04-26T04:43:38.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:37.701+0000 7fc3d9b14700 1 -- 172.21.15.77:0/2816389499 wait complete. 2024-04-26T04:43:38.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:38 smithi077 ceph-mon[26311]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:38.951 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:38 smithi077 ceph-mon[26311]: Generating ssh key... 2024-04-26T04:43:38.951 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:38 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:38.951 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:38 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:39.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcuVrs5ijiub4JGF9Ue4oXaXPGlNZKU22qe3E7PmeZTuKxUz8YAdv6QPpBLWQ+zu9pXTO9oEtKNfgOWdNKzTPcQgpDOrw5HgJpVAZarFSdHJBONetP9kF1sk27A290vA1YJISvQBHCN5Y5ohzK9aJ5afJuK6K519fDdtBnwl6I7TqM8F6cJ1V/A+/ttaZ4v53Hv0vWIJwKof0EG6oFHafBqi+44OAs9kC3sbK0yCRfu6ZCCSAL5+Gtmg9UI0a5392RHPhHELnk2KInOfq8q2UEoeK5yUVuwwOHm84w1XQjYkEfzTiKLwMWsAN/3eoKOaV7S/t0aOBwT6FyD1cJYidy+T4NrSdiWdX0K27dFSyTBpkCVqwUfT2K8F4jtwcbPfuNyDMmBB8JAaZ8gdpmv1OUIhQWADEQp1v9Veh6LwgHPPLVnhWTnpZhfCQo3esx0QJMkRvcXJprTrbw8g5neD8GXSmDDWF45ALzd4s06z5Q46g8PBng0L1PSNsxMRrpsA0= ceph-3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:43:39.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5d0142d700 1 Processor -- start 2024-04-26T04:43:39.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5d0142d700 1 -- start start 2024-04-26T04:43:39.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5d0142d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5d0142d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1031d0 con 0x7f5cfc102880 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39312/0 (socket says 172.21.15.77:39312) 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.753+0000 7f5cfaffd700 1 -- 172.21.15.77:0/391880842 learned_addr learned my addr 172.21.15.77:0/391880842 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.754+0000 7f5cfaffd700 1 -- 172.21.15.77:0/391880842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc103310 con 0x7f5cfc102880 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.754+0000 7f5cfaffd700 1 --2- 172.21.15.77:0/391880842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098e0 tx=0x7f5cec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86811026ecbec28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.754+0000 7f5cfa7fc700 1 -- 172.21.15.77:0/391880842 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cec00b860 con 0x7f5cfc102880 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.754+0000 7f5cfa7fc700 1 -- 172.21.15.77:0/391880842 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5cec00be50 con 0x7f5cfc102880 2024-04-26T04:43:39.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.754+0000 7f5cfa7fc700 1 -- 172.21.15.77:0/391880842 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cec010380 con 0x7f5cfc102880 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- 172.21.15.77:0/391880842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 msgr2=0x7f5cfc102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 --2- 172.21.15.77:0/391880842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098e0 tx=0x7f5cec009250 comp rx=0 tx=0).stop 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- 172.21.15.77:0/391880842 shutdown_connections 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 --2- 172.21.15.77:0/391880842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc102c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- 172.21.15.77:0/391880842 >> 172.21.15.77:0/391880842 conn(0x7f5cfc0fe060 msgr2=0x7f5cfc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- 172.21.15.77:0/391880842 shutdown_connections 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- 172.21.15.77:0/391880842 wait complete. 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 Processor -- start 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.755+0000 7f5d0142d700 1 -- start start 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5d0142d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5d0142d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc111b30 con 0x7f5cfc102880 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39320/0 (socket says 172.21.15.77:39320) 2024-04-26T04:43:39.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cfaffd700 1 -- 172.21.15.77:0/3848697956 learned_addr learned my addr 172.21.15.77:0/3848697956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cfaffd700 1 -- 172.21.15.77:0/3848697956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0094f0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cfaffd700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cec0107c0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5cec010db0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc111d30 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cec01ad00 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.756+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc1128a0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.757+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f5cec0193c0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.757+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc05f5c0 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.757+0000 7f5cf3fff700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 0x7f5ce8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.757+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5cec04a650 con 0x7f5cfc102880 2024-04-26T04:43:39.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.757+0000 7f5cf27fc700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 0x7f5ce8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.758+0000 7f5cf27fc700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 0x7f5ce8038d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009990 tx=0x7f5ce4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.762+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5cec022930 con 0x7f5cfc102880 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.950+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cfc0ff200 con 0x7f5ce8036850 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.950+0000 7f5cf3fff700 1 -- 172.21.15.77:0/3848697956 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f5cfc0ff200 con 0x7f5ce8036850 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 msgr2=0x7f5ce8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 0x7f5ce8038d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009990 tx=0x7f5ce4006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 msgr2=0x7f5cfc1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec009250 comp rx=0 tx=0).stop 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 shutdown_connections 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f5ce8036850 0x7f5ce8038d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 --2- 172.21.15.77:0/3848697956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cfc102880 0x7f5cfc1115f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 >> 172.21.15.77:0/3848697956 conn(0x7f5cfc0fe060 msgr2=0x7f5cfc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 shutdown_connections 2024-04-26T04:43:39.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:38.952+0000 7f5d0142d700 1 -- 172.21.15.77:0/3848697956 wait complete. 2024-04-26T04:43:39.508 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T04:43:39.508 INFO:teuthology.orchestra.run.smithi077.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T04:43:39.508 INFO:teuthology.orchestra.run.smithi077.stdout:Adding host smithi077... 2024-04-26T04:43:39.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:39 smithi077 ceph-mon[26311]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:40.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:40 smithi077 ceph-mon[26311]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:41.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:41 smithi077 ceph-mon[26311]: Deploying cephadm binary to smithi077 2024-04-26T04:43:42.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Added host 'smithi077' with addr '172.21.15.77' 2024-04-26T04:43:42.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.912+0000 7f7813f5a700 1 Processor -- start 2024-04-26T04:43:42.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.912+0000 7f7813f5a700 1 -- start start 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.912+0000 7f7813f5a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7813f5a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c105980 con 0x7f780c076c10 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7811cf6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7811cf6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39324/0 (socket says 172.21.15.77:39324) 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7811cf6700 1 -- 172.21.15.77:0/2210599380 learned_addr learned my addr 172.21.15.77:0/2210599380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7811cf6700 1 -- 172.21.15.77:0/2210599380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c105ac0 con 0x7f780c076c10 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7811cf6700 1 --2- 172.21.15.77:0/2210599380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc9c00e0d9b6f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7810cf4700 1 -- 172.21.15.77:0/2210599380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77fc00b860 con 0x7f780c076c10 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7810cf4700 1 -- 172.21.15.77:0/2210599380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f77fc00be50 con 0x7f780c076c10 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.913+0000 7f7810cf4700 1 -- 172.21.15.77:0/2210599380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77fc010380 con 0x7f780c076c10 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 -- 172.21.15.77:0/2210599380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 msgr2=0x7f780c105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 --2- 172.21.15.77:0/2210599380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc009250 comp rx=0 tx=0).stop 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 -- 172.21.15.77:0/2210599380 shutdown_connections 2024-04-26T04:43:42.504 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 --2- 172.21.15.77:0/2210599380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c105440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 -- 172.21.15.77:0/2210599380 >> 172.21.15.77:0/2210599380 conn(0x7f780c0fcef0 msgr2=0x7f780c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 -- 172.21.15.77:0/2210599380 shutdown_connections 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 -- 172.21.15.77:0/2210599380 wait complete. 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.914+0000 7f7813f5a700 1 Processor -- start 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7813f5a700 1 -- start start 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7813f5a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7813f5a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1032d0 con 0x7f780c076c10 2024-04-26T04:43:42.505 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7811cf6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7811cf6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39338/0 (socket says 172.21.15.77:39338) 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7811cf6700 1 -- 172.21.15.77:0/3606388555 learned_addr learned my addr 172.21.15.77:0/3606388555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7811cf6700 1 -- 172.21.15.77:0/3606388555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0094f0 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7811cf6700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77fc003050 tx=0x7f77fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77fc010760 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f77fc010d50 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c1013a0 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77fc01ae20 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.915+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c101890 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.916+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f77fc0108c0 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.916+0000 7f780a7fc700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 0x7f77f4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.916+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f77fc04a480 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.916+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c04bed0 con 0x7f780c076c10 2024-04-26T04:43:42.506 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.916+0000 7f78114f5700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 0x7f77f4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.917+0000 7f78114f5700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 0x7f77f4038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7800009990 tx=0x7f7800006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:39.921+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f77fc01a430 con 0x7f780c076c10 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:40.110+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}) v1 -- 0x7f780c101f70 con 0x7f77f40368a0 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.939+0000 7f780a7fc700 1 -- 172.21.15.77:0/3606388555 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f780c101f70 con 0x7f77f40368a0 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 msgr2=0x7f77f4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 0x7f77f4038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7800009990 tx=0x7f7800006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 msgr2=0x7f780c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77fc003050 tx=0x7f77fc009250 comp rx=0 tx=0).stop 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 shutdown_connections 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f77f40368a0 0x7f77f4038d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 --2- 172.21.15.77:0/3606388555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f780c076c10 0x7f780c102d90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 >> 172.21.15.77:0/3606388555 conn(0x7f780c0fcef0 msgr2=0x7f780c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 shutdown_connections 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:41.942+0000 7f7813f5a700 1 -- 172.21.15.77:0/3606388555 wait complete. 2024-04-26T04:43:42.507 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying mon service with default placement... 2024-04-26T04:43:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:42 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:42 smithi077 ceph-mon[26311]: Added host smithi077 2024-04-26T04:43:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:42 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddbc944700 1 Processor -- start 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddbc944700 1 -- start start 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddbc944700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddbc944700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8079150 con 0x7fddb8078d40 2024-04-26T04:43:43.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddb659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddb659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39352/0 (socket says 172.21.15.77:39352) 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.907+0000 7fddb659c700 1 -- 172.21.15.77:0/2602704620 learned_addr learned my addr 172.21.15.77:0/2602704620 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddb659c700 1 -- 172.21.15.77:0/2602704620 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8077790 con 0x7fddb8078d40 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddb659c700 1 --2- 172.21.15.77:0/2602704620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098e0 tx=0x7fdda8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccbfc9d153ebef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddb559a700 1 -- 172.21.15.77:0/2602704620 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda800b860 con 0x7fddb8078d40 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddb559a700 1 -- 172.21.15.77:0/2602704620 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdda800be50 con 0x7fddb8078d40 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddb559a700 1 -- 172.21.15.77:0/2602704620 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda8010380 con 0x7fddb8078d40 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddbc944700 1 -- 172.21.15.77:0/2602704620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 msgr2=0x7fddb8077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddbc944700 1 --2- 172.21.15.77:0/2602704620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098e0 tx=0x7fdda8009250 comp rx=0 tx=0).stop 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddbc944700 1 -- 172.21.15.77:0/2602704620 shutdown_connections 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddbc944700 1 --2- 172.21.15.77:0/2602704620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8077250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.908+0000 7fddbc944700 1 -- 172.21.15.77:0/2602704620 >> 172.21.15.77:0/2602704620 conn(0x7fddb80fdfd0 msgr2=0x7fddb81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:43.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 -- 172.21.15.77:0/2602704620 shutdown_connections 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 -- 172.21.15.77:0/2602704620 wait complete. 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 Processor -- start 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 -- start start 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddbc944700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8113db0 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddb659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddb659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39354/0 (socket says 172.21.15.77:39354) 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddb659c700 1 -- 172.21.15.77:0/3433800503 learned_addr learned my addr 172.21.15.77:0/3433800503 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.909+0000 7fddb659c700 1 -- 172.21.15.77:0/3433800503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80094f0 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fddb659c700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdda80090b0 tx=0x7fdda8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda8010760 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdda8010d50 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb8113fb0 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda801ace0 con 0x7fddb8078d40 2024-04-26T04:43:43.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.910+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb8114b20 con 0x7fddb8078d40 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdda802d3f0 con 0x7fddb8078d40 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb805f5c0 con 0x7fddb8078d40 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fdda6ffd700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 0x7fdda0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdda804a310 con 0x7fddb8078d40 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fddb5d9b700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 0x7fdda0038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.911+0000 7fddb5d9b700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 0x7fdda0038d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fddac009990 tx=0x7fddac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:42.916+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdda802b2f0 con 0x7fddb8078d40 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.116+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fddb80ff230 con 0x7fdda00368a0 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.123+0000 7fdda6ffd700 1 -- 172.21.15.77:0/3433800503 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fddb80ff230 con 0x7fdda00368a0 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.125+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 msgr2=0x7fdda0038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.125+0000 7fddbc944700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 0x7fdda0038d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fddac009990 tx=0x7fddac006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.125+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 msgr2=0x7fddb8113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.125+0000 7fddbc944700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdda80090b0 tx=0x7fdda8009250 comp rx=0 tx=0).stop 2024-04-26T04:43:43.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 shutdown_connections 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdda00368a0 0x7fdda0038d50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 --2- 172.21.15.77:0/3433800503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8078d40 0x7fddb8113870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 >> 172.21.15.77:0/3433800503 conn(0x7fddb80fdfd0 msgr2=0x7fddb80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 shutdown_connections 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:43.126+0000 7fddbc944700 1 -- 172.21.15.77:0/3433800503 wait complete. 2024-04-26T04:43:43.750 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying mgr service with default placement... 2024-04-26T04:43:44.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:44 smithi077 ceph-mon[26311]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:44 smithi077 ceph-mon[26311]: Saving service mon spec with placement count:5 2024-04-26T04:43:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:44 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:46.057 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:45 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:46.057 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:45 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:46.057 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:45 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.270+0000 7f39dfacb700 1 Processor -- start 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dfacb700 1 -- start start 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dfacb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dfacb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d81031e0 con 0x7f39d8102890 2024-04-26T04:43:46.586 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dd867700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dd867700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39362/0 (socket says 172.21.15.77:39362) 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dd867700 1 -- 172.21.15.77:0/3407730346 learned_addr learned my addr 172.21.15.77:0/3407730346 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.271+0000 7f39dd867700 1 -- 172.21.15.77:0/3407730346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8103320 con 0x7f39d8102890 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dd867700 1 --2- 172.21.15.77:0/3407730346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39d40098e0 tx=0x7f39d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6710f6a9794bc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dc865700 1 -- 172.21.15.77:0/3407730346 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d400b860 con 0x7f39d8102890 2024-04-26T04:43:46.587 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dc865700 1 -- 172.21.15.77:0/3407730346 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39d400be90 con 0x7f39d8102890 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dc865700 1 -- 172.21.15.77:0/3407730346 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d40103c0 con 0x7f39d8102890 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dfacb700 1 -- 172.21.15.77:0/3407730346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 msgr2=0x7f39d8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dfacb700 1 --2- 172.21.15.77:0/3407730346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39d40098e0 tx=0x7f39d4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dfacb700 1 -- 172.21.15.77:0/3407730346 shutdown_connections 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dfacb700 1 --2- 172.21.15.77:0/3407730346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8102ca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.272+0000 7f39dfacb700 1 -- 172.21.15.77:0/3407730346 >> 172.21.15.77:0/3407730346 conn(0x7f39d80fe010 msgr2=0x7f39d8100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 -- 172.21.15.77:0/3407730346 shutdown_connections 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 -- 172.21.15.77:0/3407730346 wait complete. 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 Processor -- start 2024-04-26T04:43:46.588 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 -- start start 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dfacb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8079160 con 0x7f39d8102890 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dd867700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dd867700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39364/0 (socket says 172.21.15.77:39364) 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.273+0000 7f39dd867700 1 -- 172.21.15.77:0/2053774310 learned_addr learned my addr 172.21.15.77:0/2053774310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39dd867700 1 -- 172.21.15.77:0/2053774310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40094f0 con 0x7f39d8102890 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39dd867700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39d40090b0 tx=0x7f39d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d4010840 con 0x7f39d8102890 2024-04-26T04:43:46.589 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39d4010e70 con 0x7f39d8102890 2024-04-26T04:43:46.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d80771d0 con 0x7f39d8102890 2024-04-26T04:43:46.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d80776c0 con 0x7f39d8102890 2024-04-26T04:43:46.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.274+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d4024e40 con 0x7f39d8102890 2024-04-26T04:43:46.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.275+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f39d401a570 con 0x7f39d8102890 2024-04-26T04:43:46.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.275+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc0052e0 con 0x7f39d8102890 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.275+0000 7f39ce7fc700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 0x7f39c4038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.275+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f39d404a780 con 0x7f39d8102890 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.275+0000 7f39dd066700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 0x7f39c4038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.276+0000 7f39dd066700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 0x7f39c4038cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009990 tx=0x7f39c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.280+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f39d4018940 con 0x7f39d8102890 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.508+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f39bc000bf0 con 0x7f39c4036800 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.513+0000 7f39ce7fc700 1 -- 172.21.15.77:0/2053774310 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f39bc000bf0 con 0x7f39c4036800 2024-04-26T04:43:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 msgr2=0x7f39c4038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 0x7f39c4038cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009990 tx=0x7f39c8006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 msgr2=0x7f39d8078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39d40090b0 tx=0x7f39d4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 shutdown_connections 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f39c4036800 0x7f39c4038cb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 --2- 172.21.15.77:0/2053774310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39d8102890 0x7f39d8078c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 >> 172.21.15.77:0/2053774310 conn(0x7f39d80fe010 msgr2=0x7f39d80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 shutdown_connections 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:45.515+0000 7f39dfacb700 1 -- 172.21.15.77:0/2053774310 wait complete. 2024-04-26T04:43:46.644 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying crash service with default placement... 2024-04-26T04:43:47.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:47 smithi077 ceph-mon[26311]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:47.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:47 smithi077 ceph-mon[26311]: Saving service mgr spec with placement count:2 2024-04-26T04:43:47.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:47 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3cffa700 1 Processor -- start 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3cffa700 1 -- start start 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3cffa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3cffa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38070d90 con 0x7fbc38103ef0 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39372/0 (socket says 172.21.15.77:39372) 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.984+0000 7fbc3659c700 1 -- 172.21.15.77:0/264357799 learned_addr learned my addr 172.21.15.77:0/264357799 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3659c700 1 -- 172.21.15.77:0/264357799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38104840 con 0x7fbc38103ef0 2024-04-26T04:43:47.744 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3659c700 1 --2- 172.21.15.77:0/264357799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3701ac6bc59949 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3559a700 1 -- 172.21.15.77:0/264357799 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc2800b860 con 0x7fbc38103ef0 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3559a700 1 -- 172.21.15.77:0/264357799 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc2800be90 con 0x7fbc38103ef0 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3559a700 1 -- 172.21.15.77:0/264357799 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc280103c0 con 0x7fbc38103ef0 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3cffa700 1 -- 172.21.15.77:0/264357799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 msgr2=0x7fbc38104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.985+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/264357799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098e0 tx=0x7fbc28009250 comp rx=0 tx=0).stop 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- 172.21.15.77:0/264357799 shutdown_connections 2024-04-26T04:43:47.745 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/264357799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38104300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- 172.21.15.77:0/264357799 >> 172.21.15.77:0/264357799 conn(0x7fbc380fc6f0 msgr2=0x7fbc380feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- 172.21.15.77:0/264357799 shutdown_connections 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- 172.21.15.77:0/264357799 wait complete. 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 Processor -- start 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- start start 2024-04-26T04:43:47.746 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3cffa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38103b40 con 0x7fbc38103ef0 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39386/0 (socket says 172.21.15.77:39386) 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.986+0000 7fbc3659c700 1 -- 172.21.15.77:0/2287044514 learned_addr learned my addr 172.21.15.77:0/2287044514 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc3659c700 1 -- 172.21.15.77:0/2287044514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280094f0 con 0x7fbc38103ef0 2024-04-26T04:43:47.747 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc3659c700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc28011680 tx=0x7fbc28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc28010890 con 0x7fbc38103ef0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc28010ec0 con 0x7fbc38103ef0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc38101bb0 con 0x7fbc38103ef0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc280243f0 con 0x7fbc38103ef0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.987+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc381020a0 con 0x7fbc38103ef0 2024-04-26T04:43:47.748 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fbc280109f0 con 0x7fbc38103ef0 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3804bed0 con 0x7fbc38103ef0 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc26ffd700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 0x7fbc20038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbc2804a230 con 0x7fbc38103ef0 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc35d9b700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 0x7fbc20038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.988+0000 7fbc35d9b700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 0x7fbc20038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009990 tx=0x7fbc2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:47.749 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:46.993+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbc28006d70 con 0x7fbc38103ef0 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.192+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc381027b0 con 0x7fbc20036850 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.196+0000 7fbc26ffd700 1 -- 172.21.15.77:0/2287044514 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fbc381027b0 con 0x7fbc20036850 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 msgr2=0x7fbc20038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 0x7fbc20038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009990 tx=0x7fbc2c006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 msgr2=0x7fbc38103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:47.750 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc28011680 tx=0x7fbc28009250 comp rx=0 tx=0).stop 2024-04-26T04:43:47.751 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 shutdown_connections 2024-04-26T04:43:47.751 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fbc20036850 0x7fbc20038d00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:47.751 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 --2- 172.21.15.77:0/2287044514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc38103ef0 0x7fbc38103600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:47.751 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.198+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 >> 172.21.15.77:0/2287044514 conn(0x7fbc380fc6f0 msgr2=0x7fbc380fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:47.752 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.199+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 shutdown_connections 2024-04-26T04:43:47.752 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:47.199+0000 7fbc3cffa700 1 -- 172.21.15.77:0/2287044514 wait complete. 2024-04-26T04:43:47.752 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying prometheus service with default placement... 2024-04-26T04:43:48.401 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:48 smithi077 ceph-mon[26311]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:48.401 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:48 smithi077 ceph-mon[26311]: Saving service crash spec with placement * 2024-04-26T04:43:48.401 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:48 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:48.401 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:48 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.159+0000 7fc6dc473700 1 Processor -- start 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.159+0000 7fc6dc473700 1 -- start start 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6dc473700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6dc473700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4103160 con 0x7fc6d4102810 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6da20f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6da20f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43664/0 (socket says 172.21.15.77:43664) 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6da20f700 1 -- 172.21.15.77:0/261637857 learned_addr learned my addr 172.21.15.77:0/261637857 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6da20f700 1 -- 172.21.15.77:0/261637857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d41032a0 con 0x7fc6d4102810 2024-04-26T04:43:48.976 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6da20f700 1 --2- 172.21.15.77:0/261637857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009e50 tx=0x7fc6cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c1394e0830e95f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6d920d700 1 -- 172.21.15.77:0/261637857 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6cc0092e0 con 0x7fc6d4102810 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6d920d700 1 -- 172.21.15.77:0/261637857 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6cc012430 con 0x7fc6d4102810 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.160+0000 7fc6d920d700 1 -- 172.21.15.77:0/261637857 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6cc0075a0 con 0x7fc6d4102810 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- 172.21.15.77:0/261637857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 msgr2=0x7fc6d4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 --2- 172.21.15.77:0/261637857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009e50 tx=0x7fc6cc009840 comp rx=0 tx=0).stop 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- 172.21.15.77:0/261637857 shutdown_connections 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 --2- 172.21.15.77:0/261637857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4102c20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- 172.21.15.77:0/261637857 >> 172.21.15.77:0/261637857 conn(0x7fc6d40fdfd0 msgr2=0x7fc6d41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- 172.21.15.77:0/261637857 shutdown_connections 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- 172.21.15.77:0/261637857 wait complete. 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 Processor -- start 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.161+0000 7fc6dc473700 1 -- start start 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6dc473700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6dc473700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4079160 con 0x7fc6d4102810 2024-04-26T04:43:48.977 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6da20f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6da20f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43670/0 (socket says 172.21.15.77:43670) 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6da20f700 1 -- 172.21.15.77:0/3036614566 learned_addr learned my addr 172.21.15.77:0/3036614566 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6da20f700 1 -- 172.21.15.77:0/3036614566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009ae0 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6da20f700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc012de0 tx=0x7fc6cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6cc0092e0 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6cc007ce0 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d40771d0 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6cc025930 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.162+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d40776c0 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.163+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc6cc019050 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.163+0000 7fc6caffd700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 0x7fc6c0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.163+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc6cc04a180 con 0x7fc6d4102810 2024-04-26T04:43:48.978 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.163+0000 7fc6d9a0e700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 0x7fc6c0038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.163+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b80052e0 con 0x7fc6d4102810 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.164+0000 7fc6d9a0e700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 0x7fc6c0038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009990 tx=0x7fc6c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.168+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc6cc020070 con 0x7fc6d4102810 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.374+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6b8000bf0 con 0x7fc6c0036850 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.378+0000 7fc6caffd700 1 -- 172.21.15.77:0/3036614566 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc6b8000bf0 con 0x7fc6c0036850 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 msgr2=0x7fc6c0038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 0x7fc6c0038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009990 tx=0x7fc6c4006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 msgr2=0x7fc6d4078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc012de0 tx=0x7fc6cc009840 comp rx=0 tx=0).stop 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 shutdown_connections 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fc6c0036850 0x7fc6c0038d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 --2- 172.21.15.77:0/3036614566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6d4102810 0x7fc6d4078c20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 >> 172.21.15.77:0/3036614566 conn(0x7fc6d40fdfd0 msgr2=0x7fc6d40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 shutdown_connections 2024-04-26T04:43:48.979 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:48.380+0000 7fc6dc473700 1 -- 172.21.15.77:0/3036614566 wait complete. 2024-04-26T04:43:48.980 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying grafana service with default placement... 2024-04-26T04:43:49.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:49 smithi077 ceph-mon[26311]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:49 smithi077 ceph-mon[26311]: Saving service prometheus spec with placement count:1 2024-04-26T04:43:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:49 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:50.905 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:50 smithi077 ceph-mon[26311]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:50.905 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:50 smithi077 ceph-mon[26311]: Saving service grafana spec with placement count:1 2024-04-26T04:43:50.905 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:50 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbc068b700 1 Processor -- start 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbc068b700 1 -- start start 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbc068b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbc068b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8103160 con 0x7fdbb8102810 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbbe427700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbbe427700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43686/0 (socket says 172.21.15.77:43686) 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbbe427700 1 -- 172.21.15.77:0/971630724 learned_addr learned my addr 172.21.15.77:0/971630724 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:50.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.436+0000 7fdbbe427700 1 -- 172.21.15.77:0/971630724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb81032a0 con 0x7fdbb8102810 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbbe427700 1 --2- 172.21.15.77:0/971630724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40098e0 tx=0x7fdbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c5fb70c8e6eb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbbd425700 1 -- 172.21.15.77:0/971630724 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbb400b860 con 0x7fdbb8102810 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbbd425700 1 -- 172.21.15.77:0/971630724 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbb400be90 con 0x7fdbb8102810 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbbd425700 1 -- 172.21.15.77:0/971630724 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbb40103c0 con 0x7fdbb8102810 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 -- 172.21.15.77:0/971630724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 msgr2=0x7fdbb8102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 --2- 172.21.15.77:0/971630724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40098e0 tx=0x7fdbb4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 -- 172.21.15.77:0/971630724 shutdown_connections 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 --2- 172.21.15.77:0/971630724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8102c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 -- 172.21.15.77:0/971630724 >> 172.21.15.77:0/971630724 conn(0x7fdbb80fdfd0 msgr2=0x7fdbb81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:50.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 -- 172.21.15.77:0/971630724 shutdown_connections 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.437+0000 7fdbc068b700 1 -- 172.21.15.77:0/971630724 wait complete. 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbc068b700 1 Processor -- start 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbc068b700 1 -- start start 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbc068b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbc068b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8079160 con 0x7fdbb8102810 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbbe427700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbbe427700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43688/0 (socket says 172.21.15.77:43688) 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbbe427700 1 -- 172.21.15.77:0/1469177087 learned_addr learned my addr 172.21.15.77:0/1469177087 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbbe427700 1 -- 172.21.15.77:0/1469177087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40094f0 con 0x7fdbb8102810 2024-04-26T04:43:50.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.438+0000 7fdbbe427700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4011c10 tx=0x7fdbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.439+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbb4010890 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.439+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbb4010ec0 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.439+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbb40243f0 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.439+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb80771d0 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.439+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb80776c0 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdbb401a430 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb804bed0 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbaeffd700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 0x7fdba4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdbb404a210 con 0x7fdbb8102810 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbbdc26700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 0x7fdba4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:50.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.440+0000 7fdbbdc26700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 0x7fdba4038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009990 tx=0x7fdba8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.444+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdbb402b4d0 con 0x7fdbb8102810 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.646+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb8077dd0 con 0x7fdba40368a0 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.650+0000 7fdbaeffd700 1 -- 172.21.15.77:0/1469177087 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fdbb8077dd0 con 0x7fdba40368a0 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 msgr2=0x7fdba4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 0x7fdba4038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009990 tx=0x7fdba8006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 msgr2=0x7fdbb8078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4011c10 tx=0x7fdbb4009250 comp rx=0 tx=0).stop 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 shutdown_connections 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdba40368a0 0x7fdba4038d50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 --2- 172.21.15.77:0/1469177087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbb8102810 0x7fdbb8078c20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 >> 172.21.15.77:0/1469177087 conn(0x7fdbb80fdfd0 msgr2=0x7fdbb80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 shutdown_connections 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:49.652+0000 7fdbc068b700 1 -- 172.21.15.77:0/1469177087 wait complete. 2024-04-26T04:43:50.917 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying node-exporter service with default placement... 2024-04-26T04:43:52.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.396+0000 7fee301fb700 1 Processor -- start 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.396+0000 7fee301fb700 1 -- start start 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee301fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee301fb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28103230 con 0x7fee28076bf0 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee2df97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee2df97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43692/0 (socket says 172.21.15.77:43692) 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee2df97700 1 -- 172.21.15.77:0/828453714 learned_addr learned my addr 172.21.15.77:0/828453714 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee2df97700 1 -- 172.21.15.77:0/828453714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28107d80 con 0x7fee28076bf0 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.397+0000 7fee2df97700 1 --2- 172.21.15.77:0/828453714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee240098e0 tx=0x7fee24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894233258d5151ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee2cf95700 1 -- 172.21.15.77:0/828453714 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee2400b860 con 0x7fee28076bf0 2024-04-26T04:43:52.170 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee2cf95700 1 -- 172.21.15.77:0/828453714 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2400be90 con 0x7fee28076bf0 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee2cf95700 1 -- 172.21.15.77:0/828453714 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee240103c0 con 0x7fee28076bf0 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee301fb700 1 -- 172.21.15.77:0/828453714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 msgr2=0x7fee28107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee301fb700 1 --2- 172.21.15.77:0/828453714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee240098e0 tx=0x7fee24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee301fb700 1 -- 172.21.15.77:0/828453714 shutdown_connections 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee301fb700 1 --2- 172.21.15.77:0/828453714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28107840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.398+0000 7fee301fb700 1 -- 172.21.15.77:0/828453714 >> 172.21.15.77:0/828453714 conn(0x7fee280fce90 msgr2=0x7fee280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 -- 172.21.15.77:0/828453714 shutdown_connections 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 -- 172.21.15.77:0/828453714 wait complete. 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 Processor -- start 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 -- start start 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee301fb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28113d80 con 0x7fee28076bf0 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee2df97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee2df97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43694/0 (socket says 172.21.15.77:43694) 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.399+0000 7fee2df97700 1 -- 172.21.15.77:0/1159904849 learned_addr learned my addr 172.21.15.77:0/1159904849 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee2df97700 1 -- 172.21.15.77:0/1159904849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240094f0 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee2df97700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee240094c0 tx=0x7fee24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee24010890 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee28113f80 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee24010ec0 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee240243f0 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.400+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee28114af0 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.401+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fee2402d3f0 con 0x7fee28076bf0 2024-04-26T04:43:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.401+0000 7fee1e7fc700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 0x7fee14038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.401+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fee2404a5c0 con 0x7fee28076bf0 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.401+0000 7fee2d796700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 0x7fee14038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.401+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2805f5c0 con 0x7fee28076bf0 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.402+0000 7fee2d796700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 0x7fee14038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fee18009990 tx=0x7fee18006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.406+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fee24018b00 con 0x7fee28076bf0 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.609+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fee280fe0f0 con 0x7fee14036850 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.613+0000 7fee1e7fc700 1 -- 172.21.15.77:0/1159904849 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fee280fe0f0 con 0x7fee14036850 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 msgr2=0x7fee14038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 0x7fee14038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fee18009990 tx=0x7fee18006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 msgr2=0x7fee28113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee240094c0 tx=0x7fee24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 shutdown_connections 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fee14036850 0x7fee14038d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 --2- 172.21.15.77:0/1159904849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28076bf0 0x7fee28113840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 >> 172.21.15.77:0/1159904849 conn(0x7fee280fce90 msgr2=0x7fee280fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 shutdown_connections 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:51.616+0000 7fee301fb700 1 -- 172.21.15.77:0/1159904849 wait complete. 2024-04-26T04:43:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying alertmanager service with default placement... 2024-04-26T04:43:52.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:52 smithi077 ceph-mon[26311]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:52.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:52 smithi077 ceph-mon[26311]: Saving service node-exporter spec with placement * 2024-04-26T04:43:52.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:52 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:53.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-26T04:43:53.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.678+0000 7f116471f700 1 Processor -- start 2024-04-26T04:43:53.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.678+0000 7f116471f700 1 -- start start 2024-04-26T04:43:53.499 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f116471f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f116471f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c103680 con 0x7f115c102ca0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11624bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11624bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43704/0 (socket says 172.21.15.77:43704) 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11624bb700 1 -- 172.21.15.77:0/131677053 learned_addr learned my addr 172.21.15.77:0/131677053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11624bb700 1 -- 172.21.15.77:0/131677053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c1037c0 con 0x7f115c102ca0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11624bb700 1 --2- 172.21.15.77:0/131677053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f114c0098e0 tx=0x7f114c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7cfa9ec461367b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11614b9700 1 -- 172.21.15.77:0/131677053 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c00b860 con 0x7f115c102ca0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11614b9700 1 -- 172.21.15.77:0/131677053 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f114c00be90 con 0x7f115c102ca0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.679+0000 7f11614b9700 1 -- 172.21.15.77:0/131677053 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c0103c0 con 0x7f115c102ca0 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 -- 172.21.15.77:0/131677053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 msgr2=0x7f115c1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 --2- 172.21.15.77:0/131677053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f114c0098e0 tx=0x7f114c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 -- 172.21.15.77:0/131677053 shutdown_connections 2024-04-26T04:43:53.500 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 --2- 172.21.15.77:0/131677053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c1030b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 -- 172.21.15.77:0/131677053 >> 172.21.15.77:0/131677053 conn(0x7f115c0fc720 msgr2=0x7f115c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 -- 172.21.15.77:0/131677053 shutdown_connections 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 -- 172.21.15.77:0/131677053 wait complete. 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.680+0000 7f116471f700 1 Processor -- start 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f116471f700 1 -- start start 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f116471f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f116471f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c113d60 con 0x7f115c102ca0 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f11624bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f11624bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43710/0 (socket says 172.21.15.77:43710) 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f11624bb700 1 -- 172.21.15.77:0/1125551014 learned_addr learned my addr 172.21.15.77:0/1125551014 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f11624bb700 1 -- 172.21.15.77:0/1125551014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c0094f0 con 0x7f115c102ca0 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f11624bb700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f114c011c10 tx=0x7f114c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c010890 con 0x7f115c102ca0 2024-04-26T04:43:53.501 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c113f60 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f114c010ec0 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c0243f0 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.681+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c114ad0 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.682+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f114c02d3f0 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.682+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c05f5c0 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.682+0000 7f1152ffd700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 0x7f1148038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.682+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f114c04a250 con 0x7f115c102ca0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.682+0000 7f1161cba700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 0x7f1148038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.683+0000 7f1161cba700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 0x7f1148038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1158009990 tx=0x7f1158006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:53.502 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.687+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f114c02b8e0 con 0x7f115c102ca0 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.885+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f115c0fd9d0 con 0x7f1148036850 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.892+0000 7f1152ffd700 1 -- 172.21.15.77:0/1125551014 <== mgr.14118 v2:172.21.15.77:6800/3157890049 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f115c0fd9d0 con 0x7f1148036850 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.894+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 msgr2=0x7f1148038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.894+0000 7f116471f700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 0x7f1148038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1158009990 tx=0x7f1158006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.894+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 msgr2=0x7f115c113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.894+0000 7f116471f700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f114c011c10 tx=0x7f114c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 shutdown_connections 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f1148036850 0x7f1148038d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 --2- 172.21.15.77:0/1125551014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f115c102ca0 0x7f115c113820 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 >> 172.21.15.77:0/1125551014 conn(0x7f115c0fc720 msgr2=0x7f115c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 shutdown_connections 2024-04-26T04:43:53.503 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:52.895+0000 7f116471f700 1 -- 172.21.15.77:0/1125551014 wait complete. 2024-04-26T04:43:54.098 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:53 smithi077 ceph-mon[26311]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:43:54.099 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:53 smithi077 ceph-mon[26311]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:54.099 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:53 smithi077 ceph-mon[26311]: Saving service alertmanager spec with placement count:1 2024-04-26T04:43:54.099 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:53 smithi077 ceph-mon[26311]: from='mgr.14118 172.21.15.77:0/3506613443' entity='mgr.smithi077.hkvsor' 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f3941e1d700 1 Processor -- start 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f3941e1d700 1 -- start start 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f3941e1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f3941e1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c102080 con 0x7f393c101730 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43712/0 (socket says 172.21.15.77:43712) 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393b7fe700 1 -- 172.21.15.77:0/327528345 learned_addr learned my addr 172.21.15.77:0/327528345 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:54.722 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393b7fe700 1 -- 172.21.15.77:0/327528345 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c1021c0 con 0x7f393c101730 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393b7fe700 1 --2- 172.21.15.77:0/327528345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098e0 tx=0x7f392c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc01d43b68252d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.904+0000 7f393a7fc700 1 -- 172.21.15.77:0/327528345 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392c00b860 con 0x7f393c101730 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f393a7fc700 1 -- 172.21.15.77:0/327528345 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392c00be90 con 0x7f393c101730 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f393a7fc700 1 -- 172.21.15.77:0/327528345 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392c0103c0 con 0x7f393c101730 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 -- 172.21.15.77:0/327528345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 msgr2=0x7f393c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 --2- 172.21.15.77:0/327528345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098e0 tx=0x7f392c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 -- 172.21.15.77:0/327528345 shutdown_connections 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 --2- 172.21.15.77:0/327528345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c101b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 -- 172.21.15.77:0/327528345 >> 172.21.15.77:0/327528345 conn(0x7f393c0fcef0 msgr2=0x7f393c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 -- 172.21.15.77:0/327528345 shutdown_connections 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.905+0000 7f3941e1d700 1 -- 172.21.15.77:0/327528345 wait complete. 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f3941e1d700 1 Processor -- start 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f3941e1d700 1 -- start start 2024-04-26T04:43:54.723 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f3941e1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f3941e1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c113d20 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43720/0 (socket says 172.21.15.77:43720) 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f393b7fe700 1 -- 172.21.15.77:0/2877783696 learned_addr learned my addr 172.21.15.77:0/2877783696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.906+0000 7f393b7fe700 1 -- 172.21.15.77:0/2877783696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0094f0 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f393b7fe700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f392c0090b0 tx=0x7f392c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392c010840 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392c010e70 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c113f20 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392c024e40 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.907+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c114a90 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f392c0109a0 con 0x7f393c101730 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f3923fff700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 0x7f3924038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:54.724 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f392c04a710 con 0x7f393c101730 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c05f5c0 con 0x7f393c101730 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f393affd700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 0x7f3924038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.908+0000 7f393affd700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 0x7f3924038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3930009990 tx=0x7f3930006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:53.913+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f392c024710 con 0x7f393c101730 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.096+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f393c04bed0 con 0x7f393c101730 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.102+0000 7f3923fff700 1 -- 172.21.15.77:0/2877783696 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f392c02c340 con 0x7f393c101730 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.104+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 msgr2=0x7f3924038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.104+0000 7f3941e1d700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 0x7f3924038d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3930009990 tx=0x7f3930006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.104+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 msgr2=0x7f393c1137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.104+0000 7f3941e1d700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f392c0090b0 tx=0x7f392c009250 comp rx=0 tx=0).stop 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 shutdown_connections 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f3924036850 0x7f3924038d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 --2- 172.21.15.77:0/2877783696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f393c101730 0x7f393c1137e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:54.725 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 >> 172.21.15.77:0/2877783696 conn(0x7f393c0fcef0 msgr2=0x7f393c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:54.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 shutdown_connections 2024-04-26T04:43:54.726 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:54.105+0000 7f3941e1d700 1 -- 172.21.15.77:0/2877783696 wait complete. 2024-04-26T04:43:55.346 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:55 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2877783696' entity='client.admin' 2024-04-26T04:43:55.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93ce79700 1 Processor -- start 2024-04-26T04:43:55.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93ce79700 1 -- start start 2024-04-26T04:43:55.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93ce79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93ce79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938102080 con 0x7ff938101730 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43730/0 (socket says 172.21.15.77:43730) 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93659c700 1 -- 172.21.15.77:0/4144684194 learned_addr learned my addr 172.21.15.77:0/4144684194 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.149+0000 7ff93659c700 1 -- 172.21.15.77:0/4144684194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9381021c0 con 0x7ff938101730 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93659c700 1 --2- 172.21.15.77:0/4144684194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098e0 tx=0x7ff928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53ec171816193b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93559a700 1 -- 172.21.15.77:0/4144684194 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff92800b860 con 0x7ff938101730 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93559a700 1 -- 172.21.15.77:0/4144684194 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff92800be90 con 0x7ff938101730 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93559a700 1 -- 172.21.15.77:0/4144684194 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9280103c0 con 0x7ff938101730 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93ce79700 1 -- 172.21.15.77:0/4144684194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 msgr2=0x7ff938101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93ce79700 1 --2- 172.21.15.77:0/4144684194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098e0 tx=0x7ff928009250 comp rx=0 tx=0).stop 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93ce79700 1 -- 172.21.15.77:0/4144684194 shutdown_connections 2024-04-26T04:43:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93ce79700 1 --2- 172.21.15.77:0/4144684194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938101b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.150+0000 7ff93ce79700 1 -- 172.21.15.77:0/4144684194 >> 172.21.15.77:0/4144684194 conn(0x7ff9380fcef0 msgr2=0x7ff9380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 -- 172.21.15.77:0/4144684194 shutdown_connections 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 -- 172.21.15.77:0/4144684194 wait complete. 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 Processor -- start 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 -- start start 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93ce79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938111b60 con 0x7ff938101730 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43742/0 (socket says 172.21.15.77:43742) 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93659c700 1 -- 172.21.15.77:0/1057363091 learned_addr learned my addr 172.21.15.77:0/1057363091 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.151+0000 7ff93659c700 1 -- 172.21.15.77:0/1057363091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280094f0 con 0x7ff938101730 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff93659c700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9280090b0 tx=0x7ff928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff928010840 con 0x7ff938101730 2024-04-26T04:43:55.963 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff928010e70 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff928024e40 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff938111d60 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.152+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9381128d0 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff928024430 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93805f5c0 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff926ffd700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 0x7ff920038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff92804a700 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff935d9b700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 0x7ff920038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.153+0000 7ff935d9b700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 0x7ff920038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009990 tx=0x7ff92c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.157+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff92802c930 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.344+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff93804bed0 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.350+0000 7ff926ffd700 1 -- 172.21.15.77:0/1057363091 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff92802c340 con 0x7ff938101730 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 msgr2=0x7ff920038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:55.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 0x7ff920038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009990 tx=0x7ff92c006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 msgr2=0x7ff938111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9280090b0 tx=0x7ff928009250 comp rx=0 tx=0).stop 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 shutdown_connections 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7ff920036850 0x7ff920038d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 --2- 172.21.15.77:0/1057363091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff938101730 0x7ff938111620 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 >> 172.21.15.77:0/1057363091 conn(0x7ff9380fcef0 msgr2=0x7ff9380fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 shutdown_connections 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:55.352+0000 7ff93ce79700 1 -- 172.21.15.77:0/1057363091 wait complete. 2024-04-26T04:43:55.965 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling the dashboard module... 2024-04-26T04:43:56.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:56 smithi077 ceph-mon[26311]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:43:56.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:56 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1057363091' entity='client.admin' 2024-04-26T04:43:57.669 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:57 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1752571027' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd36c14700 1 Processor -- start 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd36c14700 1 -- start start 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd36c14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd36c14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301031d0 con 0x7fdd30102880 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd349b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd349b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43756/0 (socket says 172.21.15.77:43756) 2024-04-26T04:43:57.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd349b0700 1 -- 172.21.15.77:0/3839053999 learned_addr learned my addr 172.21.15.77:0/3839053999 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd349b0700 1 -- 172.21.15.77:0/3839053999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30103310 con 0x7fdd30102880 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.411+0000 7fdd349b0700 1 --2- 172.21.15.77:0/3839053999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd240098e0 tx=0x7fdd24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3e7895ffc3ff0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd2f7fe700 1 -- 172.21.15.77:0/3839053999 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd2400b860 con 0x7fdd30102880 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd2f7fe700 1 -- 172.21.15.77:0/3839053999 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd2400be90 con 0x7fdd30102880 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd2f7fe700 1 -- 172.21.15.77:0/3839053999 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd240103c0 con 0x7fdd30102880 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 -- 172.21.15.77:0/3839053999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 msgr2=0x7fdd30102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 --2- 172.21.15.77:0/3839053999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd240098e0 tx=0x7fdd24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 -- 172.21.15.77:0/3839053999 shutdown_connections 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 --2- 172.21.15.77:0/3839053999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30102c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 -- 172.21.15.77:0/3839053999 >> 172.21.15.77:0/3839053999 conn(0x7fdd300fe060 msgr2=0x7fdd30100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 -- 172.21.15.77:0/3839053999 shutdown_connections 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.412+0000 7fdd36c14700 1 -- 172.21.15.77:0/3839053999 wait complete. 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd36c14700 1 Processor -- start 2024-04-26T04:43:57.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd36c14700 1 -- start start 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd36c14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd36c14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd30111b60 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd349b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd349b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43764/0 (socket says 172.21.15.77:43764) 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd349b0700 1 -- 172.21.15.77:0/1752571027 learned_addr learned my addr 172.21.15.77:0/1752571027 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.413+0000 7fdd349b0700 1 -- 172.21.15.77:0/1752571027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240094f0 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd349b0700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd24011680 tx=0x7fdd24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd24010890 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd24010ec0 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd30111d60 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd240243f0 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.414+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd301128d0 con 0x7fdd30102880 2024-04-26T04:43:57.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdd2401a8c0 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3005f5c0 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd2d7fa700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 0x7fdd20038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdd2404a230 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd2ffff700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 0x7fdd20038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.415+0000 7fdd2ffff700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 0x7fdd20038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009990 tx=0x7fdd1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.419+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdd2402b8e0 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:56.659+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fdd3004bed0 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.356+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fdd2402d3f0 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.358+0000 7fdd2d7fa700 1 -- 172.21.15.77:0/1752571027 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fdd2402c530 con 0x7fdd30102880 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 msgr2=0x7fdd20038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 0x7fdd20038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009990 tx=0x7fdd1c006e40 comp rx=0 tx=0).stop 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 msgr2=0x7fdd30111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd24011680 tx=0x7fdd24009250 comp rx=0 tx=0).stop 2024-04-26T04:43:57.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 shutdown_connections 2024-04-26T04:43:57.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.361+0000 7fdd36c14700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7fdd20036850 0x7fdd20038d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:57.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.362+0000 7fdd36c14700 1 --2- 172.21.15.77:0/1752571027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd30102880 0x7fdd30111620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:57.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.362+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 >> 172.21.15.77:0/1752571027 conn(0x7fdd300fe060 msgr2=0x7fdd300feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:57.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.362+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 shutdown_connections 2024-04-26T04:43:57.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:57.362+0000 7fdd36c14700 1 -- 172.21.15.77:0/1752571027 wait complete. 2024-04-26T04:43:58.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:58 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1752571027' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T04:43:58.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:58 smithi077 ceph-mon[26311]: mgrmap e9: smithi077.hkvsor(active, since 24s) 2024-04-26T04:43:59.160 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:43:59.160 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T04:43:59.160 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "smithi077.hkvsor", 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f6748d1b700 1 Processor -- start 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f6748d1b700 1 -- start start 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f6748d1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f6748d1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744103280 con 0x7f6744071290 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f674259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f674259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47404/0 (socket says 172.21.15.77:47404) 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f674259c700 1 -- 172.21.15.77:0/658101431 learned_addr learned my addr 172.21.15.77:0/658101431 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.334+0000 7f674259c700 1 -- 172.21.15.77:0/658101431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6744107dd0 con 0x7f6744071290 2024-04-26T04:43:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f674259c700 1 --2- 172.21.15.77:0/658101431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67340098e0 tx=0x7f6734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10465b6135195f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f674159a700 1 -- 172.21.15.77:0/658101431 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f673400b860 con 0x7f6744071290 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f674159a700 1 -- 172.21.15.77:0/658101431 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f673400be90 con 0x7f6744071290 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f674159a700 1 -- 172.21.15.77:0/658101431 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67340103c0 con 0x7f6744071290 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f6748d1b700 1 -- 172.21.15.77:0/658101431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 msgr2=0x7f6744107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.335+0000 7f6748d1b700 1 --2- 172.21.15.77:0/658101431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67340098e0 tx=0x7f6734009250 comp rx=0 tx=0).stop 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- 172.21.15.77:0/658101431 shutdown_connections 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 --2- 172.21.15.77:0/658101431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744107890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- 172.21.15.77:0/658101431 >> 172.21.15.77:0/658101431 conn(0x7f67440fcec0 msgr2=0x7f67440ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- 172.21.15.77:0/658101431 shutdown_connections 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- 172.21.15.77:0/658101431 wait complete. 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 Processor -- start 2024-04-26T04:43:59.162 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- start start 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f6748d1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744113d40 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f674259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f674259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47416/0 (socket says 172.21.15.77:47416) 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.336+0000 7f674259c700 1 -- 172.21.15.77:0/540552143 learned_addr learned my addr 172.21.15.77:0/540552143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f674259c700 1 -- 172.21.15.77:0/540552143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67340094f0 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f674259c700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67340094c0 tx=0x7f6734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6734010890 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6734010ec0 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67340243f0 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744113f40 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.337+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6744114ab0 con 0x7f6744071290 2024-04-26T04:43:59.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f673402d3f0 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674405f5c0 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6732ffd700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 0x7f672c038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6741d9b700 1 -- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 msgr2=0x7f672c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f673404a4f0 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.338+0000 7f6741d9b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 0x7f672c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.347+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f673402b8e0 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.539+0000 7f6741d9b700 1 -- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 msgr2=0x7f672c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.539+0000 7f6741d9b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 0x7f672c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.621+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6744114e50 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.622+0000 7f6732ffd700 1 -- 172.21.15.77:0/540552143 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f673402c350 con 0x7f6744071290 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 msgr2=0x7f672c038da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:43:59.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 0x7f672c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 msgr2=0x7f6744113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67340094c0 tx=0x7f6734009250 comp rx=0 tx=0).stop 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 shutdown_connections 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f672c0368f0 0x7f672c038da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 --2- 172.21.15.77:0/540552143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6744071290 0x7f6744113800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.627+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 >> 172.21.15.77:0/540552143 conn(0x7f67440fcec0 msgr2=0x7f67440fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.628+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 shutdown_connections 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:58.628+0000 7f6748d1b700 1 -- 172.21.15.77:0/540552143 wait complete. 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-04-26T04:43:59.165 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 9... 2024-04-26T04:43:59.536 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:43:59 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/540552143' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T04:44:03.925 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: Active manager daemon smithi077.hkvsor restarted 2024-04-26T04:44:03.925 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:44:03.925 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T04:44:03.925 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: mgrmap e10: smithi077.hkvsor(active, starting, since 0.00319962s) 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:44:03.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:03 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:44:05.376 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025c6f8700 1 Processor -- start 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025c6f8700 1 -- start start 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025c6f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025c6f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254101600 con 0x7f0254100cb0 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025a494700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025a494700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47422/0 (socket says 172.21.15.77:47422) 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025a494700 1 -- 172.21.15.77:0/2036042296 learned_addr learned my addr 172.21.15.77:0/2036042296 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.616+0000 7f025a494700 1 -- 172.21.15.77:0/2036042296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254101740 con 0x7f0254100cb0 2024-04-26T04:44:05.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.617+0000 7f025a494700 1 --2- 172.21.15.77:0/2036042296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02500098e0 tx=0x7f0250009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f47030901152f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.617+0000 7f0259492700 1 -- 172.21.15.77:0/2036042296 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f025000b860 con 0x7f0254100cb0 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.617+0000 7f0259492700 1 -- 172.21.15.77:0/2036042296 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f025000be90 con 0x7f0254100cb0 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.617+0000 7f0259492700 1 -- 172.21.15.77:0/2036042296 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02500103c0 con 0x7f0254100cb0 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- 172.21.15.77:0/2036042296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 msgr2=0x7f02541010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 --2- 172.21.15.77:0/2036042296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02500098e0 tx=0x7f0250009250 comp rx=0 tx=0).stop 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- 172.21.15.77:0/2036042296 shutdown_connections 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 --2- 172.21.15.77:0/2036042296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541010c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- 172.21.15.77:0/2036042296 >> 172.21.15.77:0/2036042296 conn(0x7f02540fc690 msgr2=0x7f02540feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- 172.21.15.77:0/2036042296 shutdown_connections 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- 172.21.15.77:0/2036042296 wait complete. 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 Processor -- start 2024-04-26T04:44:05.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- start start 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.618+0000 7f025c6f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254111b20 con 0x7f0254100cb0 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025a494700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025a494700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47428/0 (socket says 172.21.15.77:47428) 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025a494700 1 -- 172.21.15.77:0/3779864722 learned_addr learned my addr 172.21.15.77:0/3779864722 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025a494700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500094f0 con 0x7f0254100cb0 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025a494700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0250011680 tx=0x7f0250009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:05.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0250010890 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0250010ec0 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254111d20 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02500243f0 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.619+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254112890 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f025002d3f0 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f024affd700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f025405f5c0 con 0x7f02400368a0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0250010bf0 con 0x7f0254100cb0 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f0259c93700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.620+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.821+0000 7f0259c93700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:43:59.821+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:44:05.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:00.222+0000 7f0259c93700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:00.222+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:01.023+0000 7f0259c93700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:01.023+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:02.624+0000 7f0259c93700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3157890049 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:02.624+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:03.833+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f025001a8c0 con 0x7f0254100cb0 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:03.833+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 msgr2=0x7f0240038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:03.833+0000 7f024affd700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.836+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f0250018350 con 0x7f0254100cb0 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.836+0000 7f024affd700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 0x7f024003c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.836+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f025405f5c0 con 0x7f0240039fc0 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.836+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 0x7f024003c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.837+0000 7f0259c93700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 0x7f024003c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02440039d0 tx=0x7f02440092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:05.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.837+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f025405f5c0 con 0x7f0240039fc0 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f02541121b0 con 0x7f0240039fc0 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f024affd700 1 -- 172.21.15.77:0/3779864722 <== mgr.14160 v2:172.21.15.77:6800/2855259383 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f02541121b0 con 0x7f0240039fc0 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 msgr2=0x7f024003c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f025c6f8700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 0x7f024003c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f02440039d0 tx=0x7f02440092f0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 msgr2=0x7f02541115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.843+0000 7f025c6f8700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0250011680 tx=0x7f0250009250 comp rx=0 tx=0).stop 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 shutdown_connections 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f0240039fc0 0x7f024003c3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0254100cb0 0x7f02541115e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 --2- 172.21.15.77:0/3779864722 >> [v2:172.21.15.77:6800/3157890049,v1:172.21.15.77:6801/3157890049] conn(0x7f02400368a0 0x7f0240038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 >> 172.21.15.77:0/3779864722 conn(0x7f02540fc690 msgr2=0x7f02540fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 shutdown_connections 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:04.844+0000 7f025c6f8700 1 -- 172.21.15.77:0/3779864722 wait complete. 2024-04-26T04:44:05.382 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 9 is available 2024-04-26T04:44:05.383 INFO:teuthology.orchestra.run.smithi077.stdout:Generating a dashboard self-signed certificate... 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: [26/Apr/2024:04:44:04] ENGINE Bus STARTING 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: [26/Apr/2024:04:44:04] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: [26/Apr/2024:04:44:04] ENGINE Bus STARTED 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T04:44:05.946 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: mgrmap e11: smithi077.hkvsor(active, since 1.00547s) 2024-04-26T04:44:05.947 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:05 smithi077 ceph-mon[26311]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T04:44:06.974 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:06 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:06.975 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:06 smithi077 ceph-mon[26311]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:06.975 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:06 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:06.975 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:06 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:06.975 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:06 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:06.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T04:44:06.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d176bf700 1 Processor -- start 2024-04-26T04:44:06.984 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d176bf700 1 -- start start 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d176bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d176bf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10103270 con 0x7f9d10076bd0 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d1545b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d1545b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47492/0 (socket says 172.21.15.77:47492) 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d1545b700 1 -- 172.21.15.77:0/2660934525 learned_addr learned my addr 172.21.15.77:0/2660934525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.041+0000 7f9d1545b700 1 -- 172.21.15.77:0/2660934525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10107dc0 con 0x7f9d10076bd0 2024-04-26T04:44:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d1545b700 1 --2- 172.21.15.77:0/2660934525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098e0 tx=0x7f9d0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a561e452e624ee2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d07fff700 1 -- 172.21.15.77:0/2660934525 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d0c00b860 con 0x7f9d10076bd0 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d07fff700 1 -- 172.21.15.77:0/2660934525 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d0c00be90 con 0x7f9d10076bd0 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d07fff700 1 -- 172.21.15.77:0/2660934525 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d0c0103c0 con 0x7f9d10076bd0 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d176bf700 1 -- 172.21.15.77:0/2660934525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 msgr2=0x7f9d10107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d176bf700 1 --2- 172.21.15.77:0/2660934525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098e0 tx=0x7f9d0c009250 comp rx=0 tx=0).stop 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d176bf700 1 -- 172.21.15.77:0/2660934525 shutdown_connections 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d176bf700 1 --2- 172.21.15.77:0/2660934525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10107880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.042+0000 7f9d176bf700 1 -- 172.21.15.77:0/2660934525 >> 172.21.15.77:0/2660934525 conn(0x7f9d100fcef0 msgr2=0x7f9d100ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:06.986 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 -- 172.21.15.77:0/2660934525 shutdown_connections 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 -- 172.21.15.77:0/2660934525 wait complete. 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 Processor -- start 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 -- start start 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d176bf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10113d80 con 0x7f9d10076bd0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d1545b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d1545b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47504/0 (socket says 172.21.15.77:47504) 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d1545b700 1 -- 172.21.15.77:0/1221662145 learned_addr learned my addr 172.21.15.77:0/1221662145 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.043+0000 7f9d1545b700 1 -- 172.21.15.77:0/1221662145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0094f0 con 0x7f9d10076bd0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d1545b700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c006d40 tx=0x7f9d0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d0c010840 con 0x7f9d10076bd0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d0c010e70 con 0x7f9d10076bd0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10113f80 con 0x7f9d10076bd0 2024-04-26T04:44:06.987 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d0c024ea0 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.044+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d10114af0 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f9d0c0109a0 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d05ffb700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 0x7f9cfc038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9d0c04a670 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1005f5c0 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d14c5a700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 0x7f9cfc038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.045+0000 7f9d14c5a700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 0x7f9cfc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d00006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.050+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9d0c02e440 con 0x7f9d10076bd0 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.261+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d100fe0e0 con 0x7f9cfc036720 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.338+0000 7f9d05ffb700 1 -- 172.21.15.77:0/1221662145 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9d100fe0e0 con 0x7f9cfc036720 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 msgr2=0x7f9cfc038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 0x7f9cfc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d00006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:06.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 msgr2=0x7f9d10113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c006d40 tx=0x7f9d0c009250 comp rx=0 tx=0).stop 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 shutdown_connections 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f9cfc036720 0x7f9cfc038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 --2- 172.21.15.77:0/1221662145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d10076bd0 0x7f9d10113840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 >> 172.21.15.77:0/1221662145 conn(0x7f9d100fcef0 msgr2=0x7f9d100fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 shutdown_connections 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:06.340+0000 7f9d176bf700 1 -- 172.21.15.77:0/1221662145 wait complete. 2024-04-26T04:44:06.989 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial admin user... 2024-04-26T04:44:07.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:07 smithi077 ceph-mon[26311]: mgrmap e12: smithi077.hkvsor(active, since 2s) 2024-04-26T04:44:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:08 smithi077 ceph-mon[26311]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:08 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FZ4N8zviKdSdICtbACEn7.xhHAWIBRhwHTS3fSmIkIsJbporB9W1u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714106647, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.399+0000 7f2c0f773700 1 Processor -- start 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0f773700 1 -- start start 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0f773700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0f773700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08103290 con 0x7f2c08076bb0 2024-04-26T04:44:09.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0d50f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0d50f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47518/0 (socket says 172.21.15.77:47518) 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0d50f700 1 -- 172.21.15.77:0/2948757053 learned_addr learned my addr 172.21.15.77:0/2948757053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0d50f700 1 -- 172.21.15.77:0/2948757053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08107de0 con 0x7f2c08076bb0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.400+0000 7f2c0d50f700 1 --2- 172.21.15.77:0/2948757053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80098e0 tx=0x7f2bf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae18401cbbd9c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2bfffff700 1 -- 172.21.15.77:0/2948757053 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf800b860 con 0x7f2c08076bb0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2bfffff700 1 -- 172.21.15.77:0/2948757053 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bf800be90 con 0x7f2c08076bb0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2bfffff700 1 -- 172.21.15.77:0/2948757053 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf80103c0 con 0x7f2c08076bb0 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 -- 172.21.15.77:0/2948757053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 msgr2=0x7f2c081078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 --2- 172.21.15.77:0/2948757053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80098e0 tx=0x7f2bf8009250 comp rx=0 tx=0).stop 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 -- 172.21.15.77:0/2948757053 shutdown_connections 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 --2- 172.21.15.77:0/2948757053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c081078a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 -- 172.21.15.77:0/2948757053 >> 172.21.15.77:0/2948757053 conn(0x7f2c080fcef0 msgr2=0x7f2c080ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 -- 172.21.15.77:0/2948757053 shutdown_connections 2024-04-26T04:44:09.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.401+0000 7f2c0f773700 1 -- 172.21.15.77:0/2948757053 wait complete. 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0f773700 1 Processor -- start 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0f773700 1 -- start start 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0f773700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0f773700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08113d60 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0d50f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0d50f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47524/0 (socket says 172.21.15.77:47524) 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0d50f700 1 -- 172.21.15.77:0/1659708159 learned_addr learned my addr 172.21.15.77:0/1659708159 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0d50f700 1 -- 172.21.15.77:0/1659708159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80094f0 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.402+0000 7f2c0d50f700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006d40 tx=0x7f2bf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.403+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf8010840 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.403+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bf8010e70 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.403+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c08113f60 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.403+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf8024ea0 con 0x7f2c08076bb0 2024-04-26T04:44:09.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.403+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c08114ad0 con 0x7f2c08076bb0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f2bf801a4c0 con 0x7f2c08076bb0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0805f5c0 con 0x7f2c08076bb0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2bfdffb700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 0x7f2bf4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2bf804a820 con 0x7f2c08076bb0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2c0cd0e700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 0x7f2bf4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.404+0000 7f2c0cd0e700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 0x7f2bf4038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009990 tx=0x7f2c04006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.409+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2bf8018940 con 0x7f2c08076bb0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.626+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2c0805f3b0 con 0x7f2bf40368a0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.850+0000 7f2bfdffb700 1 -- 172.21.15.77:0/1659708159 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f2c0805f3b0 con 0x7f2bf40368a0 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.852+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 msgr2=0x7f2bf4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.852+0000 7f2c0f773700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 0x7f2bf4038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009990 tx=0x7f2c04006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.852+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 msgr2=0x7f2c08113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:09.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006d40 tx=0x7f2bf8009250 comp rx=0 tx=0).stop 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 shutdown_connections 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f2bf40368a0 0x7f2bf4038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 --2- 172.21.15.77:0/1659708159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08076bb0 0x7f2c08113820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 >> 172.21.15.77:0/1659708159 conn(0x7f2c080fcef0 msgr2=0x7f2c080fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 shutdown_connections 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:07.853+0000 7f2c0f773700 1 -- 172.21.15.77:0/1659708159 wait complete. 2024-04-26T04:44:09.250 INFO:teuthology.orchestra.run.smithi077.stdout:Fetching dashboard port number... 2024-04-26T04:44:10.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T04:44:10.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd4211b0700 1 Processor -- start 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd4211b0700 1 -- start start 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd4211b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd4211b0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c101880 con 0x7fd41c100f30 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56916/0 (socket says 172.21.15.77:56916) 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.670+0000 7fd41ad9d700 1 -- 172.21.15.77:0/3455853531 learned_addr learned my addr 172.21.15.77:0/3455853531 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:10.449 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd41ad9d700 1 -- 172.21.15.77:0/3455853531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c1019c0 con 0x7fd41c100f30 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd41ad9d700 1 --2- 172.21.15.77:0/3455853531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00aa80 tx=0x7fd40c00a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3c1cfdc0371e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd419d9b700 1 -- 172.21.15.77:0/3455853531 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd40c00cba0 con 0x7fd41c100f30 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd419d9b700 1 -- 172.21.15.77:0/3455853531 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd40c00cd00 con 0x7fd41c100f30 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd419d9b700 1 -- 172.21.15.77:0/3455853531 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd40c010720 con 0x7fd41c100f30 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd4211b0700 1 -- 172.21.15.77:0/3455853531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 msgr2=0x7fd41c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.671+0000 7fd4211b0700 1 --2- 172.21.15.77:0/3455853531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00aa80 tx=0x7fd40c00a3f0 comp rx=0 tx=0).stop 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- 172.21.15.77:0/3455853531 shutdown_connections 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 --2- 172.21.15.77:0/3455853531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c101340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- 172.21.15.77:0/3455853531 >> 172.21.15.77:0/3455853531 conn(0x7fd41c0fc6f0 msgr2=0x7fd41c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- 172.21.15.77:0/3455853531 shutdown_connections 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- 172.21.15.77:0/3455853531 wait complete. 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 Processor -- start 2024-04-26T04:44:10.450 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- start start 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.672+0000 7fd4211b0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c1053c0 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56926/0 (socket says 172.21.15.77:56926) 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd41ad9d700 1 -- 172.21.15.77:0/893392755 learned_addr learned my addr 172.21.15.77:0/893392755 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd41ad9d700 1 -- 172.21.15.77:0/893392755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c00a690 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd41ad9d700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0108d0 tx=0x7fd40c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd40c018760 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd40c0188c0 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c103430 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd40c0203e0 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.673+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c103920 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.674+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fd40c020650 con 0x7fd41c100f30 2024-04-26T04:44:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.674+0000 7fd4037fe700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 0x7fd404038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.674+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd40c04a440 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.674+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c04bed0 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.674+0000 7fd41a59c700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 0x7fd404038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.675+0000 7fd41a59c700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 0x7fd404038da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd410009990 tx=0x7fd410006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.679+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd40c026020 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.852+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fd40c018a30 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.868+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd41c1023f0 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.868+0000 7fd4037fe700 1 -- 172.21.15.77:0/893392755 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd40c029770 con 0x7fd41c100f30 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 msgr2=0x7fd404038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 0x7fd404038da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd410009990 tx=0x7fd410006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 msgr2=0x7fd41c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0108d0 tx=0x7fd40c01a040 comp rx=0 tx=0).stop 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 shutdown_connections 2024-04-26T04:44:10.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd4040368f0 0x7fd404038da0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 --2- 172.21.15.77:0/893392755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd41c100f30 0x7fd41c104e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 >> 172.21.15.77:0/893392755 conn(0x7fd41c0fc6f0 msgr2=0x7fd41c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 shutdown_connections 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:09.870+0000 7fd4211b0700 1 -- 172.21.15.77:0/893392755 wait complete. 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:44:10.453 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-04-26T04:44:10.458 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:44:10.458 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-04-26T04:44:10.458 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-04-26T04:44:10.458 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph Dashboard is now available at: 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout: URL: https://smithi077:8443/ 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout: User: admin 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout: Password: jkznxcntfe 2024-04-26T04:44:10.460 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:10.461 INFO:teuthology.orchestra.run.smithi077.stdout:Saving cluster configuration to /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config directory 2024-04-26T04:44:10.461 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling autotune for osd_memory_target 2024-04-26T04:44:11.081 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:10 smithi077 ceph-mon[26311]: mgrmap e13: smithi077.hkvsor(active, since 6s) 2024-04-26T04:44:11.081 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:10 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/893392755' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T04:44:11.650 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.887+0000 7f44cf884700 1 Processor -- start 2024-04-26T04:44:11.650 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.887+0000 7f44cf884700 1 -- start start 2024-04-26T04:44:11.650 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.887+0000 7f44cf884700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.887+0000 7f44cf884700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c81059a0 con 0x7f44c8076bf0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44cd620700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44cd620700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56936/0 (socket says 172.21.15.77:56936) 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44cd620700 1 -- 172.21.15.77:0/3832117134 learned_addr learned my addr 172.21.15.77:0/3832117134 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44cd620700 1 -- 172.21.15.77:0/3832117134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8105ae0 con 0x7f44c8076bf0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44cd620700 1 --2- 172.21.15.77:0/3832117134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44c40098e0 tx=0x7f44c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f656f180b2baa11c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44bffff700 1 -- 172.21.15.77:0/3832117134 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44c400b860 con 0x7f44c8076bf0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44bffff700 1 -- 172.21.15.77:0/3832117134 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44c400be90 con 0x7f44c8076bf0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.888+0000 7f44bffff700 1 -- 172.21.15.77:0/3832117134 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44c40103c0 con 0x7f44c8076bf0 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- 172.21.15.77:0/3832117134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 msgr2=0x7f44c8105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 --2- 172.21.15.77:0/3832117134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44c40098e0 tx=0x7f44c4009250 comp rx=0 tx=0).stop 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- 172.21.15.77:0/3832117134 shutdown_connections 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 --2- 172.21.15.77:0/3832117134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8105460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- 172.21.15.77:0/3832117134 >> 172.21.15.77:0/3832117134 conn(0x7f44c80fcef0 msgr2=0x7f44c80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- 172.21.15.77:0/3832117134 shutdown_connections 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- 172.21.15.77:0/3832117134 wait complete. 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 Processor -- start 2024-04-26T04:44:11.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.889+0000 7f44cf884700 1 -- start start 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cf884700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cf884700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8111b80 con 0x7f44c8076bf0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cd620700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cd620700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56946/0 (socket says 172.21.15.77:56946) 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cd620700 1 -- 172.21.15.77:0/3035508458 learned_addr learned my addr 172.21.15.77:0/3035508458 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cd620700 1 -- 172.21.15.77:0/3035508458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40094f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cd620700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44c40094c0 tx=0x7f44c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44c4010890 con 0x7f44c8076bf0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c8111d80 con 0x7f44c8076bf0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44c4010ec0 con 0x7f44c8076bf0 2024-04-26T04:44:11.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44c40243f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.890+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c81128f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.891+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f44c402d3f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.891+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c805f5c0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.891+0000 7f44bdffb700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 0x7f44b4038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.891+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f44c404a5f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.892+0000 7f44cce1f700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 0x7f44b4038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.892+0000 7f44cce1f700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 0x7f44b4038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009990 tx=0x7f44b8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:10.896+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f44c402b2f0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.080+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f44c804bed0 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.081+0000 7f44bdffb700 1 -- 172.21.15.77:0/3035508458 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f44c402c350 con 0x7f44c8076bf0 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 msgr2=0x7f44b4038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 0x7f44b4038e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009990 tx=0x7f44b8006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 msgr2=0x7f44c8111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:11.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44c40094c0 tx=0x7f44c4009250 comp rx=0 tx=0).stop 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 shutdown_connections 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f44b4036990 0x7f44b4038e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 --2- 172.21.15.77:0/3035508458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44c8076bf0 0x7f44c8111640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 >> 172.21.15.77:0/3035508458 conn(0x7f44c80fcef0 msgr2=0x7f44c80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 shutdown_connections 2024-04-26T04:44:11.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:11.083+0000 7f44cf884700 1 -- 172.21.15.77:0/3035508458 wait complete. 2024-04-26T04:44:12.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f329942a700 1 Processor -- start 2024-04-26T04:44:12.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f329942a700 1 -- start start 2024-04-26T04:44:12.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f329942a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:12.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f329942a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294102080 con 0x7f3294101730 2024-04-26T04:44:12.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f3292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:12.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f3292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56960/0 (socket says 172.21.15.77:56960) 2024-04-26T04:44:12.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f3292ffd700 1 -- 172.21.15.77:0/521382910 learned_addr learned my addr 172.21.15.77:0/521382910 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:12.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.066+0000 7f3292ffd700 1 -- 172.21.15.77:0/521382910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32941021c0 con 0x7f3294101730 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f3292ffd700 1 --2- 172.21.15.77:0/521382910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f32840098e0 tx=0x7f3284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e11a8e93a8bb716 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f3291ffb700 1 -- 172.21.15.77:0/521382910 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328400b860 con 0x7f3294101730 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f3291ffb700 1 -- 172.21.15.77:0/521382910 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f328400be90 con 0x7f3294101730 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f3291ffb700 1 -- 172.21.15.77:0/521382910 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32840103c0 con 0x7f3294101730 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 -- 172.21.15.77:0/521382910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 msgr2=0x7f3294101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 --2- 172.21.15.77:0/521382910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f32840098e0 tx=0x7f3284009250 comp rx=0 tx=0).stop 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 -- 172.21.15.77:0/521382910 shutdown_connections 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 --2- 172.21.15.77:0/521382910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294101b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 -- 172.21.15.77:0/521382910 >> 172.21.15.77:0/521382910 conn(0x7f32940fcef0 msgr2=0x7f32940ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 -- 172.21.15.77:0/521382910 shutdown_connections 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.067+0000 7f329942a700 1 -- 172.21.15.77:0/521382910 wait complete. 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f329942a700 1 Processor -- start 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f329942a700 1 -- start start 2024-04-26T04:44:12.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f329942a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f329942a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294111b50 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f3292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f3292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56970/0 (socket says 172.21.15.77:56970) 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.068+0000 7f3292ffd700 1 -- 172.21.15.77:0/828661271 learned_addr learned my addr 172.21.15.77:0/828661271 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f3292ffd700 1 -- 172.21.15.77:0/828661271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840094f0 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f3292ffd700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32840090b0 tx=0x7f3284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3284010840 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3284010e70 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3294111d50 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3284024e40 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.069+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32941128c0 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f328401a530 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329405f5c0 con 0x7f3294101730 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f327b7fe700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 0x7f327c038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:12.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f328404a9f0 con 0x7f3294101730 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f32927fc700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 0x7f327c038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.070+0000 7f32927fc700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 0x7f327c038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3288009990 tx=0x7f3288006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.075+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3284018940 con 0x7f3294101730 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.363+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f329404bed0 con 0x7f3294101730 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.365+0000 7f327b7fe700 1 -- 172.21.15.77:0/828661271 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3284018350 con 0x7f3294101730 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.367+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 msgr2=0x7f327c038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.367+0000 7f329942a700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 0x7f327c038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3288009990 tx=0x7f3288006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.367+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 msgr2=0x7f3294111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.367+0000 7f329942a700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32840090b0 tx=0x7f3284009250 comp rx=0 tx=0).stop 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 shutdown_connections 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f327c0368f0 0x7f327c038da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 --2- 172.21.15.77:0/828661271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3294101730 0x7f3294111610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 >> 172.21.15.77:0/828661271 conn(0x7f32940fcef0 msgr2=0x7f32940fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:12.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 shutdown_connections 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-04-26T04:44:12.368+0000 7f329942a700 1 -- 172.21.15.77:0/828661271 wait complete. 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T04:44:12.970 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: ceph telemetry on 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout:For more information see: 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:12.971 INFO:teuthology.orchestra.run.smithi077.stdout:Bootstrap complete. 2024-04-26T04:44:12.998 INFO:tasks.cephadm:Fetching config... 2024-04-26T04:44:12.998 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:44:12.998 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T04:44:13.013 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T04:44:13.013 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:44:13.014 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T04:44:13.070 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T04:44:13.070 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:44:13.071 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/keyring of=/dev/stdout 2024-04-26T04:44:13.134 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T04:44:13.134 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:44:13.134 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T04:44:13.190 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T04:44:13.190 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcuVrs5ijiub4JGF9Ue4oXaXPGlNZKU22qe3E7PmeZTuKxUz8YAdv6QPpBLWQ+zu9pXTO9oEtKNfgOWdNKzTPcQgpDOrw5HgJpVAZarFSdHJBONetP9kF1sk27A290vA1YJISvQBHCN5Y5ohzK9aJ5afJuK6K519fDdtBnwl6I7TqM8F6cJ1V/A+/ttaZ4v53Hv0vWIJwKof0EG6oFHafBqi+44OAs9kC3sbK0yCRfu6ZCCSAL5+Gtmg9UI0a5392RHPhHELnk2KInOfq8q2UEoeK5yUVuwwOHm84w1XQjYkEfzTiKLwMWsAN/3eoKOaV7S/t0aOBwT6FyD1cJYidy+T4NrSdiWdX0K27dFSyTBpkCVqwUfT2K8F4jtwcbPfuNyDMmBB8JAaZ8gdpmv1OUIhQWADEQp1v9Veh6LwgHPPLVnhWTnpZhfCQo3esx0QJMkRvcXJprTrbw8g5neD8GXSmDDWF45ALzd4s06z5Q46g8PBng0L1PSNsxMRrpsA0= ceph-3c1ee652-0387-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T04:44:13.267 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcuVrs5ijiub4JGF9Ue4oXaXPGlNZKU22qe3E7PmeZTuKxUz8YAdv6QPpBLWQ+zu9pXTO9oEtKNfgOWdNKzTPcQgpDOrw5HgJpVAZarFSdHJBONetP9kF1sk27A290vA1YJISvQBHCN5Y5ohzK9aJ5afJuK6K519fDdtBnwl6I7TqM8F6cJ1V/A+/ttaZ4v53Hv0vWIJwKof0EG6oFHafBqi+44OAs9kC3sbK0yCRfu6ZCCSAL5+Gtmg9UI0a5392RHPhHELnk2KInOfq8q2UEoeK5yUVuwwOHm84w1XQjYkEfzTiKLwMWsAN/3eoKOaV7S/t0aOBwT6FyD1cJYidy+T4NrSdiWdX0K27dFSyTBpkCVqwUfT2K8F4jtwcbPfuNyDMmBB8JAaZ8gdpmv1OUIhQWADEQp1v9Veh6LwgHPPLVnhWTnpZhfCQo3esx0QJMkRvcXJprTrbw8g5neD8GXSmDDWF45ALzd4s06z5Q46g8PBng0L1PSNsxMRrpsA0= ceph-3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:44:13.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcuVrs5ijiub4JGF9Ue4oXaXPGlNZKU22qe3E7PmeZTuKxUz8YAdv6QPpBLWQ+zu9pXTO9oEtKNfgOWdNKzTPcQgpDOrw5HgJpVAZarFSdHJBONetP9kF1sk27A290vA1YJISvQBHCN5Y5ohzK9aJ5afJuK6K519fDdtBnwl6I7TqM8F6cJ1V/A+/ttaZ4v53Hv0vWIJwKof0EG6oFHafBqi+44OAs9kC3sbK0yCRfu6ZCCSAL5+Gtmg9UI0a5392RHPhHELnk2KInOfq8q2UEoeK5yUVuwwOHm84w1XQjYkEfzTiKLwMWsAN/3eoKOaV7S/t0aOBwT6FyD1cJYidy+T4NrSdiWdX0K27dFSyTBpkCVqwUfT2K8F4jtwcbPfuNyDMmBB8JAaZ8gdpmv1OUIhQWADEQp1v9Veh6LwgHPPLVnhWTnpZhfCQo3esx0QJMkRvcXJprTrbw8g5neD8GXSmDDWF45ALzd4s06z5Q46g8PBng0L1PSNsxMRrpsA0= ceph-3c1ee652-0387-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T04:44:13.317 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcuVrs5ijiub4JGF9Ue4oXaXPGlNZKU22qe3E7PmeZTuKxUz8YAdv6QPpBLWQ+zu9pXTO9oEtKNfgOWdNKzTPcQgpDOrw5HgJpVAZarFSdHJBONetP9kF1sk27A290vA1YJISvQBHCN5Y5ohzK9aJ5afJuK6K519fDdtBnwl6I7TqM8F6cJ1V/A+/ttaZ4v53Hv0vWIJwKof0EG6oFHafBqi+44OAs9kC3sbK0yCRfu6ZCCSAL5+Gtmg9UI0a5392RHPhHELnk2KInOfq8q2UEoeK5yUVuwwOHm84w1XQjYkEfzTiKLwMWsAN/3eoKOaV7S/t0aOBwT6FyD1cJYidy+T4NrSdiWdX0K27dFSyTBpkCVqwUfT2K8F4jtwcbPfuNyDMmBB8JAaZ8gdpmv1OUIhQWADEQp1v9Veh6LwgHPPLVnhWTnpZhfCQo3esx0QJMkRvcXJprTrbw8g5neD8GXSmDDWF45ALzd4s06z5Q46g8PBng0L1PSNsxMRrpsA0= ceph-3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:44:13.330 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T04:44:13.382 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:13 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/828661271' entity='client.admin' 2024-04-26T04:44:13.579 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 -- 172.21.15.77:0/2410459653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 msgr2=0x7fe3340ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 --2- 172.21.15.77:0/2410459653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe3340ff630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3240048c0 tx=0x7fe324009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 -- 172.21.15.77:0/2410459653 shutdown_connections 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 --2- 172.21.15.77:0/2410459653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe3340ff630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 -- 172.21.15.77:0/2410459653 >> 172.21.15.77:0/2410459653 conn(0x7fe3340f70a0 msgr2=0x7fe3340f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 -- 172.21.15.77:0/2410459653 shutdown_connections 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 -- 172.21.15.77:0/2410459653 wait complete. 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.206+0000 7fe33a35f700 1 Processor -- start 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe33a35f700 1 -- start start 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe33a35f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe33a35f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33410d5d0 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe333fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe333fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56992/0 (socket says 172.21.15.77:56992) 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe333fff700 1 -- 172.21.15.77:0/878563828 learned_addr learned my addr 172.21.15.77:0/878563828 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe333fff700 1 -- 172.21.15.77:0/878563828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324005040 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe333fff700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe324009680 tx=0x7fe324009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe32400bed0 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe32400b5c0 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.207+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33410d7d0 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.208+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe32401c5b0 con 0x7fe334065d30 2024-04-26T04:44:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.208+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33410e340 con 0x7fe334065d30 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.208+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fe32401c710 con 0x7fe334065d30 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.208+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe334102f40 con 0x7fe334065d30 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.209+0000 7fe330ff9700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 0x7fe31c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.209+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe324013b60 con 0x7fe334065d30 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.209+0000 7fe3337fe700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 0x7fe31c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:16.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.209+0000 7fe3337fe700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 0x7fe31c038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe328009990 tx=0x7fe328006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:16.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.213+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe324025780 con 0x7fe334065d30 2024-04-26T04:44:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.410+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fe33410ee10 con 0x7fe334065d30 2024-04-26T04:44:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.414+0000 7fe330ff9700 1 -- 172.21.15.77:0/878563828 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe324004d30 con 0x7fe334065d30 2024-04-26T04:44:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.415+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 msgr2=0x7fe31c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.415+0000 7fe33a35f700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 0x7fe31c038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe328009990 tx=0x7fe328006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.415+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 msgr2=0x7fe33410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.415+0000 7fe33a35f700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe324009680 tx=0x7fe324009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 shutdown_connections 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fe31c0368d0 0x7fe31c038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 --2- 172.21.15.77:0/878563828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe334065d30 0x7fe33410d090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 >> 172.21.15.77:0/878563828 conn(0x7fe3340f70a0 msgr2=0x7fe334103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 shutdown_connections 2024-04-26T04:44:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:16.416+0000 7fe33a35f700 1 -- 172.21.15.77:0/878563828 wait complete. 2024-04-26T04:44:17.175 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T04:44:17.175 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T04:44:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:17 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/878563828' entity='client.admin' 2024-04-26T04:44:17.609 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:44:18.816 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:18.816 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:18.816 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:44:18.816 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:18.817 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:44:18.817 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-26T04:44:18.817 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:18.817 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:18 smithi077 ceph-mon[26311]: Deploying daemon crash.smithi077 on smithi077 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 -- 172.21.15.77:0/2659593352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 msgr2=0x7ffac80fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 --2- 172.21.15.77:0/2659593352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80fbd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffab80048c0 tx=0x7ffab8009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 -- 172.21.15.77:0/2659593352 shutdown_connections 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 --2- 172.21.15.77:0/2659593352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80fbd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 -- 172.21.15.77:0/2659593352 >> 172.21.15.77:0/2659593352 conn(0x7ffac80f70a0 msgr2=0x7ffac80f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 -- 172.21.15.77:0/2659593352 shutdown_connections 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.169+0000 7ffaccae2700 1 -- 172.21.15.77:0/2659593352 wait complete. 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffaccae2700 1 Processor -- start 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffaccae2700 1 -- start start 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffaccae2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffaccae2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80f9570 con 0x7ffac80fb940 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffac659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffac659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35116/0 (socket says 172.21.15.77:35116) 2024-04-26T04:44:19.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffac659c700 1 -- 172.21.15.77:0/3341577469 learned_addr learned my addr 172.21.15.77:0/3341577469 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:19.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffac659c700 1 -- 172.21.15.77:0/3341577469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8005040 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffac659c700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffab80096b0 tx=0x7ffab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.170+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffab800bed0 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac80f7640 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffab800b5c0 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffab801c5b0 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80f7b30 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7ffab801c820 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac805f5c0 con 0x7ffac80fb940 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.171+0000 7ffab6ffd700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 0x7ffab0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:19.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.172+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffab8049c40 con 0x7ffac80fb940 2024-04-26T04:44:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.172+0000 7ffac5d9b700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 0x7ffab0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.172+0000 7ffac5d9b700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 0x7ffab0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009990 tx=0x7ffabc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:19.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.176+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffab8011d40 con 0x7ffac80fb940 2024-04-26T04:44:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.369+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ffac80f7f40 con 0x7ffab0036880 2024-04-26T04:44:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.373+0000 7ffab6ffd700 1 -- 172.21.15.77:0/3341577469 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffac80f7f40 con 0x7ffab0036880 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 msgr2=0x7ffab0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 0x7ffab0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009990 tx=0x7ffabc006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 msgr2=0x7ffac80f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffab80096b0 tx=0x7ffab8009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 shutdown_connections 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7ffab0036880 0x7ffab0038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 --2- 172.21.15.77:0/3341577469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffac80fb940 0x7ffac80f9030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 >> 172.21.15.77:0/3341577469 conn(0x7ffac80f70a0 msgr2=0x7ffac8065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 shutdown_connections 2024-04-26T04:44:19.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:19.375+0000 7ffaccae2700 1 -- 172.21.15.77:0/3341577469 wait complete. 2024-04-26T04:44:20.328 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2024-04-26T04:44:20.328 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:44:20.328 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2024-04-26T04:44:20.346 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:44:20.346 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:20.402 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2024-04-26T04:44:20.402 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch host add smithi078 2024-04-26T04:44:20.686 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:20 smithi077 ceph-mon[26311]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:20.686 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:20 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:20.770 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:44:22.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- 172.21.15.77:0/1250305163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 msgr2=0x7f4cc00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 --2- 172.21.15.77:0/1250305163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc00fce40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80048c0 tx=0x7f4ca8009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- 172.21.15.77:0/1250305163 shutdown_connections 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 --2- 172.21.15.77:0/1250305163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc00fce40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- 172.21.15.77:0/1250305163 >> 172.21.15.77:0/1250305163 conn(0x7f4cc00f8170 msgr2=0x7f4cc00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- 172.21.15.77:0/1250305163 shutdown_connections 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- 172.21.15.77:0/1250305163 wait complete. 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 Processor -- start 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.293+0000 7f4cc4928700 1 -- start start 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cc4928700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cc4928700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc010d5f0 con 0x7f4cc00fca30 2024-04-26T04:44:22.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cbe59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cbe59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35146/0 (socket says 172.21.15.77:35146) 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cbe59c700 1 -- 172.21.15.77:0/3043152939 learned_addr learned my addr 172.21.15.77:0/3043152939 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cbe59c700 1 -- 172.21.15.77:0/3043152939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8005040 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cbe59c700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009680 tx=0x7f4ca8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ca800b5c0 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ca800be70 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc010d7f0 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ca80135a0 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.294+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc010e360 con 0x7f4cc00fca30 2024-04-26T04:44:22.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.295+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f4ca8013860 con 0x7f4cc00fca30 2024-04-26T04:44:22.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.295+0000 7f4cb6ffd700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 0x7f4cac038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:22.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.295+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4ca8049360 con 0x7f4cc00fca30 2024-04-26T04:44:22.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.295+0000 7f4cbdd9b700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 0x7f4cac038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:22.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.295+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc0102f40 con 0x7f4cc00fca30 2024-04-26T04:44:22.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.296+0000 7f4cbdd9b700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 0x7f4cac038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009990 tx=0x7f4cb0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:22.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.300+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4ca8021020 con 0x7f4cc00fca30 2024-04-26T04:44:22.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:22.492+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc0102880 con 0x7f4cac036880 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: Deploying daemon node-exporter.smithi077 on smithi077 2024-04-26T04:44:24.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:23 smithi077 ceph-mon[26311]: Deploying cephadm binary to smithi078 2024-04-26T04:44:24.427 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi078' with addr '172.21.15.78' 2024-04-26T04:44:24.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.424+0000 7f4cb6ffd700 1 -- 172.21.15.77:0/3043152939 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f4cc0102880 con 0x7f4cac036880 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 msgr2=0x7f4cac038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 0x7f4cac038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009990 tx=0x7f4cb0006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 msgr2=0x7f4cc010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009680 tx=0x7f4ca8009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 shutdown_connections 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4cac036880 0x7f4cac038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 --2- 172.21.15.77:0/3043152939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc00fca30 0x7f4cc010d0b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 >> 172.21.15.77:0/3043152939 conn(0x7f4cc00f8170 msgr2=0x7f4cc0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 shutdown_connections 2024-04-26T04:44:24.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:24.427+0000 7f4cc4928700 1 -- 172.21.15.77:0/3043152939 wait complete. 2024-04-26T04:44:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:24 smithi077 ceph-mon[26311]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:24 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:24 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:25.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T04:44:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:44:26.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:25 smithi077 ceph-mon[26311]: Added host smithi078 2024-04-26T04:44:27.116 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:26 smithi077 ceph-mon[26311]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:28.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.045+0000 7f6d1f776700 1 -- 172.21.15.77:0/1980601026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 msgr2=0x7f6d180fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.045+0000 7f6d1f776700 1 --2- 172.21.15.77:0/1980601026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d180fce40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d140048c0 tx=0x7f6d14009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- 172.21.15.77:0/1980601026 shutdown_connections 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 --2- 172.21.15.77:0/1980601026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d180fce40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- 172.21.15.77:0/1980601026 >> 172.21.15.77:0/1980601026 conn(0x7f6d180f8170 msgr2=0x7f6d180fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- 172.21.15.77:0/1980601026 shutdown_connections 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- 172.21.15.77:0/1980601026 wait complete. 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 Processor -- start 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- start start 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1f776700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1810d5a0 con 0x7f6d180fca30 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1d512700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1d512700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57886/0 (socket says 172.21.15.77:57886) 2024-04-26T04:44:28.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.046+0000 7f6d1d512700 1 -- 172.21.15.77:0/3987686986 learned_addr learned my addr 172.21.15.77:0/3987686986 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d1d512700 1 -- 172.21.15.77:0/3987686986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14005040 con 0x7f6d180fca30 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d1d512700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d140096b0 tx=0x7f6d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d1400bed0 con 0x7f6d180fca30 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d1400b5c0 con 0x7f6d180fca30 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1810d7a0 con 0x7f6d180fca30 2024-04-26T04:44:28.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d1401c5b0 con 0x7f6d180fca30 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.047+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1810e310 con 0x7f6d180fca30 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.048+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f6d1401c820 con 0x7f6d180fca30 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.048+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d18102f40 con 0x7f6d180fca30 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.048+0000 7f6d0dffb700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 0x7f6d04038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.048+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6d14049ce0 con 0x7f6d180fca30 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.048+0000 7f6d1cd11700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 0x7f6d04038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:28.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.049+0000 7f6d1cd11700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 0x7f6d04038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009990 tx=0x7f6d08006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:28.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.053+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6d14023300 con 0x7f6d180fca30 2024-04-26T04:44:28.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.254+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d1805f660 con 0x7f6d04036880 2024-04-26T04:44:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.255+0000 7f6d0dffb700 1 -- 172.21.15.77:0/3987686986 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f6d1805f660 con 0x7f6d04036880 2024-04-26T04:44:28.256 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:44:28.257 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2024-04-26T04:44:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.257+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 msgr2=0x7f6d04038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.257+0000 7f6d1f776700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 0x7f6d04038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009990 tx=0x7f6d08006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 msgr2=0x7f6d1810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d140096b0 tx=0x7f6d14009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 shutdown_connections 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f6d04036880 0x7f6d04038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 --2- 172.21.15.77:0/3987686986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d180fca30 0x7f6d1810d060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 >> 172.21.15.77:0/3987686986 conn(0x7f6d180f8170 msgr2=0x7f6d18103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 shutdown_connections 2024-04-26T04:44:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:28.258+0000 7f6d1f776700 1 -- 172.21.15.77:0/3987686986 wait complete. 2024-04-26T04:44:28.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:28 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:28.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:28 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:28.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:28 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:28.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:28 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:28.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:28 smithi077 ceph-mon[26311]: Deploying daemon alertmanager.smithi077 on smithi077 2024-04-26T04:44:29.438 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T04:44:29.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T04:44:29.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:29 smithi077 ceph-mon[26311]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:29 smithi077 ceph-mon[26311]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:44:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:29 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:29.686 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:44:31.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:31 smithi077 ceph-mon[26311]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 -- 172.21.15.77:0/1404943715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 msgr2=0x7fd8e80fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/1404943715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fe550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0048c0 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 -- 172.21.15.77:0/1404943715 shutdown_connections 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/1404943715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fe550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 -- 172.21.15.77:0/1404943715 >> 172.21.15.77:0/1404943715 conn(0x7fd8e80f6820 msgr2=0x7fd8e80f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 -- 172.21.15.77:0/1404943715 shutdown_connections 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.442+0000 7fd8eec2d700 1 -- 172.21.15.77:0/1404943715 wait complete. 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8eec2d700 1 Processor -- start 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8eec2d700 1 -- start start 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8eec2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8eec2d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80fdd90 con 0x7fd8e80fe140 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8ec9c9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8ec9c9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57900/0 (socket says 172.21.15.77:57900) 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.443+0000 7fd8ec9c9700 1 -- 172.21.15.77:0/3994292589 learned_addr learned my addr 172.21.15.77:0/3994292589 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8ec9c9700 1 -- 172.21.15.77:0/3994292589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc005040 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8ec9c9700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0096b0 tx=0x7fd8dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc00bed0 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8dc00b5c0 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e80fbe60 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8dc01c5b0 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.444+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e80fc350 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fd8dc01c820 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8e57fa700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 0x7fd8d8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e805f5c0 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd8dc0499f0 con 0x7fd8e80fe140 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8e7fff700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 0x7fd8d8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.445+0000 7fd8e7fff700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 0x7fd8d8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.450+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8dc023300 con 0x7fd8e80fe140 2024-04-26T04:44:32.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:32.643+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd8e804bed0 con 0x7fd8e80fe140 2024-04-26T04:44:33.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.222+0000 7fd8e57fa700 1 -- 172.21.15.77:0/3994292589 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd8dc011b60 con 0x7fd8e80fe140 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 msgr2=0x7fd8d8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 0x7fd8d8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 msgr2=0x7fd8e80fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0096b0 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 shutdown_connections 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd8d8036880 0x7fd8d8038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 --2- 172.21.15.77:0/3994292589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8e80fe140 0x7fd8e80fd850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 >> 172.21.15.77:0/3994292589 conn(0x7fd8e80f6820 msgr2=0x7fd8e8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 shutdown_connections 2024-04-26T04:44:33.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:44:33.225+0000 7fd8eec2d700 1 -- 172.21.15.77:0/3994292589 wait complete. 2024-04-26T04:44:33.228 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2024-04-26T04:44:33.318 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:33 smithi077 ceph-mon[26311]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:33.318 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:33 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3994292589' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T04:44:34.285 INFO:tasks.cephadm:Adding mon.smithi077 on smithi077 2024-04-26T04:44:34.286 INFO:tasks.cephadm:Adding mon.smithi078 on smithi078 2024-04-26T04:44:34.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi077:172.21.15.77=smithi077;smithi078:172.21.15.78=smithi078' 2024-04-26T04:44:34.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3994292589' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T04:44:34.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:34 smithi077 ceph-mon[26311]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T04:44:34.630 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:35.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:35 smithi077 ceph-mon[26311]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- 172.21.15.78:0/1845106110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 msgr2=0x7f54e00fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 --2- 172.21.15.78:0/1845106110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e00fbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0048c0 tx=0x7f54dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- 172.21.15.78:0/1845106110 shutdown_connections 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 --2- 172.21.15.78:0/1845106110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e00fbd50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- 172.21.15.78:0/1845106110 >> 172.21.15.78:0/1845106110 conn(0x7f54e00f70a0 msgr2=0x7f54e00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- 172.21.15.78:0/1845106110 shutdown_connections 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- 172.21.15.78:0/1845106110 wait complete. 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 Processor -- start 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- start start 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.233+0000 7f54e751e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e010d5c0 con 0x7f54e00fb940 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e52ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e52ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:60748/0 (socket says 172.21.15.78:60748) 2024-04-26T04:44:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e52ba700 1 -- 172.21.15.78:0/422930778 learned_addr learned my addr 172.21.15.78:0/422930778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e52ba700 1 -- 172.21.15.78:0/422930778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc005040 con 0x7f54e00fb940 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e52ba700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0096b0 tx=0x7f54dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc00bed0 con 0x7f54e00fb940 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc00b5c0 con 0x7f54e00fb940 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e010d7c0 con 0x7f54e00fb940 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc01c5b0 con 0x7f54e00fb940 2024-04-26T04:44:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.234+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e010e330 con 0x7f54e00fb940 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.235+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f54dc013420 con 0x7f54e00fb940 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.235+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e0101e50 con 0x7f54e00fb940 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.235+0000 7f54d1ffb700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 0x7f54cc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.235+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f54dc049b10 con 0x7f54e00fb940 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.236+0000 7f54e4ab9700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 0x7f54cc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.236+0000 7f54e4ab9700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 0x7f54cc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.241+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f54dc025400 con 0x7f54e00fb940 2024-04-26T04:44:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.434+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 --> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi077:172.21.15.77=smithi077;smithi078:172.21.15.78=smithi078", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e0066480 con 0x7f54cc036880 2024-04-26T04:44:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.440+0000 7f54d1ffb700 1 -- 172.21.15.78:0/422930778 <== mgr.14160 v2:172.21.15.77:6800/2855259383 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f54e0066480 con 0x7f54cc036880 2024-04-26T04:44:36.441 INFO:teuthology.orchestra.run.smithi078.stdout:Scheduled mon update... 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 msgr2=0x7f54cc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 0x7f54cc038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 msgr2=0x7f54e010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0096b0 tx=0x7f54dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 shutdown_connections 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f54cc036880 0x7f54cc038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 --2- 172.21.15.78:0/422930778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00fb940 0x7f54e010d080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 >> 172.21.15.78:0/422930778 conn(0x7f54e00f70a0 msgr2=0x7f54e0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 shutdown_connections 2024-04-26T04:44:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:36.442+0000 7f54e751e700 1 -- 172.21.15.78:0/422930778 wait complete. 2024-04-26T04:44:37.033 DEBUG:teuthology.orchestra.run.smithi078:mon.smithi078> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service 2024-04-26T04:44:37.036 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:37.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:37.363 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:37.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:37.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:37.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-26T04:44:37.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:37 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:38.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:38 smithi077 ceph-mon[26311]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi077:172.21.15.77=smithi077;smithi078:172.21.15.78=smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:38.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:38 smithi077 ceph-mon[26311]: Saving service mon spec with placement smithi077:172.21.15.77=smithi077;smithi078:172.21.15.78=smithi078;count:2 2024-04-26T04:44:38.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:38 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-26T04:44:38.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:38 smithi077 ceph-mon[26311]: Deploying daemon grafana.smithi077 on smithi077 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2367911051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 msgr2=0x7fbd580ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2367911051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd580ff630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009c60 tx=0x7fbd50004d10 comp rx=0 tx=0).stop 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2367911051 shutdown_connections 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2367911051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd580ff630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2367911051 >> 172.21.15.78:0/2367911051 conn(0x7fbd580f70a0 msgr2=0x7fbd580f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2367911051 shutdown_connections 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2367911051 wait complete. 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.834+0000 7fbd5fa82700 1 Processor -- start 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5fa82700 1 -- start start 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5fa82700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5fa82700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5810d5d0 con 0x7fbd58065d30 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5d81e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:38.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5d81e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:60768/0 (socket says 172.21.15.78:60768) 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5d81e700 1 -- 172.21.15.78:0/2761567183 learned_addr learned my addr 172.21.15.78:0/2761567183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5d81e700 1 -- 172.21.15.78:0/2761567183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50009940 con 0x7fbd58065d30 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5d81e700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd500097b0 tx=0x7fbd50004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd50012410 con 0x7fbd58065d30 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd50012a40 con 0x7fbd58065d30 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.835+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5810d7d0 con 0x7fbd58065d30 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.836+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd5001ba20 con 0x7fbd58065d30 2024-04-26T04:44:38.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.836+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5810e340 con 0x7fbd58065d30 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.836+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fbd50019040 con 0x7fbd58065d30 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.836+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd58102f40 con 0x7fbd58065d30 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.837+0000 7fbd4e7fc700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 0x7fbd44038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.837+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbd50049cb0 con 0x7fbd58065d30 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.837+0000 7fbd5d01d700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 0x7fbd44038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.837+0000 7fbd5d01d700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 0x7fbd44038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009990 tx=0x7fbd48006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:38.842+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbd50012570 con 0x7fbd58065d30 2024-04-26T04:44:39.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.101+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd5810ed50 con 0x7fbd58065d30 2024-04-26T04:44:39.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.101+0000 7fbd4e7fc700 1 -- 172.21.15.78:0/2761567183 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbd50017090 con 0x7fbd58065d30 2024-04-26T04:44:39.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:39.103 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:39.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.103+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 msgr2=0x7fbd44038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.103+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 0x7fbd44038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009990 tx=0x7fbd48006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.103+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 msgr2=0x7fbd5810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.103+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd500097b0 tx=0x7fbd50004d10 comp rx=0 tx=0).stop 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 shutdown_connections 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbd440368d0 0x7fbd44038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 --2- 172.21.15.78:0/2761567183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd58065d30 0x7fbd5810d090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 >> 172.21.15.78:0/2761567183 conn(0x7fbd580f70a0 msgr2=0x7fbd58103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 shutdown_connections 2024-04-26T04:44:39.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:39.104+0000 7fbd5fa82700 1 -- 172.21.15.78:0/2761567183 wait complete. 2024-04-26T04:44:39.106 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:39.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:39 smithi077 ceph-mon[26311]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:39.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:39 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:39.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:39 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2761567183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:40.658 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:40.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:40.905 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:41.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:41 smithi077 ceph-mon[26311]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:42.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 -- 172.21.15.78:0/4219355632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 msgr2=0x7f44180fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:42.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 --2- 172.21.15.78:0/4219355632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f44180fbd40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44080048c0 tx=0x7f4408009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:42.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 -- 172.21.15.78:0/4219355632 shutdown_connections 2024-04-26T04:44:42.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 --2- 172.21.15.78:0/4219355632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f44180fbd40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:42.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 -- 172.21.15.78:0/4219355632 >> 172.21.15.78:0/4219355632 conn(0x7f44180f7070 msgr2=0x7f44180f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 -- 172.21.15.78:0/4219355632 shutdown_connections 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.440+0000 7f441dbe6700 1 -- 172.21.15.78:0/4219355632 wait complete. 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f441dbe6700 1 Processor -- start 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f441dbe6700 1 -- start start 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f441dbe6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f441dbe6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441810fdd0 con 0x7f44180fb930 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f44177fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f44177fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:60786/0 (socket says 172.21.15.78:60786) 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.441+0000 7f44177fe700 1 -- 172.21.15.78:0/1515101649 learned_addr learned my addr 172.21.15.78:0/1515101649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:42.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f44177fe700 1 -- 172.21.15.78:0/1515101649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408005040 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f44177fe700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44080096b0 tx=0x7f4408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f440800bed0 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f440800b5c0 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441810ff10 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f440801c5b0 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.442+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4418110400 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.443+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f4408013420 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.443+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44181058f0 con 0x7f44180fb930 2024-04-26T04:44:42.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.443+0000 7f4414ff9700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 0x7f4404038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:42.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.443+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4408049b10 con 0x7f44180fb930 2024-04-26T04:44:42.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.443+0000 7f440effd700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 0x7f4404038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:42.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.444+0000 7f440effd700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 0x7f4404038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4400009990 tx=0x7f4400006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.452+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4408025400 con 0x7f44180fb930 2024-04-26T04:44:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.708+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f441805f5c0 con 0x7f44180fb930 2024-04-26T04:44:42.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.709+0000 7f4414ff9700 1 -- 172.21.15.78:0/1515101649 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4408004d30 con 0x7f44180fb930 2024-04-26T04:44:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:42.710 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 msgr2=0x7f4404038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 0x7f4404038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4400009990 tx=0x7f4400006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 msgr2=0x7f4418066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44080096b0 tx=0x7f4408009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 shutdown_connections 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f4404036880 0x7f4404038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 --2- 172.21.15.78:0/1515101649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44180fb930 0x7f4418066820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 >> 172.21.15.78:0/1515101649 conn(0x7f44180f7070 msgr2=0x7f4418102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 shutdown_connections 2024-04-26T04:44:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:42.711+0000 7f441dbe6700 1 -- 172.21.15.78:0/1515101649 wait complete. 2024-04-26T04:44:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:43 smithi077 ceph-mon[26311]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:43 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1515101649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:44.307 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:44.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:44.616 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:45.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:45 smithi077 ceph-mon[26311]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- 172.21.15.78:0/3618280948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 msgr2=0x7fd5780fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 --2- 172.21.15.78:0/3618280948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd5780fce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5680048c0 tx=0x7fd568009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- 172.21.15.78:0/3618280948 shutdown_connections 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 --2- 172.21.15.78:0/3618280948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd5780fce40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- 172.21.15.78:0/3618280948 >> 172.21.15.78:0/3618280948 conn(0x7fd5780f8170 msgr2=0x7fd5780fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- 172.21.15.78:0/3618280948 shutdown_connections 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- 172.21.15.78:0/3618280948 wait complete. 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 Processor -- start 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 -- start start 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.987+0000 7fd57e13c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd57e13c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57810d5f0 con 0x7fd5780fca30 2024-04-26T04:44:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd5777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:45.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd5777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:50048/0 (socket says 172.21.15.78:50048) 2024-04-26T04:44:45.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd5777fe700 1 -- 172.21.15.78:0/4234963032 learned_addr learned my addr 172.21.15.78:0/4234963032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:45.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd5777fe700 1 -- 172.21.15.78:0/4234963032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568005040 con 0x7fd5780fca30 2024-04-26T04:44:45.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd5777fe700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5680096b0 tx=0x7fd568009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56800bed0 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57810d7f0 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56800b5c0 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56801c5b0 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.988+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57810e360 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.989+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fd568013420 con 0x7fd5780fca30 2024-04-26T04:44:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.989+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd578102f40 con 0x7fd5780fca30 2024-04-26T04:44:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.989+0000 7fd55ffff700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 0x7fd560038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.989+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd568049b10 con 0x7fd5780fca30 2024-04-26T04:44:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.989+0000 7fd576ffd700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 0x7fd560038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.990+0000 7fd576ffd700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 0x7fd560038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009990 tx=0x7fd56c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:45.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:45.998+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd568025400 con 0x7fd5780fca30 2024-04-26T04:44:46.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.256+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd57810ed70 con 0x7fd5780fca30 2024-04-26T04:44:46.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.256+0000 7fd55ffff700 1 -- 172.21.15.78:0/4234963032 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd568004d30 con 0x7fd5780fca30 2024-04-26T04:44:46.257 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:46.257 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:46.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 msgr2=0x7fd560038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:46.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 0x7fd560038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009990 tx=0x7fd56c006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:46.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 msgr2=0x7fd57810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:46.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5680096b0 tx=0x7fd568009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 shutdown_connections 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd560036880 0x7fd560038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 --2- 172.21.15.78:0/4234963032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5780fca30 0x7fd57810d0b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 >> 172.21.15.78:0/4234963032 conn(0x7fd5780f8170 msgr2=0x7fd578103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 shutdown_connections 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:46.258+0000 7fd57e13c700 1 -- 172.21.15.78:0/4234963032 wait complete. 2024-04-26T04:44:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:47.454 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:47 smithi077 ceph-mon[26311]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:47.454 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:47 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/4234963032' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:47.915 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:47.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:48.210 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:49.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:49 smithi077 ceph-mon[26311]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 -- 172.21.15.78:0/1391727807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 msgr2=0x7fbe700fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 --2- 172.21.15.78:0/1391727807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe700fbd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe680048c0 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 -- 172.21.15.78:0/1391727807 shutdown_connections 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 --2- 172.21.15.78:0/1391727807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe700fbd50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 -- 172.21.15.78:0/1391727807 >> 172.21.15.78:0/1391727807 conn(0x7fbe700f70a0 msgr2=0x7fbe700f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 -- 172.21.15.78:0/1391727807 shutdown_connections 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.630+0000 7fbe7861a700 1 -- 172.21.15.78:0/1391727807 wait complete. 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe7861a700 1 Processor -- start 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe7861a700 1 -- start start 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe7861a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe7861a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7010d5c0 con 0x7fbe700fb940 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe763b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe763b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:50070/0 (socket says 172.21.15.78:50070) 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe763b6700 1 -- 172.21.15.78:0/3135259128 learned_addr learned my addr 172.21.15.78:0/3135259128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.631+0000 7fbe763b6700 1 -- 172.21.15.78:0/3135259128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68005040 con 0x7fbe700fb940 2024-04-26T04:44:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe763b6700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe680096b0 tx=0x7fbe68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe6800bed0 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe6800b5c0 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7010d7c0 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe6801c5b0 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.632+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7010e330 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fbe68013420 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe70101e50 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe66ffd700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 0x7fbe5c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbe68049ab0 con 0x7fbe700fb940 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe75bb5700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 0x7fbe5c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.633+0000 7fbe75bb5700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 0x7fbe5c038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.638+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbe68025400 con 0x7fbe700fb940 2024-04-26T04:44:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.898+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbe7010e570 con 0x7fbe700fb940 2024-04-26T04:44:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.899+0000 7fbe66ffd700 1 -- 172.21.15.78:0/3135259128 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbe68004d30 con 0x7fbe700fb940 2024-04-26T04:44:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:49.900 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 msgr2=0x7fbe5c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 0x7fbe5c038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 msgr2=0x7fbe7010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe680096b0 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 shutdown_connections 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fbe5c036880 0x7fbe5c038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 --2- 172.21.15.78:0/3135259128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe700fb940 0x7fbe7010d080 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 >> 172.21.15.78:0/3135259128 conn(0x7fbe700f70a0 msgr2=0x7fbe70065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 shutdown_connections 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:49.901+0000 7fbe7861a700 1 -- 172.21.15.78:0/3135259128 wait complete. 2024-04-26T04:44:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:50.320 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/3135259128' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:50.321 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:50 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:51.469 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:51 smithi077 ceph-mon[26311]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:51.469 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:51 smithi077 ceph-mon[26311]: Deploying daemon prometheus.smithi077 on smithi077 2024-04-26T04:44:51.523 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:51.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 -- 172.21.15.78:0/2957369816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 msgr2=0x7f55640fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 --2- 172.21.15.78:0/2957369816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f55640fce30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55540048c0 tx=0x7f5554009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 -- 172.21.15.78:0/2957369816 shutdown_connections 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 --2- 172.21.15.78:0/2957369816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f55640fce30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 -- 172.21.15.78:0/2957369816 >> 172.21.15.78:0/2957369816 conn(0x7f55640f81a0 msgr2=0x7f55640fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 -- 172.21.15.78:0/2957369816 shutdown_connections 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.243+0000 7f556baae700 1 -- 172.21.15.78:0/2957369816 wait complete. 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556baae700 1 Processor -- start 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556baae700 1 -- start start 2024-04-26T04:44:53.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556baae700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556baae700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556410d5c0 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556984a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556984a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:53390/0 (socket says 172.21.15.78:53390) 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556984a700 1 -- 172.21.15.78:0/2340910990 learned_addr learned my addr 172.21.15.78:0/2340910990 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.244+0000 7f556984a700 1 -- 172.21.15.78:0/2340910990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554005040 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f556984a700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55540096b0 tx=0x7f5554009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f555400bed0 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f555400b5c0 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556410d7c0 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f555401c5b0 con 0x7f55640fca20 2024-04-26T04:44:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.245+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556410e330 con 0x7f55640fca20 2024-04-26T04:44:53.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5554013420 con 0x7f55640fca20 2024-04-26T04:44:53.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5564102f30 con 0x7f55640fca20 2024-04-26T04:44:53.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f555a7fc700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 0x7f5550038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:53.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5554049a50 con 0x7f55640fca20 2024-04-26T04:44:53.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f5569049700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 0x7f5550038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:53.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.246+0000 7f5569049700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 0x7f5550038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5560009990 tx=0x7f5560006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:53.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.252+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5554025400 con 0x7f55640fca20 2024-04-26T04:44:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.509+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f556410ed40 con 0x7f55640fca20 2024-04-26T04:44:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.510+0000 7f555a7fc700 1 -- 172.21.15.78:0/2340910990 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5554004d30 con 0x7f55640fca20 2024-04-26T04:44:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:53.511 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 msgr2=0x7f5550038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 0x7f5550038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5560009990 tx=0x7f5560006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 msgr2=0x7f556410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55540096b0 tx=0x7f5554009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 shutdown_connections 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f5550036880 0x7f5550038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 --2- 172.21.15.78:0/2340910990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55640fca20 0x7f556410d080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 >> 172.21.15.78:0/2340910990 conn(0x7f55640f81a0 msgr2=0x7f5564103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 shutdown_connections 2024-04-26T04:44:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:53.512+0000 7f556baae700 1 -- 172.21.15.78:0/2340910990 wait complete. 2024-04-26T04:44:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:53.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:53 smithi077 ceph-mon[26311]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:54.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:54 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2340910990' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:54.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:54 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:44:55.087 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:55.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:55.326 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:55.373 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:55.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:55 smithi077 ceph-mon[26311]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:56.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- 172.21.15.78:0/4201652916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 msgr2=0x7f97cc0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 --2- 172.21.15.78:0/4201652916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc0fe490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0048c0 tx=0x7f97bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- 172.21.15.78:0/4201652916 shutdown_connections 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 --2- 172.21.15.78:0/4201652916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc0fe490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- 172.21.15.78:0/4201652916 >> 172.21.15.78:0/4201652916 conn(0x7f97cc0f6820 msgr2=0x7f97cc0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- 172.21.15.78:0/4201652916 shutdown_connections 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- 172.21.15.78:0/4201652916 wait complete. 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 Processor -- start 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.698+0000 7f97d14b4700 1 -- start start 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97d14b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97d14b4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc10d5e0 con 0x7f97cc0fe080 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97caffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97caffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:53400/0 (socket says 172.21.15.78:53400) 2024-04-26T04:44:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97caffd700 1 -- 172.21.15.78:0/73491915 learned_addr learned my addr 172.21.15.78:0/73491915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97caffd700 1 -- 172.21.15.78:0/73491915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc005040 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97caffd700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0096b0 tx=0x7f97bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97bc00bed0 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc10d7e0 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97bc00b5c0 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97bc01c5b0 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.699+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc10e350 con 0x7f97cc0fe080 2024-04-26T04:44:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.700+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f97bc01c820 con 0x7f97cc0fe080 2024-04-26T04:44:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.700+0000 7f97b37fe700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 0x7f97b4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.700+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f97bc049ab0 con 0x7f97cc0fe080 2024-04-26T04:44:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.700+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc102f50 con 0x7f97cc0fe080 2024-04-26T04:44:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.701+0000 7f97ca7fc700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 0x7f97b4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.701+0000 7f97ca7fc700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 0x7f97b4038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009990 tx=0x7f97c0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:56.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.706+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f97bc023300 con 0x7f97cc0fe080 2024-04-26T04:44:56.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.966+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f97cc05f5c0 con 0x7f97cc0fe080 2024-04-26T04:44:56.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.967+0000 7f97b37fe700 1 -- 172.21.15.78:0/73491915 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f97cc05f5c0 con 0x7f97cc0fe080 2024-04-26T04:44:56.968 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:44:56.968 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:44:56.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.968+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 msgr2=0x7f97b4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:56.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.968+0000 7f97d14b4700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 0x7f97b4038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009990 tx=0x7f97c0006e40 comp rx=0 tx=0).stop 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.968+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 msgr2=0x7f97cc10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.968+0000 7f97d14b4700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0096b0 tx=0x7f97bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 shutdown_connections 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f97b4036880 0x7f97b4038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 --2- 172.21.15.78:0/73491915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97cc0fe080 0x7f97cc10d0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 >> 172.21.15.78:0/73491915 conn(0x7f97cc0f6820 msgr2=0x7f97cc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 shutdown_connections 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:44:56.969+0000 7f97d14b4700 1 -- 172.21.15.78:0/73491915 wait complete. 2024-04-26T04:44:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:44:57.532 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:57 smithi077 ceph-mon[26311]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:44:57.532 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:57 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/73491915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:44:58.556 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:44:58.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:44:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:44:59.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:44:59 smithi077 ceph-mon[26311]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- 172.21.15.78:0/386724305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 msgr2=0x7f77940fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 --2- 172.21.15.78:0/386724305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77940fc1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77880048c0 tx=0x7f7788009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- 172.21.15.78:0/386724305 shutdown_connections 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 --2- 172.21.15.78:0/386724305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77940fc1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- 172.21.15.78:0/386724305 >> 172.21.15.78:0/386724305 conn(0x7f77940f6820 msgr2=0x7f77940f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- 172.21.15.78:0/386724305 shutdown_connections 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- 172.21.15.78:0/386724305 wait complete. 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 Processor -- start 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.299+0000 7f779aeca700 1 -- start start 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f779aeca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f779aeca700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7794113c00 con 0x7f77940fbdb0 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f7798c66700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f7798c66700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:53420/0 (socket says 172.21.15.78:53420) 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f7798c66700 1 -- 172.21.15.78:0/2700044314 learned_addr learned my addr 172.21.15.78:0/2700044314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f7798c66700 1 -- 172.21.15.78:0/2700044314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788005040 con 0x7f77940fbdb0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f7798c66700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7788009680 tx=0x7f7788009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f778800b5c0 con 0x7f77940fbdb0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f778800be70 con 0x7f77940fbdb0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7794113e00 con 0x7f77940fbdb0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77880135a0 con 0x7f77940fbdb0 2024-04-26T04:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.300+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7794114970 con 0x7f77940fbdb0 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.301+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f7788013860 con 0x7f77940fbdb0 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.301+0000 7f77917fa700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 0x7f7784038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.301+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794100d20 con 0x7f77940fbdb0 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.301+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f77880495d0 con 0x7f77940fbdb0 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.301+0000 7f7793fff700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 0x7f7784038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.302+0000 7f7793fff700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 0x7f7784038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7780009990 tx=0x7f7780006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.306+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7788011960 con 0x7f77940fbdb0 2024-04-26T04:45:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.565+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7794115380 con 0x7f77940fbdb0 2024-04-26T04:45:00.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.565+0000 7f77917fa700 1 -- 172.21.15.78:0/2700044314 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7788041030 con 0x7f77940fbdb0 2024-04-26T04:45:00.567 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:00.567 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 msgr2=0x7f7784038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 0x7f7784038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7780009990 tx=0x7f7780006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 msgr2=0x7f77941136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7788009680 tx=0x7f7788009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 shutdown_connections 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f7784036880 0x7f7784038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 --2- 172.21.15.78:0/2700044314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77940fbdb0 0x7f77941136c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 >> 172.21.15.78:0/2700044314 conn(0x7f77940f6820 msgr2=0x7f779410aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 shutdown_connections 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:00.567+0000 7f779aeca700 1 -- 172.21.15.78:0/2700044314 wait complete. 2024-04-26T04:45:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2700044314' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:01.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:01 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-26T04:45:02.074 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:02.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:02.315 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:02.362 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:02 smithi077 ceph-mon[26311]: from='mgr.14160 172.21.15.77:0/868029926' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-26T04:45:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:02 smithi077 ceph-mon[26311]: mgrmap e14: smithi077.hkvsor(active, since 58s) 2024-04-26T04:45:03.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.845+0000 7fd2fa215700 1 -- 172.21.15.78:0/621525981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 msgr2=0x7fd2f40fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.845+0000 7fd2fa215700 1 --2- 172.21.15.78:0/621525981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f40fce40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40048c0 tx=0x7fd2e4009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.845+0000 7fd2fa215700 1 -- 172.21.15.78:0/621525981 shutdown_connections 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.845+0000 7fd2fa215700 1 --2- 172.21.15.78:0/621525981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f40fce40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.845+0000 7fd2fa215700 1 -- 172.21.15.78:0/621525981 >> 172.21.15.78:0/621525981 conn(0x7fd2f40f8170 msgr2=0x7fd2f40fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 -- 172.21.15.78:0/621525981 shutdown_connections 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 -- 172.21.15.78:0/621525981 wait complete. 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 Processor -- start 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 -- start start 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.846+0000 7fd2fa215700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f410d5f0 con 0x7fd2f40fca30 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2f37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:55018/0 (socket says 172.21.15.78:55018) 2024-04-26T04:45:03.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2f37fe700 1 -- 172.21.15.78:0/3467925078 learned_addr learned my addr 172.21.15.78:0/3467925078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:03.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2f37fe700 1 -- 172.21.15.78:0/3467925078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e4005040 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2f37fe700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40096b0 tx=0x7fd2e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2e400bed0 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2e400b5c0 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f410d7f0 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2e401c5b0 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.847+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f410e360 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7fd2e4013420 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4102f40 con 0x7fd2f40fca30 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2dbfff700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 0x7fd2dc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:03.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd2e4049a60 con 0x7fd2f40fca30 2024-04-26T04:45:03.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2f2ffd700 1 -- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 msgr2=0x7fd2dc038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2855259383 2024-04-26T04:45:03.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.848+0000 7fd2f2ffd700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 0x7fd2dc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:45:03.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:03.853+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd2e4025400 con 0x7fd2f40fca30 2024-04-26T04:45:04.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.049+0000 7fd2f2ffd700 1 -- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 msgr2=0x7fd2dc038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2855259383 2024-04-26T04:45:04.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.049+0000 7fd2f2ffd700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 0x7fd2dc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:45:04.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.108+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd2f410ed70 con 0x7fd2f40fca30 2024-04-26T04:45:04.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.109+0000 7fd2dbfff700 1 -- 172.21.15.78:0/3467925078 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd2e4004d30 con 0x7fd2f40fca30 2024-04-26T04:45:04.111 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:04.111 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:04.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 msgr2=0x7fd2dc038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:45:04.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 0x7fd2dc038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 msgr2=0x7fd2f410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40096b0 tx=0x7fd2e4009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 shutdown_connections 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7fd2dc0368d0 0x7fd2dc038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 --2- 172.21.15.78:0/3467925078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2f40fca30 0x7fd2f410d0b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 >> 172.21.15.78:0/3467925078 conn(0x7fd2f40f8170 msgr2=0x7fd2f4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 shutdown_connections 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:04.111+0000 7fd2fa215700 1 -- 172.21.15.78:0/3467925078 wait complete. 2024-04-26T04:45:04.114 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:04.471 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:04 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/3467925078' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:05.697 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:05.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:05.989 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 -- 172.21.15.78:0/3972081677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 msgr2=0x7f8ee00ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 --2- 172.21.15.78:0/3972081677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee00ff630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0048c0 tx=0x7f8edc009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 -- 172.21.15.78:0/3972081677 shutdown_connections 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 --2- 172.21.15.78:0/3972081677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee00ff630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 -- 172.21.15.78:0/3972081677 >> 172.21.15.78:0/3972081677 conn(0x7f8ee00f70a0 msgr2=0x7f8ee00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 -- 172.21.15.78:0/3972081677 shutdown_connections 2024-04-26T04:45:07.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.331+0000 7f8ee7957700 1 -- 172.21.15.78:0/3972081677 wait complete. 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee7957700 1 Processor -- start 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee7957700 1 -- start start 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee7957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee7957700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee010d5d0 con 0x7f8ee0065d30 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee56f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee56f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:55034/0 (socket says 172.21.15.78:55034) 2024-04-26T04:45:07.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee56f3700 1 -- 172.21.15.78:0/1768156998 learned_addr learned my addr 172.21.15.78:0/1768156998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:07.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.332+0000 7f8ee56f3700 1 -- 172.21.15.78:0/1768156998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc005040 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ee56f3700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00b900 tx=0x7f8edc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8edc00b5c0 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8edc00be70 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee010d7d0 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8edc01c590 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.333+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee010e340 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f8edc004d30 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee0102f40 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ed5ffb700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 0x7f8ecc038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8edc0493e0 con 0x7f8ee0065d30 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ee4ef2700 1 -- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 msgr2=0x7f8ecc038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2855259383 2024-04-26T04:45:07.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.334+0000 7f8ee4ef2700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 0x7f8ecc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:45:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.339+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8edc01b350 con 0x7f8ee0065d30 2024-04-26T04:45:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.535+0000 7f8ee4ef2700 1 -- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 msgr2=0x7f8ecc038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2855259383 2024-04-26T04:45:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.535+0000 7f8ee4ef2700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 0x7f8ecc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:45:07.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.599+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ee010ed50 con 0x7f8ee0065d30 2024-04-26T04:45:07.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.599+0000 7f8ed5ffb700 1 -- 172.21.15.78:0/1768156998 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8edc021020 con 0x7f8ee0065d30 2024-04-26T04:45:07.601 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:07.602 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 msgr2=0x7f8ecc038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 0x7f8ecc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 msgr2=0x7f8ee010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00b900 tx=0x7f8edc009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 shutdown_connections 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:6800/2855259383,v1:172.21.15.77:6801/2855259383] conn(0x7f8ecc036920 0x7f8ecc038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 --2- 172.21.15.78:0/1768156998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ee0065d30 0x7f8ee010d090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 >> 172.21.15.78:0/1768156998 conn(0x7f8ee00f70a0 msgr2=0x7f8ee0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 shutdown_connections 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:07.601+0000 7f8ee7957700 1 -- 172.21.15.78:0/1768156998 wait complete. 2024-04-26T04:45:07.604 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:07.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:07 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1768156998' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: Active manager daemon smithi077.hkvsor restarted 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: osdmap e5: 0 total, 0 up, 0 in 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: mgrmap e15: smithi077.hkvsor(active, starting, since 0.00302048s) 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:08.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:08.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:45:08.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:45:09.148 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:09.148 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: mgrmap e16: smithi077.hkvsor(active, since 1.0072s) 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: [26/Apr/2024:04:45:09] ENGINE Bus STARTING 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: [26/Apr/2024:04:45:09] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: [26/Apr/2024:04:45:09] ENGINE Bus STARTED 2024-04-26T04:45:10.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 -- 172.21.15.78:0/2020824435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 msgr2=0x7ff40c0fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 --2- 172.21.15.78:0/2020824435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c0fbd40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0048c0 tx=0x7ff3fc009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 -- 172.21.15.78:0/2020824435 shutdown_connections 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 --2- 172.21.15.78:0/2020824435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c0fbd40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 -- 172.21.15.78:0/2020824435 >> 172.21.15.78:0/2020824435 conn(0x7ff40c0f7070 msgr2=0x7ff40c0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 -- 172.21.15.78:0/2020824435 shutdown_connections 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.910+0000 7ff41384b700 1 -- 172.21.15.78:0/2020824435 wait complete. 2024-04-26T04:45:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff41384b700 1 Processor -- start 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff41384b700 1 -- start start 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff41384b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff41384b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10f7f0 con 0x7ff40c0fb930 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff4115e7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff4115e7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:55052/0 (socket says 172.21.15.78:55052) 2024-04-26T04:45:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff4115e7700 1 -- 172.21.15.78:0/2139135183 learned_addr learned my addr 172.21.15.78:0/2139135183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.911+0000 7ff4115e7700 1 -- 172.21.15.78:0/2139135183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc005040 con 0x7ff40c0fb930 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff4115e7700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0096b0 tx=0x7ff3fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3fc00bed0 con 0x7ff40c0fb930 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c10f9f0 con 0x7ff40c0fb930 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3fc00b5c0 con 0x7ff40c0fb930 2024-04-26T04:45:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3fc01c5b0 con 0x7ff40c0fb930 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.912+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c110560 con 0x7ff40c0fb930 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.913+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7ff3fc01c820 con 0x7ff40c0fb930 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.913+0000 7ff401ffb700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 0x7ff3f8038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.913+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c105140 con 0x7ff40c0fb930 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.913+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff3fc049a70 con 0x7ff40c0fb930 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.914+0000 7ff410de6700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 0x7ff3f8038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.914+0000 7ff410de6700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 0x7ff3f8038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff408009990 tx=0x7ff408006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:10.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:10.920+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff3fc023300 con 0x7ff40c0fb930 2024-04-26T04:45:11.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.138+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7ff3fc013a80 con 0x7ff40c0fb930 2024-04-26T04:45:11.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.187+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff40c110f70 con 0x7ff40c0fb930 2024-04-26T04:45:11.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.188+0000 7ff401ffb700 1 -- 172.21.15.78:0/2139135183 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff3fc024e30 con 0x7ff40c0fb930 2024-04-26T04:45:11.190 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:11.190 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 msgr2=0x7ff3f8038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 0x7ff3f8038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff408009990 tx=0x7ff408006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 msgr2=0x7ff40c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0096b0 tx=0x7ff3fc009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 shutdown_connections 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3f8036710 0x7ff3f8038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 --2- 172.21.15.78:0/2139135183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff40c0fb930 0x7ff40c10f2b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 >> 172.21.15.78:0/2139135183 conn(0x7ff40c0f7070 msgr2=0x7ff40c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 shutdown_connections 2024-04-26T04:45:11.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:11.191+0000 7ff41384b700 1 -- 172.21.15.78:0/2139135183 wait complete. 2024-04-26T04:45:11.193 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:11.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:12.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:12 smithi077 ceph-mon[26311]: mgrmap e17: smithi077.hkvsor(active, since 2s) 2024-04-26T04:45:12.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:12 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2139135183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:13.373 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:13.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:14.194 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:45:15.437 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:15.437 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:15.437 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:15.437 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:15.437 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 -- 172.21.15.78:0/436272668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 msgr2=0x7fce600ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 --2- 172.21.15.78:0/436272668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce600ff6d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce500048c0 tx=0x7fce50009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 -- 172.21.15.78:0/436272668 shutdown_connections 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 --2- 172.21.15.78:0/436272668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce600ff6d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 -- 172.21.15.78:0/436272668 >> 172.21.15.78:0/436272668 conn(0x7fce600f70a0 msgr2=0x7fce600f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.625+0000 7fce67853700 1 -- 172.21.15.78:0/436272668 shutdown_connections 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce67853700 1 -- 172.21.15.78:0/436272668 wait complete. 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce67853700 1 Processor -- start 2024-04-26T04:45:15.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce67853700 1 -- start start 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce67853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce67853700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010d5e0 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce655ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce655ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:59224/0 (socket says 172.21.15.78:59224) 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.626+0000 7fce655ef700 1 -- 172.21.15.78:0/208924228 learned_addr learned my addr 172.21.15.78:0/208924228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce655ef700 1 -- 172.21.15.78:0/208924228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50005040 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce655ef700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b880 tx=0x7fce50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce5000bbc0 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce5000bd20 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6010d7e0 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce5001c690 con 0x7fce60065c90 2024-04-26T04:45:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.627+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6010e350 con 0x7fce60065c90 2024-04-26T04:45:15.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.628+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fce50013420 con 0x7fce60065c90 2024-04-26T04:45:15.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.628+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce60102f50 con 0x7fce60065c90 2024-04-26T04:45:15.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.628+0000 7fce55ffb700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 0x7fce4c038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:15.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.628+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fce50049d20 con 0x7fce60065c90 2024-04-26T04:45:15.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.629+0000 7fce64dee700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 0x7fce4c038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:15.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.629+0000 7fce64dee700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 0x7fce4c038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009990 tx=0x7fce5c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:15.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.633+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fce50013b10 con 0x7fce60065c90 2024-04-26T04:45:15.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.896+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fce6010ed60 con 0x7fce60065c90 2024-04-26T04:45:15.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.897+0000 7fce55ffb700 1 -- 172.21.15.78:0/208924228 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fce50011370 con 0x7fce60065c90 2024-04-26T04:45:15.898 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:15.898 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:15.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 msgr2=0x7fce4c038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 0x7fce4c038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009990 tx=0x7fce5c006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 msgr2=0x7fce6010d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce5000b880 tx=0x7fce50009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 shutdown_connections 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fce4c0369c0 0x7fce4c038e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 --2- 172.21.15.78:0/208924228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce60065c90 0x7fce6010d0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 >> 172.21.15.78:0/208924228 conn(0x7fce600f70a0 msgr2=0x7fce60103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 shutdown_connections 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:15.899+0000 7fce67853700 1 -- 172.21.15.78:0/208924228 wait complete. 2024-04-26T04:45:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/208924228' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:16.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:17.487 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:17.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:45:17.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-26T04:45:17.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:18.696 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:18.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:18 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:45:18.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:18 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:45:18.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:18 smithi077 ceph-mon[26311]: Deploying daemon crash.smithi078 on smithi078 2024-04-26T04:45:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- 172.21.15.78:0/3540317607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 msgr2=0x7f1a30102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 --2- 172.21.15.78:0/3540317607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30102e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a200048c0 tx=0x7f1a20009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- 172.21.15.78:0/3540317607 shutdown_connections 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 --2- 172.21.15.78:0/3540317607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30102e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- 172.21.15.78:0/3540317607 >> 172.21.15.78:0/3540317607 conn(0x7f1a300f85b0 msgr2=0x7f1a300fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- 172.21.15.78:0/3540317607 shutdown_connections 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- 172.21.15.78:0/3540317607 wait complete. 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 Processor -- start 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.289+0000 7f1a3664e700 1 -- start start 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a3664e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a3664e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a300707a0 con 0x7f1a30102ab0 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:59242/0 (socket says 172.21.15.78:59242) 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a2ffff700 1 -- 172.21.15.78:0/2191075147 learned_addr learned my addr 172.21.15.78:0/2191075147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.290+0000 7f1a2ffff700 1 -- 172.21.15.78:0/2191075147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20005040 con 0x7f1a30102ab0 2024-04-26T04:45:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a2ffff700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000b7f0 tx=0x7f1a20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a20009370 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a300709a0 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a20013420 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1a2001c590 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.291+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a30070e90 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.292+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f1a20013a70 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.292+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a300710d0 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.292+0000 7f1a2cff9700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 0x7f1a18038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.292+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1a20049400 con 0x7f1a30102ab0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.292+0000 7f1a2f7fe700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 0x7f1a18038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.293+0000 7f1a2f7fe700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 0x7f1a18038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009990 tx=0x7f1a24006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.297+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1a300710d0 con 0x7f1a30102ab0 2024-04-26T04:45:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.565+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1a300710d0 con 0x7f1a30102ab0 2024-04-26T04:45:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.566+0000 7f1a2cff9700 1 -- 172.21.15.78:0/2191075147 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1a300710d0 con 0x7f1a30102ab0 2024-04-26T04:45:20.567 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:20.567 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 msgr2=0x7f1a18038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 0x7f1a18038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009990 tx=0x7f1a24006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 msgr2=0x7f1a30070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000b7f0 tx=0x7f1a20009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 shutdown_connections 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1a18036970 0x7f1a18038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 --2- 172.21.15.78:0/2191075147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30102ab0 0x7f1a30070260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 >> 172.21.15.78:0/2191075147 conn(0x7f1a300f85b0 msgr2=0x7f1a300f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 shutdown_connections 2024-04-26T04:45:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:20.568+0000 7f1a3664e700 1 -- 172.21.15.78:0/2191075147 wait complete. 2024-04-26T04:45:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:20.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:20 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2191075147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:22.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:21 smithi077 ceph-mon[26311]: Deploying daemon node-exporter.smithi078 on smithi078 2024-04-26T04:45:22.341 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:22.341 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 -- 172.21.15.78:0/642994640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 msgr2=0x7f47000fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 --2- 172.21.15.78:0/642994640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fde10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46f00048c0 tx=0x7f46f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 -- 172.21.15.78:0/642994640 shutdown_connections 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 --2- 172.21.15.78:0/642994640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fde10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 -- 172.21.15.78:0/642994640 >> 172.21.15.78:0/642994640 conn(0x7f47000f74f0 msgr2=0x7f47000f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 -- 172.21.15.78:0/642994640 shutdown_connections 2024-04-26T04:45:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.079+0000 7f470609a700 1 -- 172.21.15.78:0/642994640 wait complete. 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f470609a700 1 Processor -- start 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f470609a700 1 -- start start 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f470609a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f470609a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000fd870 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:54202/0 (socket says 172.21.15.78:54202) 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f46ff7fe700 1 -- 172.21.15.78:0/652856970 learned_addr learned my addr 172.21.15.78:0/652856970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.080+0000 7f46ff7fe700 1 -- 172.21.15.78:0/652856970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0005040 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f46ff7fe700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b820 tx=0x7f46f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46f00093c0 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47000fbe10 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46f0013420 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46f001c5b0 con 0x7f47000fda40 2024-04-26T04:45:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.081+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47000fc280 con 0x7f47000fda40 2024-04-26T04:45:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.082+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f46f0013a90 con 0x7f47000fda40 2024-04-26T04:45:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.082+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700102580 con 0x7f47000fda40 2024-04-26T04:45:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.082+0000 7f46e7fff700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 0x7f46e8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.082+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f46f0049a70 con 0x7f47000fda40 2024-04-26T04:45:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.082+0000 7f46feffd700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 0x7f46e8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.083+0000 7f46feffd700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 0x7f46e8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009990 tx=0x7f46f4006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.087+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f46f00098a0 con 0x7f47000fda40 2024-04-26T04:45:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.351+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f470004bed0 con 0x7f47000fda40 2024-04-26T04:45:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.352+0000 7f46e7fff700 1 -- 172.21.15.78:0/652856970 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f46f0011370 con 0x7f47000fda40 2024-04-26T04:45:24.354 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:24.354 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 msgr2=0x7f46e8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 0x7f46e8038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009990 tx=0x7f46f4006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 msgr2=0x7f47000fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b820 tx=0x7f46f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 shutdown_connections 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f46e8036970 0x7f46e8038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 --2- 172.21.15.78:0/652856970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47000fda40 0x7f47000fb870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 >> 172.21.15.78:0/652856970 conn(0x7f47000f74f0 msgr2=0x7f47001036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 shutdown_connections 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:24.354+0000 7f470609a700 1 -- 172.21.15.78:0/652856970 wait complete. 2024-04-26T04:45:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:24.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:24.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:24.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:24 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/652856970' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:26.190 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:26.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T04:45:28.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:45:28.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- 172.21.15.78:0/2121815642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 msgr2=0x7f2a90102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 --2- 172.21.15.78:0/2121815642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90102e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0048c0 tx=0x7f2a8c009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- 172.21.15.78:0/2121815642 shutdown_connections 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 --2- 172.21.15.78:0/2121815642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90102e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- 172.21.15.78:0/2121815642 >> 172.21.15.78:0/2121815642 conn(0x7f2a900f85b0 msgr2=0x7f2a900fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- 172.21.15.78:0/2121815642 shutdown_connections 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- 172.21.15.78:0/2121815642 wait complete. 2024-04-26T04:45:28.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 Processor -- start 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.949+0000 7f2a97c84700 1 -- start start 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a97c84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a97c84700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90113670 con 0x7f2a90102ab0 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a95a20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a95a20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:54218/0 (socket says 172.21.15.78:54218) 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a95a20700 1 -- 172.21.15.78:0/1170497375 learned_addr learned my addr 172.21.15.78:0/1170497375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:28.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a95a20700 1 -- 172.21.15.78:0/1170497375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c005040 con 0x7f2a90102ab0 2024-04-26T04:45:28.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.950+0000 7f2a95a20700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b820 tx=0x7f2a8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:28.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.951+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a8c0093c0 con 0x7f2a90102ab0 2024-04-26T04:45:28.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.951+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a90113870 con 0x7f2a90102ab0 2024-04-26T04:45:28.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.951+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a8c013420 con 0x7f2a90102ab0 2024-04-26T04:45:28.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.951+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a8c01c5b0 con 0x7f2a90102ab0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.951+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a90113d60 con 0x7f2a90102ab0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a900fe1b0 con 0x7f2a90102ab0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f2a8c009520 con 0x7f2a90102ab0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a867fc700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 0x7f2a7c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2a8c049a60 con 0x7f2a90102ab0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a9521f700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 0x7f2a7c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:28.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.952+0000 7f2a9521f700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 0x7f2a7c038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009990 tx=0x7f2a80006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:28.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:28.957+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2a8c013a90 con 0x7f2a90102ab0 2024-04-26T04:45:29.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.219+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2a9005f5c0 con 0x7f2a90102ab0 2024-04-26T04:45:29.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.220+0000 7f2a867fc700 1 -- 172.21.15.78:0/1170497375 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2a8c011370 con 0x7f2a90102ab0 2024-04-26T04:45:29.221 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:29.221 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:29.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 msgr2=0x7f2a7c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:29.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 0x7f2a7c038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009990 tx=0x7f2a80006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:29.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 msgr2=0x7f2a90113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:29.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b820 tx=0x7f2a8c009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 shutdown_connections 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2a7c036970 0x7f2a7c038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 --2- 172.21.15.78:0/1170497375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a90102ab0 0x7f2a90113130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 >> 172.21.15.78:0/1170497375 conn(0x7f2a900f85b0 msgr2=0x7f2a900f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 shutdown_connections 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:29.222+0000 7f2a97c84700 1 -- 172.21.15.78:0/1170497375 wait complete. 2024-04-26T04:45:29.224 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:28 smithi077 ceph-mon[26311]: Deploying daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:45:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:28 smithi077 ceph-mon[26311]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:30.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:29 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1170497375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:30.789 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:30.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: Deploying daemon mon.smithi078 on smithi078 2024-04-26T04:45:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:31 smithi077 ceph-mon[26311]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:33.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:33 smithi077 ceph-mon[26311]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:33.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:33 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 -- 172.21.15.78:0/3901122008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 msgr2=0x7f7580102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 --2- 172.21.15.78:0/3901122008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f7580102f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75740048c0 tx=0x7f7574009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 -- 172.21.15.78:0/3901122008 shutdown_connections 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 --2- 172.21.15.78:0/3901122008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f7580102f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 -- 172.21.15.78:0/3901122008 >> 172.21.15.78:0/3901122008 conn(0x7f75800f85c0 msgr2=0x7f75800fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 -- 172.21.15.78:0/3901122008 shutdown_connections 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.110+0000 7f7586e49700 1 -- 172.21.15.78:0/3901122008 wait complete. 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7586e49700 1 Processor -- start 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7586e49700 1 -- start start 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7586e49700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7586e49700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580115930 con 0x7f7580102bb0 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7584be5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7584be5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.78:43612/0 (socket says 172.21.15.78:43612) 2024-04-26T04:45:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.111+0000 7f7584be5700 1 -- 172.21.15.78:0/1047408816 learned_addr learned my addr 172.21.15.78:0/1047408816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f7584be5700 1 -- 172.21.15.78:0/1047408816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574005040 con 0x7f7580102bb0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f7584be5700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f757400b820 tx=0x7f7574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f75740093c0 con 0x7f7580102bb0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7580115b30 con 0x7f7580102bb0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7574013420 con 0x7f7580102bb0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f757401c5b0 con 0x7f7580102bb0 2024-04-26T04:45:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.112+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7580116020 con 0x7f7580102bb0 2024-04-26T04:45:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.113+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f757401c7f0 con 0x7f7580102bb0 2024-04-26T04:45:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.113+0000 7f757d7fa700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 0x7f7570038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.113+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758005f5c0 con 0x7f7580102bb0 2024-04-26T04:45:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.113+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7574049ab0 con 0x7f7580102bb0 2024-04-26T04:45:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.113+0000 7f757ffff700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 0x7f7570038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.114+0000 7f757ffff700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 0x7f7570038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f756c009990 tx=0x7f756c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.118+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7574023300 con 0x7f7580102bb0 2024-04-26T04:45:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.407+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7580110a80 con 0x7f7580102bb0 2024-04-26T04:45:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.407+0000 7f757d7fa700 1 -- 172.21.15.78:0/1047408816 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7574011b60 con 0x7f7580102bb0 2024-04-26T04:45:34.409 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:34.409 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:42:57.348674Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:45:34.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.409+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 msgr2=0x7f7570038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:34.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.409+0000 7f7586e49700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 0x7f7570038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f756c009990 tx=0x7f756c006e40 comp rx=0 tx=0).stop 2024-04-26T04:45:34.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.409+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 msgr2=0x7f75801153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.409+0000 7f7586e49700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f757400b820 tx=0x7f7574009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 shutdown_connections 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f7570036970 0x7f7570038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 --2- 172.21.15.78:0/1047408816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7580102bb0 0x7f75801153f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 >> 172.21.15.78:0/1047408816 conn(0x7f75800f85c0 msgr2=0x7f75800f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 shutdown_connections 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:34.410+0000 7f7586e49700 1 -- 172.21.15.78:0/1047408816 wait complete. 2024-04-26T04:45:34.411 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 1 2024-04-26T04:45:34.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1047408816' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:36.284 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:45:36.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:45:36.537 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: mon.smithi077 calling monitor election 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: mon.smithi078 calling monitor election 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2521112547' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: mon.smithi077 is new leader, mons smithi077,smithi078 in quorum (ranks 0,1) 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: monmap e2: 2 mons at {smithi077=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],smithi078=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: fsmap 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: osdmap e5: 0 total, 0 up, 0 in 2024-04-26T04:45:40.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: mgrmap e17: smithi077.hkvsor(active, since 31s) 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: overall HEALTH_OK 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2521112547' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2521112547' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2521112547' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:45:40.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:39 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:39 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:40.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 -- 172.21.15.78:0/1667068600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf50003580 msgr2=0x7fbf50005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/1667068600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf50003580 0x7fbf50005a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf68065f20 tx=0x7fbf6400b040 comp rx=0 tx=0).stop 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 -- 172.21.15.78:0/1667068600 shutdown_connections 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/1667068600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf50003580 0x7fbf50005a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 -- 172.21.15.78:0/1667068600 >> 172.21.15.78:0/1667068600 conn(0x7fbf680f75d0 msgr2=0x7fbf680f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 -- 172.21.15.78:0/1667068600 shutdown_connections 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 -- 172.21.15.78:0/1667068600 wait complete. 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.949+0000 7fbf6efe9700 1 Processor -- start 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6efe9700 1 -- start start 2024-04-26T04:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6efe9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6efe9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 0x7fbf68113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6efe9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6811a3e0 con 0x7fbf68119e20 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6efe9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf68113da0 con 0x7fbf50003580 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41946/0 (socket says 172.21.15.78:41946) 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 -- 172.21.15.78:0/2447768959 learned_addr learned my addr 172.21.15.78:0/2447768959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 -- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 msgr2=0x7fbf681198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 -- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 msgr2=0x7fbf681198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf6cd85700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.950+0000 7fbf5ffff700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 0x7fbf68113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5ffff700 1 -- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 msgr2=0x7fbf681198e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5ffff700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5ffff700 1 -- 172.21.15.78:0/2447768959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf64005040 con 0x7fbf68119e20 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5ffff700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 0x7fbf68113860 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b790 tx=0x7fbf580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf58011810 con 0x7fbf68119e20 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf58011e40 con 0x7fbf68119e20 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf68114080 con 0x7fbf68119e20 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf5800f2e0 con 0x7fbf68119e20 2024-04-26T04:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.951+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf681145c0 con 0x7fbf68119e20 2024-04-26T04:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.952+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf680ff7e0 con 0x7fbf68119e20 2024-04-26T04:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.952+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fbf5800f460 con 0x7fbf68119e20 2024-04-26T04:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.953+0000 7fbf5d7fa700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 0x7fbf5406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.953+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbf58086350 con 0x7fbf68119e20 2024-04-26T04:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.953+0000 7fbf6cd85700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 0x7fbf5406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:40.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.953+0000 7fbf6cd85700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 0x7fbf5406b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf640045a0 tx=0x7fbf64018040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:40.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:40.958+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbf580572c0 con 0x7fbf68119e20 2024-04-26T04:45:41.082 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: mgrmap e18: smithi077.hkvsor(active, since 31s), standbys: smithi078.navvuw 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:41.083 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:40 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:41.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.224+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbf6810aeb0 con 0x7fbf68119e20 2024-04-26T04:45:41.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.225+0000 7fbf5d7fa700 1 -- 172.21.15.78:0/2447768959 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7fbf58015020 con 0x7fbf68119e20 2024-04-26T04:45:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-26T04:45:41.226 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","modified":"2024-04-26T04:45:34.756719Z","created":"2024-04-26T04:42:57.348674Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi077","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi078","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T04:45:41.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 msgr2=0x7fbf5406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 0x7fbf5406b810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf640045a0 tx=0x7fbf64018040 comp rx=0 tx=0).stop 2024-04-26T04:45:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 msgr2=0x7fbf68113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 0x7fbf68113860 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b790 tx=0x7fbf580091a0 comp rx=0 tx=0).stop 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 shutdown_connections 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbf54069360 0x7fbf5406b810 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf50003580 0x7fbf681198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 --2- 172.21.15.78:0/2447768959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf68119e20 0x7fbf68113860 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 >> 172.21.15.78:0/2447768959 conn(0x7fbf680f75d0 msgr2=0x7fbf680f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 shutdown_connections 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:45:41.227+0000 7fbf6efe9700 1 -- 172.21.15.78:0/2447768959 wait complete. 2024-04-26T04:45:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:dumped monmap epoch 2 2024-04-26T04:45:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: mgrmap e18: smithi077.hkvsor(active, since 31s), standbys: smithi078.navvuw 2024-04-26T04:45:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:45:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:40 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:45:41.803 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T04:45:41.803 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T04:45:42.061 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:45:42.183 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:41 smithi077 ceph-mon[26311]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:42.183 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:41 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/2447768959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:41 smithi078 ceph-mon[33746]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:41 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/2447768959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:43.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Reconfiguring mon.smithi077 (unknown last config time)... 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:43.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:43 smithi077 ceph-mon[26311]: Reconfiguring daemon mon.smithi077 on smithi077 2024-04-26T04:45:43.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Reconfiguring mon.smithi077 (unknown last config time)... 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:45:43.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:43.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:43 smithi078 ceph-mon[33746]: Reconfiguring daemon mon.smithi077 on smithi077 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.182+0000 7f5a2d46d700 1 -- 172.21.15.77:0/2937480457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fda30 msgr2=0x7f5a280fde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.182+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/2937480457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fda30 0x7f5a280fde00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a180048c0 tx=0x7f5a18009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 -- 172.21.15.77:0/2937480457 shutdown_connections 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/2937480457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fda30 0x7f5a280fde00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 -- 172.21.15.77:0/2937480457 >> 172.21.15.77:0/2937480457 conn(0x7f5a280f74f0 msgr2=0x7f5a280f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 -- 172.21.15.77:0/2937480457 shutdown_connections 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 -- 172.21.15.77:0/2937480457 wait complete. 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 Processor -- start 2024-04-26T04:45:44.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.183+0000 7f5a2d46d700 1 -- start start 2024-04-26T04:45:44.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a280fda30 0x7f5a280fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:44.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:44.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28119590 con 0x7f5a280fbe30 2024-04-26T04:45:44.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281196d0 con 0x7f5a280fda30 2024-04-26T04:45:44.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40018/0 (socket says 172.21.15.77:40018) 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 -- 172.21.15.77:0/3954462083 learned_addr learned my addr 172.21.15.77:0/3954462083 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 -- 172.21.15.77:0/3954462083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a280fda30 msgr2=0x7f5a280fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a26ffd700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a280fda30 0x7f5a280fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a280fda30 0x7f5a280fb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 -- 172.21.15.77:0/3954462083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18005040 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a267fc700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00b790 tx=0x7f5a1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1c010870 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a1c00f3e0 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a281198d0 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1c010ea0 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.184+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28119e10 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.185+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a281026d0 con 0x7f5a280fbe30 2024-04-26T04:45:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.186+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f5a1c00f550 con 0x7f5a280fbe30 2024-04-26T04:45:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.186+0000 7f5a0f7fe700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 0x7f5a1006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.186+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5a1c0866a0 con 0x7f5a280fbe30 2024-04-26T04:45:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.186+0000 7f5a26ffd700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 0x7f5a1006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.187+0000 7f5a26ffd700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 0x7f5a1006b7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009b80 tx=0x7f5a18009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:44.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.191+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5a1c053a30 con 0x7f5a280fbe30 2024-04-26T04:45:44.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.381+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5a28066520 con 0x7f5a280fbe30 2024-04-26T04:45:44.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.381+0000 7f5a0f7fe700 1 -- 172.21.15.77:0/3954462083 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f5a1c0140f0 con 0x7f5a280fbe30 2024-04-26T04:45:44.383 INFO:teuthology.orchestra.run.smithi077.stdout:# minimal ceph.conf for 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:45:44.383 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2024-04-26T04:45:44.384 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:45:44.384 INFO:teuthology.orchestra.run.smithi077.stdout: mon_host = [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] 2024-04-26T04:45:44.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 msgr2=0x7f5a1006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:44.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 0x7f5a1006b7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009b80 tx=0x7f5a18009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:44.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 msgr2=0x7f5a280fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:44.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00b790 tx=0x7f5a1c009250 comp rx=0 tx=0).stop 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 shutdown_connections 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5a10069320 0x7f5a1006b7d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a280fda30 0x7f5a280fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 --2- 172.21.15.77:0/3954462083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a280fbe30 0x7f5a280fc2a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.383+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 >> 172.21.15.77:0/3954462083 conn(0x7f5a280f74f0 msgr2=0x7f5a280f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.384+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 shutdown_connections 2024-04-26T04:45:44.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:44.384+0000 7f5a2d46d700 1 -- 172.21.15.77:0/3954462083 wait complete. 2024-04-26T04:45:44.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:44 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:44.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:44 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3954462083' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:44.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:44 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:44.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:44 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3954462083' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:45.201 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T04:45:45.201 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:45:45.202 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T04:45:45.227 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:45:45.227 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:45:45.292 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:45:45.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T04:45:45.318 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:45:45.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:45:45.381 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T04:45:45.381 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:45:45.382 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:45:45.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:45:45.397 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-04-26T04:45:45.451 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:45:45.451 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:45.451 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T04:45:45.451 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:45.451 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:45.452 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 04:45:00.284447853 +0000 2024-04-26T04:45:45.452 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 04:38:56.864862198 +0000 2024-04-26T04:45:45.452 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 04:38:56.864862198 +0000 2024-04-26T04:45:45.452 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 04:38:56.864862198 +0000 2024-04-26T04:45:45.452 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:45:45.513 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T04:45:45.513 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T04:45:45.513 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000267897 s, 1.9 MB/s 2024-04-26T04:45:45.514 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:45:45.570 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-04-26T04:45:45.624 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 04:45:00.284447853 +0000 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 04:38:57.214854511 +0000 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 04:38:57.214854511 +0000 2024-04-26T04:45:45.625 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 04:38:57.214854511 +0000 2024-04-26T04:45:45.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:45:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T04:45:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T04:45:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000217408 s, 2.4 MB/s 2024-04-26T04:45:45.689 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:45:45.745 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-04-26T04:45:45.800 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:45:45.800 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:45.800 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T04:45:45.800 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:45.800 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:45.801 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 04:45:00.285447833 +0000 2024-04-26T04:45:45.801 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 04:38:57.575846583 +0000 2024-04-26T04:45:45.801 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 04:38:57.575846583 +0000 2024-04-26T04:45:45.801 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 04:38:57.575846583 +0000 2024-04-26T04:45:45.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:45:45.860 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:45.860 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:45.860 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:45.860 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: Reconfiguring mgr.smithi077.hkvsor (unknown last config time)... 2024-04-26T04:45:45.861 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:45:45.861 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:45:45.861 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:45.861 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:45 smithi077 ceph-mon[26311]: Reconfiguring daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:45:45.862 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T04:45:45.862 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T04:45:45.862 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000173107 s, 3.0 MB/s 2024-04-26T04:45:45.864 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:45:45.920 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: Reconfiguring mgr.smithi077.hkvsor (unknown last config time)... 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:45.952 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:45 smithi078 ceph-mon[33746]: Reconfiguring daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-04-26 04:45:00.285447833 +0000 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 04:38:57.903839380 +0000 2024-04-26T04:45:45.975 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 04:38:57.903839380 +0000 2024-04-26T04:45:45.976 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-04-26 04:38:57.903839380 +0000 2024-04-26T04:45:45.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:45:46.037 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-04-26T04:45:46.037 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-04-26T04:45:46.037 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000185079 s, 2.8 MB/s 2024-04-26T04:45:46.039 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:45:46.094 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:45:46.094 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:45:46.110 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:45:46.110 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-26 04:45:32.978443526 +0000 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-26 04:38:56.859724896 +0000 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-26 04:38:56.859724896 +0000 2024-04-26T04:45:46.165 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-26 04:38:56.859724896 +0000 2024-04-26T04:45:46.166 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:45:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-26T04:45:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-26T04:45:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000285292 s, 1.8 MB/s 2024-04-26T04:45:46.228 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:45:46.284 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:46.339 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-26 04:45:32.979443511 +0000 2024-04-26T04:45:46.340 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-26 04:38:57.412715756 +0000 2024-04-26T04:45:46.340 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-26 04:38:57.412715756 +0000 2024-04-26T04:45:46.340 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-26 04:38:57.412715756 +0000 2024-04-26T04:45:46.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:45:46.401 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-26T04:45:46.402 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-26T04:45:46.402 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000222142 s, 2.3 MB/s 2024-04-26T04:45:46.403 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:45:46.458 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-26 04:45:32.979443511 +0000 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-26 04:38:57.752710137 +0000 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-26 04:38:57.752710137 +0000 2024-04-26T04:45:46.512 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-26 04:38:57.752710137 +0000 2024-04-26T04:45:46.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:45:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-26T04:45:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-26T04:45:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000229846 s, 2.2 MB/s 2024-04-26T04:45:46.576 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:45:46.632 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-26 04:45:32.980443495 +0000 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-26 04:38:58.084704650 +0000 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-26 04:38:58.084704650 +0000 2024-04-26T04:45:46.687 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-26 04:38:58.084704650 +0000 2024-04-26T04:45:46.688 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:45:46.749 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-26T04:45:46.749 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-26T04:45:46.749 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000173322 s, 3.0 MB/s 2024-04-26T04:45:46.750 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:45:46.805 INFO:tasks.cephadm:Deploying osd.0 on smithi077 with /dev/vg_nvme/lv_4... 2024-04-26T04:45:46.805 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T04:45:47.289 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:45:47.661 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: Reconfiguring crash.smithi077 (monmap changed)... 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: Reconfiguring daemon crash.smithi077 on smithi077 2024-04-26T04:45:47.662 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:47 smithi077 ceph-mon[26311]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:47.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:47.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:47.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: Reconfiguring crash.smithi077 (monmap changed)... 2024-04-26T04:45:47.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:45:47.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:47.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: Reconfiguring daemon crash.smithi077 on smithi077 2024-04-26T04:45:47.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:47 smithi078 ceph-mon[33746]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:48.810 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:48.810 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:48.810 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:48 smithi077 ceph-mon[26311]: Reconfiguring alertmanager.smithi077 (dependencies changed)... 2024-04-26T04:45:48.810 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:48 smithi077 ceph-mon[26311]: Reconfiguring daemon alertmanager.smithi077 on smithi077 2024-04-26T04:45:49.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:49.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:49.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:48 smithi078 ceph-mon[33746]: Reconfiguring alertmanager.smithi077 (dependencies changed)... 2024-04-26T04:45:49.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:48 smithi078 ceph-mon[33746]: Reconfiguring daemon alertmanager.smithi077 on smithi077 2024-04-26T04:45:49.626 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T04:45:49.626 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T04:45:49.626 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T04:45:49.626 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T04:45:49.626 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112369 s, 933 MB/s 2024-04-26T04:45:49.627 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T04:45:49.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:49 smithi077 ceph-mon[26311]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:49 smithi078 ceph-mon[33746]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:50.237 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-04-26T04:45:51.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:51 smithi077 ceph-mon[26311]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:51 smithi078 ceph-mon[33746]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:52.598 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:45:53.876 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:53 smithi077 ceph-mon[26311]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:53.876 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:53 smithi078 ceph-mon[33746]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:54.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- 172.21.15.77:0/2971203641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 msgr2=0x7f48e0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/2971203641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e0102340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48d0004940 tx=0x7f48d0009e30 comp rx=0 tx=0).stop 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- 172.21.15.77:0/2971203641 shutdown_connections 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/2971203641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e0102340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/2971203641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e00ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- 172.21.15.77:0/2971203641 >> 172.21.15.77:0/2971203641 conn(0x7f48e00f7430 msgr2=0x7f48e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- 172.21.15.77:0/2971203641 shutdown_connections 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- 172.21.15.77:0/2971203641 wait complete. 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 Processor -- start 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.295+0000 7f48e7ad7700 1 -- start start 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e010e560 con 0x7f48e00fff60 2024-04-26T04:45:54.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e010edb0 con 0x7f48e0066240 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43410/0 (socket says 172.21.15.77:43410) 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 -- 172.21.15.77:0/4099133142 learned_addr learned my addr 172.21.15.77:0/4099133142 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5873700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 -- 172.21.15.77:0/4099133142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 msgr2=0x7f48e010da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e010da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 -- 172.21.15.77:0/4099133142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0005040 con 0x7f48e00fff60 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5873700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e5072700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009150 tx=0x7f48d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d000e040 con 0x7f48e00fff60 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48d000bbf0 con 0x7f48e00fff60 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e010f030 con 0x7f48e00fff60 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48d001c630 con 0x7f48e00fff60 2024-04-26T04:45:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.296+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e010f570 con 0x7f48e00fff60 2024-04-26T04:45:54.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.297+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e005f5c0 con 0x7f48e00fff60 2024-04-26T04:45:54.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.298+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f48d0013420 con 0x7f48e00fff60 2024-04-26T04:45:54.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.298+0000 7f48d67fc700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 0x7f48cc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:54.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.298+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f48d00874a0 con 0x7f48e00fff60 2024-04-26T04:45:54.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.298+0000 7f48e5873700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 0x7f48cc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:54.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.298+0000 7f48e5873700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 0x7f48cc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009fd0 tx=0x7f48dc009f00 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.303+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f48d0058340 con 0x7f48e00fff60 2024-04-26T04:45:54.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:45:54.502+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f48e010f8a0 con 0x7f48cc069210 2024-04-26T04:45:54.838 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:45:54.838 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:45:54.838 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:45:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:45:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:55.918 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:55 smithi077 ceph-mon[26311]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:55.918 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:55 smithi077 ceph-mon[26311]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:55 smithi078 ceph-mon[33746]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:55 smithi078 ceph-mon[33746]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:57.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:57 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:57.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:57 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:57.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:57 smithi077 ceph-mon[26311]: Reconfiguring grafana.smithi077 (dependencies changed)... 2024-04-26T04:45:57.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:57 smithi077 ceph-mon[26311]: Reconfiguring daemon grafana.smithi077 on smithi077 2024-04-26T04:45:57.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:57 smithi077 ceph-mon[26311]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:57 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:57 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:45:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:57 smithi078 ceph-mon[33746]: Reconfiguring grafana.smithi077 (dependencies changed)... 2024-04-26T04:45:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:57 smithi078 ceph-mon[33746]: Reconfiguring daemon grafana.smithi077 on smithi077 2024-04-26T04:45:57.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:57 smithi078 ceph-mon[33746]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:58.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:58 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/197902390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "074d417d-f631-4ee8-b43f-670fc24d4908"}]: dispatch 2024-04-26T04:45:58.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:58 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/197902390' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "074d417d-f631-4ee8-b43f-670fc24d4908"}]': finished 2024-04-26T04:45:58.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:58 smithi077 ceph-mon[26311]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T04:45:58.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:58 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:45:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:58 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/197902390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "074d417d-f631-4ee8-b43f-670fc24d4908"}]: dispatch 2024-04-26T04:45:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:58 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/197902390' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "074d417d-f631-4ee8-b43f-670fc24d4908"}]': finished 2024-04-26T04:45:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:58 smithi078 ceph-mon[33746]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T04:45:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:58 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:45:59.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:59 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1600675647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:45:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:45:59 smithi077 ceph-mon[26311]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:45:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:59 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1600675647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:45:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:45:59 smithi078 ceph-mon[33746]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:01.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:01 smithi077 ceph-mon[26311]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:01.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:01 smithi078 ceph-mon[33746]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:03.725 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:03 smithi077 ceph-mon[26311]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:03.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:03 smithi078 ceph-mon[33746]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:05.715 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:05 smithi077 ceph-mon[26311]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:05 smithi078 ceph-mon[33746]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:07.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:07 smithi077 ceph-mon[26311]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:07.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:07.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:07 smithi078 ceph-mon[33746]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:08.626 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:08 smithi077 ceph-mon[26311]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:46:08.626 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:08 smithi077 ceph-mon[26311]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:46:08.626 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:46:08.626 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:08.626 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:08 smithi078 ceph-mon[33746]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:46:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:08 smithi078 ceph-mon[33746]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:46:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:46:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:08.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:09.548 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:09 smithi077 ceph-mon[26311]: Deploying daemon osd.0 on smithi077 2024-04-26T04:46:09.548 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:09 smithi077 ceph-mon[26311]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:09 smithi078 ceph-mon[33746]: Deploying daemon osd.0 on smithi077 2024-04-26T04:46:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:09 smithi078 ceph-mon[33746]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:11.534 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:11 smithi077 ceph-mon[26311]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:11 smithi078 ceph-mon[33746]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:13.830 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:13.830 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:13.830 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:13.830 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: Reconfiguring crash.smithi078 (monmap changed)... 2024-04-26T04:46:13.830 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:46:13.831 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:13.831 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:13 smithi077 ceph-mon[26311]: Reconfiguring daemon crash.smithi078 on smithi078 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: Reconfiguring crash.smithi078 (monmap changed)... 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:13.976 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:13 smithi078 ceph-mon[33746]: Reconfiguring daemon crash.smithi078 on smithi078 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: Reconfiguring mgr.smithi078.navvuw (monmap changed)... 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: Reconfiguring daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:46:15.269 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:15 smithi077 ceph-mon[26311]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: Reconfiguring mgr.smithi078.navvuw (monmap changed)... 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: Reconfiguring daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:46:15.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:15 smithi078 ceph-mon[33746]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:16.570 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:16.570 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:16.570 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: Reconfiguring mon.smithi078 (monmap changed)... 2024-04-26T04:46:16.570 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:46:16.571 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:46:16.571 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:16.571 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:16 smithi078 ceph-mon[33746]: Reconfiguring daemon mon.smithi078 on smithi078 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: Reconfiguring mon.smithi078 (monmap changed)... 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:16.664 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:16 smithi077 ceph-mon[26311]: Reconfiguring daemon mon.smithi078 on smithi078 2024-04-26T04:46:17.858 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:46:17.859 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.860 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T04:46:17.889 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:17.890 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:19.692 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:19 smithi077 ceph-mon[26311]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:19.693 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:19 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:19.693 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:19 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:19 smithi078 ceph-mon[33746]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:19 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:19 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:20.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.969+0000 7f48d67fc700 1 -- 172.21.15.77:0/4099133142 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f48e010f8a0 con 0x7f48cc069210 2024-04-26T04:46:20.971 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 0 on host 'smithi077' 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 msgr2=0x7f48cc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 0x7f48cc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009fd0 tx=0x7f48dc009f00 comp rx=0 tx=0).stop 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 msgr2=0x7f48e010df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009150 tx=0x7f48d0009e30 comp rx=0 tx=0).stop 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 shutdown_connections 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f48cc069210 0x7f48cc06b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48e0066240 0x7f48e010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 --2- 172.21.15.77:0/4099133142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48e00fff60 0x7f48e010df60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 >> 172.21.15.77:0/4099133142 conn(0x7f48e00f7430 msgr2=0x7f48e00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 shutdown_connections 2024-04-26T04:46:20.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:20.971+0000 7f48e7ad7700 1 -- 172.21.15.77:0/4099133142 wait complete. 2024-04-26T04:46:21.548 DEBUG:teuthology.orchestra.run.smithi077:osd.0> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T04:46:21.551 INFO:tasks.cephadm:Deploying osd.1 on smithi077 with /dev/vg_nvme/lv_3... 2024-04-26T04:46:21.551 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T04:46:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:21 smithi077 ceph-mon[26311]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:21 smithi077 ceph-mon[26311]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:46:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:21 smithi078 ceph-mon[33746]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:21 smithi078 ceph-mon[33746]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:46:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:46:23.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-mon[26311]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:46:23.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-mon[26311]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T04:46:23.239 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:23.239 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-mon[26311]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:46:23.239 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[44715]: 2024-04-26T04:46:22.953+0000 7f272c480700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T04:46:23.239 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:46:22 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[44715]: 2024-04-26T04:46:22.957+0000 7f272927f700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:46:23.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:22 smithi078 ceph-mon[33746]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:46:23.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:22 smithi078 ceph-mon[33746]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T04:46:23.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:22 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:23.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:22 smithi078 ceph-mon[33746]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:46:24.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: osdmap e8: 1 total, 0 up, 1 in 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:46:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: osdmap e8: 1 total, 0 up, 1 in 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:25.069 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:24 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:46:25.069 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:24 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:46:25.069 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:24 smithi077 ceph-mon[26311]: osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] boot 2024-04-26T04:46:25.070 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:24 smithi077 ceph-mon[26311]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T04:46:25.070 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:24 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:46:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:24 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:46:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:24 smithi078 ceph-mon[33746]: osd.0 [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] boot 2024-04-26T04:46:25.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:24 smithi078 ceph-mon[33746]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T04:46:25.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126201 s, 831 MB/s 2024-04-26T04:46:25.522 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T04:46:26.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:25 smithi077 ceph-mon[26311]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:26.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:25 smithi077 ceph-mon[26311]: osdmap e10: 1 total, 1 up, 1 in 2024-04-26T04:46:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:25 smithi078 ceph-mon[33746]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:46:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:25 smithi078 ceph-mon[33746]: osdmap e10: 1 total, 1 up, 1 in 2024-04-26T04:46:26.921 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-04-26T04:46:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:46:28.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:27 smithi077 ceph-mon[26311]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:27 smithi078 ceph-mon[33746]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:29.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- 172.21.15.77:0/3024421330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 msgr2=0x7fd8ec0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:29.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 --2- 172.21.15.77:0/3024421330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec0fe440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004940 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:46:29.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- 172.21.15.77:0/3024421330 shutdown_connections 2024-04-26T04:46:29.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 --2- 172.21.15.77:0/3024421330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec0fe440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 --2- 172.21.15.77:0/3024421330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 0x7fd8ec0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- 172.21.15.77:0/3024421330 >> 172.21.15.77:0/3024421330 conn(0x7fd8ec0f8540 msgr2=0x7fd8ec0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- 172.21.15.77:0/3024421330 shutdown_connections 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- 172.21.15.77:0/3024421330 wait complete. 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 Processor -- start 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.544+0000 7fd8f3179700 1 -- start start 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8f3179700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 0x7fd8ec10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8f3179700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8f3179700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec10e450 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8f3179700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec10e5c0 con 0x7fd8ec0fce60 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35074/0 (socket says 172.21.15.77:35074) 2024-04-26T04:46:29.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 -- 172.21.15.77:0/2617661585 learned_addr learned my addr 172.21.15.77:0/2617661585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8f0f15700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 0x7fd8ec10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 -- 172.21.15.77:0/2617661585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 msgr2=0x7fd8ec10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 0x7fd8ec10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 -- 172.21.15.77:0/2617661585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc005040 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e3fff700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009150 tx=0x7fd8dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.545+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc00e040 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.546+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec112fc0 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.546+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8dc004290 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.546+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc01c630 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.546+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec113550 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.547+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec04c710 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.547+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fd8dc00bbf0 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.547+0000 7fd8e17fa700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 0x7fd8d406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.547+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fd8dc086f80 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.548+0000 7fd8f0f15700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 0x7fd8d406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.548+0000 7fd8f0f15700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 0x7fd8d406b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8007fd0 tx=0x7fd8e8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:29.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.553+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8dc0b9250 con 0x7fd8ec0fdfd0 2024-04-26T04:46:29.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:29.756+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8ec1023e0 con 0x7fd8d4069210 2024-04-26T04:46:29.865 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi077 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi077 to 12864M 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:29.866 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi077 2024-04-26T04:46:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi077 to 12864M 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:30.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:30.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:46:30.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:46:30.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:30.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:30.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:30 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:31.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:31.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:46:31.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:46:31.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:31.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:31.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:30 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:31.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:31 smithi077 ceph-mon[26311]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:31 smithi078 ceph-mon[33746]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:33.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:33 smithi077 ceph-mon[26311]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:33 smithi078 ceph-mon[33746]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:34.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2115059452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beee2092-5b6a-45a9-93ae-0f368a2d613f"}]: dispatch 2024-04-26T04:46:34.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2115059452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beee2092-5b6a-45a9-93ae-0f368a2d613f"}]': finished 2024-04-26T04:46:34.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:34 smithi077 ceph-mon[26311]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T04:46:34.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:34 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:34 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2115059452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beee2092-5b6a-45a9-93ae-0f368a2d613f"}]: dispatch 2024-04-26T04:46:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:34 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2115059452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beee2092-5b6a-45a9-93ae-0f368a2d613f"}]': finished 2024-04-26T04:46:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:34 smithi078 ceph-mon[33746]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T04:46:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:34 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2592423430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:35.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2592423430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:36.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:37.926 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:37 smithi077 ceph-mon[26311]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:37 smithi078 ceph-mon[33746]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:38.980 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:39.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:39.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:39 smithi077 ceph-mon[26311]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:39 smithi078 ceph-mon[33746]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:41.916 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:41 smithi077 ceph-mon[26311]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:41.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:41 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:46:41.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:41 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:41 smithi078 ceph-mon[33746]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:41 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:46:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:41 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:42.849 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:42 smithi077 ceph-mon[26311]: Deploying daemon osd.1 on smithi077 2024-04-26T04:46:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:42 smithi078 ceph-mon[33746]: Deploying daemon osd.1 on smithi077 2024-04-26T04:46:43.873 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:43 smithi077 ceph-mon[26311]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:43.918 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:43 smithi078 ceph-mon[33746]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:45.708 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:45 smithi077 ceph-mon[26311]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:46.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:45 smithi078 ceph-mon[33746]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:47.560 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:47.561 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:47.561 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:47 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:47.561 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:47 smithi078 ceph-mon[33746]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:47.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:47.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:47.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:47 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:47.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:47 smithi077 ceph-mon[26311]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:49.481 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:49.481 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:49.481 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:49 smithi077 ceph-mon[26311]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:49 smithi078 ceph-mon[33746]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.259+0000 7fd8e17fa700 1 -- 172.21.15.77:0/2617661585 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd8ec1023e0 con 0x7fd8d4069210 2024-04-26T04:46:50.260 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 1 on host 'smithi077' 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 msgr2=0x7fd8d406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 0x7fd8d406b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8007fd0 tx=0x7fd8e8009040 comp rx=0 tx=0).stop 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 msgr2=0x7fd8ec10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009150 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 shutdown_connections 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd8d4069210 0x7fd8d406b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8ec0fce60 0x7fd8ec10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 --2- 172.21.15.77:0/2617661585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8ec0fdfd0 0x7fd8ec10de50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 >> 172.21.15.77:0/2617661585 conn(0x7fd8ec0f8540 msgr2=0x7fd8ec0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 shutdown_connections 2024-04-26T04:46:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:50.262+0000 7fd8f3179700 1 -- 172.21.15.77:0/2617661585 wait complete. 2024-04-26T04:46:51.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:51 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:51.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:51 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:51.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:51 smithi077 ceph-mon[26311]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:46:51.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:51 smithi077 ceph-mon[26311]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:51.469 DEBUG:teuthology.orchestra.run.smithi077:osd.1> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T04:46:51.471 INFO:tasks.cephadm:Deploying osd.2 on smithi077 with /dev/vg_nvme/lv_2... 2024-04-26T04:46:51.472 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T04:46:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:51 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:51 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:51 smithi078 ceph-mon[33746]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:46:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:51 smithi078 ceph-mon[33746]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:52.190 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:46:52.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-mon[26311]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:46:52.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-mon[26311]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T04:46:52.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:52.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-mon[26311]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:46:52.493 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[52055]: 2024-04-26T04:46:52.224+0000 7fa252770700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T04:46:52.493 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:46:52 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[52055]: 2024-04-26T04:46:52.229+0000 7fa24cd6a700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:46:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:52 smithi078 ceph-mon[33746]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:46:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:52 smithi078 ceph-mon[33746]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T04:46:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:52 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:52 smithi078 ceph-mon[33746]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:46:53.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:53 smithi077 ceph-mon[26311]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:46:53.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:53 smithi077 ceph-mon[26311]: osdmap e13: 2 total, 1 up, 2 in 2024-04-26T04:46:53.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:53.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:53.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:53 smithi077 ceph-mon[26311]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:53 smithi078 ceph-mon[33746]: from='osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:46:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:53 smithi078 ceph-mon[33746]: osdmap e13: 2 total, 1 up, 2 in 2024-04-26T04:46:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:53 smithi078 ceph-mon[33746]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:46:54.311 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:46:54.311 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:46:54.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:54.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] boot 2024-04-26T04:46:54.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T04:46:54.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:54.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:46:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:46:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: osd.1 [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] boot 2024-04-26T04:46:54.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T04:46:54.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:54.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:54 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T04:46:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T04:46:54.764 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-04-26T04:46:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-04-26T04:46:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115498 s, 908 MB/s 2024-04-26T04:46:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-04-26T04:46:55.382 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:55 smithi077 ceph-mon[26311]: osdmap e15: 2 total, 2 up, 2 in 2024-04-26T04:46:55.382 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:55 smithi077 ceph-mon[26311]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:55.382 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:55.382 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:55.407 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-04-26T04:46:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:55 smithi078 ceph-mon[33746]: osdmap e15: 2 total, 2 up, 2 in 2024-04-26T04:46:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:55 smithi078 ceph-mon[33746]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:46:56.683 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:46:57.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:57 smithi077 ceph-mon[26311]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:57 smithi078 ceph-mon[33746]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 -- 172.21.15.77:0/741599366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 msgr2=0x7f0b8c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/741599366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c0fc150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004940 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 -- 172.21.15.77:0/741599366 shutdown_connections 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/741599366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 0x7f0b8c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/741599366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c0fc150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 -- 172.21.15.77:0/741599366 >> 172.21.15.77:0/741599366 conn(0x7f0b8c0f74c0 msgr2=0x7f0b8c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 -- 172.21.15.77:0/741599366 shutdown_connections 2024-04-26T04:46:58.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.364+0000 7f0b90bd8700 1 -- 172.21.15.77:0/741599366 wait complete. 2024-04-26T04:46:58.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 Processor -- start 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 -- start start 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 0x7f0b8c10ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c10d4e0 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b90bd8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c10dcd0 con 0x7f0b8c0fcff0 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46538/0 (socket says 172.21.15.77:46538) 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 -- 172.21.15.77:0/2952471612 learned_addr learned my addr 172.21.15.77:0/2952471612 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b83fff700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 0x7f0b8c10ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:58.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 -- 172.21.15.77:0/2952471612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 msgr2=0x7f0b8c10ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 0x7f0b8c10ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 -- 172.21.15.77:0/2952471612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c005040 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b8a59c700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b3b0 tx=0x7f0b7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.365+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c00e040 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.366+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b7c00bcc0 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.366+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c10df50 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.366+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c01c630 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.366+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c10e510 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.367+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c05f5c0 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.367+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f0b7c013980 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.367+0000 7f0b82ffd700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 0x7f0b7806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:58.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.367+0000 7f0b83fff700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 0x7f0b7806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:58.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.367+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f0b7c0872b0 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.368+0000 7f0b83fff700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 0x7f0b7806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:58.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.372+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0b7c057cc0 con 0x7f0b8c0fbd40 2024-04-26T04:46:58.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:46:58.570+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b8c10e7a0 con 0x7f0b78069210 2024-04-26T04:46:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:59 smithi078 ceph-mon[33746]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:59 smithi078 ceph-mon[33746]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:59 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:46:59.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:59 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:46:59.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:46:59 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:59 smithi077 ceph-mon[26311]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:46:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:59 smithi077 ceph-mon[26311]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:59 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:46:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:59 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:46:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:46:59 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi077 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi077 to 6432M 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi077 2024-04-26T04:47:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi077 to 6432M 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:02.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:02.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:02 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3734717846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c0054c-1766-462b-9bc6-fe56deac78e4"}]: dispatch 2024-04-26T04:47:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:02 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3734717846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c0054c-1766-462b-9bc6-fe56deac78e4"}]': finished 2024-04-26T04:47:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:02 smithi077 ceph-mon[26311]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T04:47:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:02 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3907817439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:02 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3734717846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c0054c-1766-462b-9bc6-fe56deac78e4"}]: dispatch 2024-04-26T04:47:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:02 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3734717846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c0054c-1766-462b-9bc6-fe56deac78e4"}]': finished 2024-04-26T04:47:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:02 smithi078 ceph-mon[33746]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T04:47:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:02 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3907817439' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:04.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:03 smithi077 ceph-mon[26311]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:03 smithi078 ceph-mon[33746]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:06.128 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:06.129 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:06.129 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:06.129 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:06.129 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:06.129 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:08.212 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:07 smithi077 ceph-mon[26311]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:07 smithi078 ceph-mon[33746]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:09.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:09 smithi077 ceph-mon[26311]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:09.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:47:09.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:09 smithi078 ceph-mon[33746]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:47:10.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:10.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:10 smithi077 ceph-mon[26311]: Deploying daemon osd.2 on smithi077 2024-04-26T04:47:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:10 smithi078 ceph-mon[33746]: Deploying daemon osd.2 on smithi077 2024-04-26T04:47:11.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:11 smithi077 ceph-mon[26311]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:11 smithi078 ceph-mon[33746]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:13 smithi077 ceph-mon[26311]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:13 smithi078 ceph-mon[33746]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:15.213 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:15.214 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:15.214 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:15.214 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:15 smithi077 ceph-mon[26311]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:15.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:15.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:15.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:15.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:15 smithi078 ceph-mon[33746]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:17.669 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:17 smithi077 ceph-mon[26311]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:17.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:17 smithi078 ceph-mon[33746]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:17.995 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 2 on host 'smithi077' 2024-04-26T04:47:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.993+0000 7f0b82ffd700 1 -- 172.21.15.77:0/2952471612 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b8c10e7a0 con 0x7f0b78069210 2024-04-26T04:47:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 msgr2=0x7f0b7806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 0x7f0b7806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b74006fd0 tx=0x7f0b74008040 comp rx=0 tx=0).stop 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 msgr2=0x7f0b8c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b3b0 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 shutdown_connections 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0b78069210 0x7f0b7806b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b8c0fbd40 0x7f0b8c10c910 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 --2- 172.21.15.77:0/2952471612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b8c0fcff0 0x7f0b8c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 >> 172.21.15.77:0/2952471612 conn(0x7f0b8c0f74c0 msgr2=0x7f0b8c0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 shutdown_connections 2024-04-26T04:47:17.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:47:17.996+0000 7f0b90bd8700 1 -- 172.21.15.77:0/2952471612 wait complete. 2024-04-26T04:47:19.226 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:18 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:19.226 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:18 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:19.226 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:18 smithi077 ceph-mon[26311]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:47:19.252 DEBUG:teuthology.orchestra.run.smithi077:osd.2> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T04:47:19.254 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_4... 2024-04-26T04:47:19.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T04:47:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:18 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:18 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:18 smithi078 ceph-mon[33746]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:47:19.511 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:47:20.156 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-mon[26311]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:20.156 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-mon[26311]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:47:20.156 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-mon[26311]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T04:47:20.156 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:20.156 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-mon[26311]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:47:20.157 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:19 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[59848]: 2024-04-26T04:47:19.995+0000 7f4abfa1d700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T04:47:20.157 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:20 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[59848]: 2024-04-26T04:47:20.003+0000 7f4aba818700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:47:20.233 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:19 smithi078 ceph-mon[33746]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:20.233 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:19 smithi078 ceph-mon[33746]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:47:20.233 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:19 smithi078 ceph-mon[33746]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T04:47:20.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:19 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:20.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:19 smithi078 ceph-mon[33746]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:47:21.254 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T04:47:21.255 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T04:47:21.255 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-26T04:47:21.255 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-26T04:47:21.255 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129915 s, 807 MB/s 2024-04-26T04:47:21.255 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-26T04:47:21.257 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:47:21.257 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: osdmap e18: 3 total, 2 up, 3 in 2024-04-26T04:47:21.257 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:21.258 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:21.258 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:21.258 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:21.258 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:21 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: from='osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: osdmap e18: 3 total, 2 up, 3 in 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:21.266 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:21 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:21.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-04-26T04:47:22.108 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:47:22.214 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:22 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:47:22.215 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:22 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:47:22.215 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:22 smithi078 ceph-mon[33746]: osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] boot 2024-04-26T04:47:22.215 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:22 smithi078 ceph-mon[33746]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T04:47:22.215 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:22 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:22.292 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:22 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:47:22.292 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:22 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:47:22.292 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:22 smithi077 ceph-mon[26311]: osd.2 [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] boot 2024-04-26T04:47:22.292 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:22 smithi077 ceph-mon[26311]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T04:47:22.292 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:22 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:47:23.086 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:23 smithi078 ceph-mon[33746]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T04:47:23.086 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:23 smithi078 ceph-mon[33746]: pgmap v75: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:23.086 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-26T04:47:23.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:23 smithi077 ceph-mon[26311]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T04:47:23.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:23 smithi077 ceph-mon[26311]: pgmap v75: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:23.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-26T04:47:23.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 -- 172.21.15.78:0/2704487376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fdfd0 msgr2=0x7f74440fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:23.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 --2- 172.21.15.78:0/2704487376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fdfd0 0x7f74440fe440 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7434004940 tx=0x7f7434009e30 comp rx=0 tx=0).stop 2024-04-26T04:47:23.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 -- 172.21.15.78:0/2704487376 shutdown_connections 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 --2- 172.21.15.78:0/2704487376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fdfd0 0x7f74440fe440 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 --2- 172.21.15.78:0/2704487376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fce60 0x7f74440fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 -- 172.21.15.78:0/2704487376 >> 172.21.15.78:0/2704487376 conn(0x7f74440f8540 msgr2=0x7f74440fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 -- 172.21.15.78:0/2704487376 shutdown_connections 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.553+0000 7f7448835700 1 -- 172.21.15.78:0/2704487376 wait complete. 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 Processor -- start 2024-04-26T04:47:23.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 -- start start 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fdfd0 0x7f744410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744410e450 con 0x7f74440fdfd0 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7448835700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744410e5c0 con 0x7f74440fce60 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:23.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59518/0 (socket says 172.21.15.78:59518) 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 -- 172.21.15.78:0/2346037169 learned_addr learned my addr 172.21.15.78:0/2346037169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f7441d9b700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fdfd0 0x7f744410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 -- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fdfd0 msgr2=0x7f744410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fdfd0 0x7f744410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.554+0000 7f744259c700 1 -- 172.21.15.78:0/2346037169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434005040 con 0x7f74440fce60 2024-04-26T04:47:23.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f744259c700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7438002a50 tx=0x7f74380046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7438007cc0 con 0x7f74440fce60 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f743800f040 con 0x7f74440fce60 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444112fc0 con 0x7f74440fce60 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743800da30 con 0x7f74440fce60 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.555+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7444113550 con 0x7f74440fce60 2024-04-26T04:47:23.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.556+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744404c710 con 0x7f74440fce60 2024-04-26T04:47:23.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.556+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f743800db90 con 0x7f74440fce60 2024-04-26T04:47:23.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.556+0000 7f7432ffd700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 0x7f742c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:23.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.557+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f74380864d0 con 0x7f74440fce60 2024-04-26T04:47:23.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.557+0000 7f7441d9b700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 0x7f742c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:23.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.557+0000 7f7441d9b700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 0x7f742c06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f743400b970 tx=0x7f7434009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:23.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.561+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7438056c30 con 0x7f74440fce60 2024-04-26T04:47:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:23.754+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7444113790 con 0x7f742c069210 2024-04-26T04:47:24.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-26T04:47:24.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T04:47:24.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi077 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi077 to 4288M 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:24.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:24.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:24 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:24.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi077 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi077 to 4288M 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:24.460 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:24 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:24.992 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:24 smithi077 sudo[64170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:47:24.992 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:24 smithi077 sudo[64170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:24.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:24 smithi077 sudo[64170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:24.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:24 smithi077 sudo[64170]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:25.287 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:25 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T04:47:25.288 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:25 smithi077 ceph-mon[26311]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T04:47:25.288 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:25 smithi077 ceph-mon[26311]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:25.288 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64243]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:47:25.288 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:25.288 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64243]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:25.288 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64243]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:25.322 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:25 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T04:47:25.322 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:25 smithi078 ceph-mon[33746]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T04:47:25.322 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:25 smithi078 ceph-mon[33746]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:25.546 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:47:25.546 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:25.546 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:25.546 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64279]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:25.862 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64284]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:47:25.862 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64284]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:25.862 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64284]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:25.863 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64284]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:26.126 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64333]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:47:26.126 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64333]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:26.126 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64333]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:26.126 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:25 smithi077 sudo[64333]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: osdmap e23: 3 total, 3 up, 3 in 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/883622837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]: dispatch 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]: dispatch 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]': finished 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T04:47:26.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:26.389 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64378]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:47:26.390 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:26.390 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64378]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:26.390 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64378]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:26.390 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:47:26.390 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:26.390 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64364]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:26.390 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64364]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: osdmap e23: 3 total, 3 up, 3 in 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/883622837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]: dispatch 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]: dispatch 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be335c4c-983a-4c4e-84e8-d53e881c90b9"}]': finished 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T04:47:26.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:26.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64419]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:47:26.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:26.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64419]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:26.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:26 smithi077 sudo[64419]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:27.251 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 sudo[37293]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:47:27.251 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 sudo[37293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:27.251 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:26 smithi078 sudo[37293]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:27.251 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 sudo[37293]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:27.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:27.260+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7438053570 con 0x7f74440fce60 2024-04-26T04:47:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 sudo[37318]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:47:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 sudo[37318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:47:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 sudo[37318]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T04:47:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 sudo[37318]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:47:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/269994235' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:47:27.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/269994235' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:47:27.617 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:47:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:47:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:47:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: mgrmap e19: smithi077.hkvsor(active, since 2m), standbys: smithi078.navvuw 2024-04-26T04:47:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:28.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:28.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:28.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:47:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:47:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: mgrmap e19: smithi077.hkvsor(active, since 2m), standbys: smithi078.navvuw 2024-04-26T04:47:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:29.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:29.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:29.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:29 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:29.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:29 smithi077 ceph-mon[26311]: pgmap v82: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:29 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:29 smithi078 ceph-mon[33746]: pgmap v82: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:31 smithi078 ceph-mon[33746]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:31.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:31 smithi077 ceph-mon[26311]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:33.742 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:33 smithi078 ceph-mon[33746]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:33.743 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:33 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:47:33.743 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:33 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:33.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:33 smithi077 ceph-mon[26311]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:33.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:33 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:47:33.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:33 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:34.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:34 smithi078 ceph-mon[33746]: Deploying daemon osd.3 on smithi078 2024-04-26T04:47:34.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:34 smithi077 ceph-mon[26311]: Deploying daemon osd.3 on smithi078 2024-04-26T04:47:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:35 smithi078 ceph-mon[33746]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:35 smithi077 ceph-mon[26311]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:37.656 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:37 smithi078 ceph-mon[33746]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:37.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:37 smithi077 ceph-mon[26311]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:39.994 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:39 smithi078 ceph-mon[33746]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:39 smithi077 ceph-mon[26311]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:41.168 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-04-26T04:47:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.167+0000 7f7432ffd700 1 -- 172.21.15.78:0/2346037169 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7444113790 con 0x7f742c069210 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 msgr2=0x7f742c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 0x7f742c06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f743400b970 tx=0x7f7434009e30 comp rx=0 tx=0).stop 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 msgr2=0x7f744410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7438002a50 tx=0x7f74380046b0 comp rx=0 tx=0).stop 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 shutdown_connections 2024-04-26T04:47:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f742c069210 0x7f742c06b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440fce60 0x7f744410d910 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 --2- 172.21.15.78:0/2346037169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74440fdfd0 0x7f744410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 >> 172.21.15.78:0/2346037169 conn(0x7f74440f8540 msgr2=0x7f74440f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 shutdown_connections 2024-04-26T04:47:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:41.169+0000 7f7448835700 1 -- 172.21.15.78:0/2346037169 wait complete. 2024-04-26T04:47:42.040 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T04:47:42.043 INFO:tasks.cephadm:Deploying osd.4 on smithi078 with /dev/vg_nvme/lv_3... 2024-04-26T04:47:42.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T04:47:42.062 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:41 smithi078 ceph-mon[33746]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:42.062 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:41 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:42.062 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:41 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:42.062 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:41 smithi078 ceph-mon[33746]: from='osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:47:42.062 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:41 smithi078 ceph-mon[33746]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:47:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:41 smithi077 ceph-mon[26311]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:41 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:41 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:41 smithi077 ceph-mon[26311]: from='osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:47:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:41 smithi077 ceph-mon[26311]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:47:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:47:43.356 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:47:43.356 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T04:47:43.356 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:43.357 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:47:43.357 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:47:43.357 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:43.357 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:43.357 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-mon[33746]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:43.357 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[39314]: 2024-04-26T04:47:43.166+0000 7f28a13ed700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T04:47:43.357 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:47:43 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[39314]: 2024-04-26T04:47:43.172+0000 7f289c1e8700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:47:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:43.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:43 smithi077 ceph-mon[26311]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013625 s, 770 MB/s 2024-04-26T04:47:44.174 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-26T04:47:44.270 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:44 smithi078 ceph-mon[33746]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:47:44.270 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:44 smithi078 ceph-mon[33746]: osdmap e26: 4 total, 3 up, 4 in 2024-04-26T04:47:44.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:44 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:44.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:44 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:44 smithi077 ceph-mon[26311]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:47:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:44 smithi077 ceph-mon[26311]: osdmap e26: 4 total, 3 up, 4 in 2024-04-26T04:47:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:44 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:44.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:44 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:45.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-04-26T04:47:45.393 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] boot 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:45.394 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:45 smithi078 ceph-mon[33746]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: osd.3 [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] boot 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-26T04:47:45.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:45 smithi077 ceph-mon[26311]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T04:47:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:47:47.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:47 smithi077 ceph-mon[26311]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T04:47:47.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:47 smithi077 ceph-mon[26311]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:47:47.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:47 smithi078 ceph-mon[33746]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T04:47:47.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:47 smithi078 ceph-mon[33746]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 -- 172.21.15.78:0/2100998432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c066eb0 msgr2=0x7f173c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 --2- 172.21.15.78:0/2100998432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c066eb0 0x7f173c0ffa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f172c004940 tx=0x7f172c009e30 comp rx=0 tx=0).stop 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 -- 172.21.15.78:0/2100998432 shutdown_connections 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 --2- 172.21.15.78:0/2100998432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c0fff70 0x7f173c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 --2- 172.21.15.78:0/2100998432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c066eb0 0x7f173c0ffa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 -- 172.21.15.78:0/2100998432 >> 172.21.15.78:0/2100998432 conn(0x7f173c0f7440 msgr2=0x7f173c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.968+0000 7f174170a700 1 -- 172.21.15.78:0/2100998432 shutdown_connections 2024-04-26T04:47:47.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 -- 172.21.15.78:0/2100998432 wait complete. 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 Processor -- start 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 -- start start 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 0x7f173c10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10e4a0 con 0x7f173c066eb0 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f174170a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10e610 con 0x7f173c0fff70 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.969+0000 7f173a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 0x7f173c10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33844/0 (socket says 172.21.15.78:33844) 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 -- 172.21.15.78:0/3407125376 learned_addr learned my addr 172.21.15.78:0/3407125376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 -- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 msgr2=0x7f173c10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 0x7f173c10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 -- 172.21.15.78:0/3407125376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c005040 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173affd700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 0x7f173c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f173a7fc700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1730002a50 tx=0x7f17300046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1730007cc0 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c113070 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173000f040 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173000da30 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.970+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c113630 con 0x7f173c0fff70 2024-04-26T04:47:47.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.971+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c04c710 con 0x7f173c0fff70 2024-04-26T04:47:47.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.972+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f173000db90 con 0x7f173c0fff70 2024-04-26T04:47:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.972+0000 7f17237fe700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.972+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f17300867d0 con 0x7f173c0fff70 2024-04-26T04:47:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.972+0000 7f173affd700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.973+0000 7f173affd700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 0x7f172406b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f172c004890 tx=0x7f172c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:47.977+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1730056c30 con 0x7f173c0fff70 2024-04-26T04:47:48.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:47:48.170+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f173c1137c0 con 0x7f17240692a0 2024-04-26T04:47:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T04:47:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi078 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi078 to 15936M 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:48 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi078 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi078 to 15936M 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:48.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:48.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:48.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:48.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:48 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: osdmap e31: 4 total, 4 up, 4 in 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:49 smithi077 ceph-mon[26311]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: osdmap e31: 4 total, 4 up, 4 in 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:49 smithi078 ceph-mon[33746]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:47:50.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:50 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/3956642761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]: dispatch 2024-04-26T04:47:50.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:50 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]: dispatch 2024-04-26T04:47:50.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:50 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]': finished 2024-04-26T04:47:50.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:50 smithi078 ceph-mon[33746]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T04:47:50.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:50 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:47:50.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:50 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/3956642761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]: dispatch 2024-04-26T04:47:50.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:50 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]: dispatch 2024-04-26T04:47:50.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:50 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5de5516b-b1f0-405b-984d-1d9ce0900adc"}]': finished 2024-04-26T04:47:50.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:50 smithi077 ceph-mon[26311]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T04:47:50.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:50 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:47:51.517 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:51 smithi077 ceph-mon[26311]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-26T04:47:51.517 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:51 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/717118076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:51.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:51 smithi078 ceph-mon[33746]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-26T04:47:51.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:51 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/717118076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:47:53.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:53 smithi078 ceph-mon[33746]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T04:47:53.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:53 smithi077 ceph-mon[26311]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T04:47:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:55.660 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:55 smithi078 ceph-mon[33746]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T04:47:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:55 smithi077 ceph-mon[26311]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T04:47:57.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:57 smithi078 ceph-mon[33746]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T04:47:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:57 smithi077 ceph-mon[26311]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T04:47:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:58 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:47:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:58 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:58.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:58 smithi078 ceph-mon[33746]: Deploying daemon osd.4 on smithi078 2024-04-26T04:47:58.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:58 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:47:58.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:58 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:58.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:58 smithi077 ceph-mon[26311]: Deploying daemon osd.4 on smithi078 2024-04-26T04:47:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:47:59 smithi078 ceph-mon[33746]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T04:47:59.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:47:59 smithi077 ceph-mon[26311]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T04:48:01.649 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:01 smithi078 ceph-mon[33746]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:01.950 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:01 smithi077 ceph-mon[26311]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:03.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:03 smithi077 ceph-mon[26311]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:03.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:03 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:03.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:03 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:03.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:03 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:03 smithi078 ceph-mon[33746]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:03 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:03 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:03 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:05 smithi078 ceph-mon[33746]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:05.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:05 smithi077 ceph-mon[26311]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.383+0000 7f17237fe700 1 -- 172.21.15.78:0/3407125376 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f173c1137c0 con 0x7f17240692a0 2024-04-26T04:48:06.385 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 4 on host 'smithi078' 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 msgr2=0x7f172406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 0x7f172406b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f172c004890 tx=0x7f172c009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 msgr2=0x7f173c10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1730002a50 tx=0x7f17300046b0 comp rx=0 tx=0).stop 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 shutdown_connections 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f173c066eb0 0x7f173c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 --2- 172.21.15.78:0/3407125376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f173c0fff70 0x7f173c10dea0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 >> 172.21.15.78:0/3407125376 conn(0x7f173c0f7440 msgr2=0x7f173c0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 shutdown_connections 2024-04-26T04:48:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:06.386+0000 7f174170a700 1 -- 172.21.15.78:0/3407125376 wait complete. 2024-04-26T04:48:06.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:06.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:48:06.978 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:06 smithi078 ceph-mon[33746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:06.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:06.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:48:06.995 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:06 smithi077 ceph-mon[26311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:48:07.001 DEBUG:teuthology.orchestra.run.smithi078:osd.4> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T04:48:07.003 INFO:tasks.cephadm:Deploying osd.5 on smithi078 with /dev/vg_nvme/lv_2... 2024-04-26T04:48:07.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: from='osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:07.908 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:07 smithi078 ceph-mon[33746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:07.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:48:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T04:48:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: from='osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:07 smithi077 ceph-mon[26311]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:08.179 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:48:08.589 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:48:08 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[45748]: 2024-04-26T04:48:08.386+0000 7f9962536700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T04:48:08.589 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:48:08 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[45748]: 2024-04-26T04:48:08.394+0000 7f995cb30700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:48:09.662 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:48:09.662 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: osdmap e34: 5 total, 4 up, 5 in 2024-04-26T04:48:09.663 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:09.663 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:09.663 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:09.663 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:09.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:48:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: osdmap e34: 5 total, 4 up, 5 in 2024-04-26T04:48:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-26T04:48:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:10.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:10 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:48:10.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:10 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:48:10.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:10 smithi078 ceph-mon[33746]: osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] boot 2024-04-26T04:48:10.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:10 smithi078 ceph-mon[33746]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T04:48:10.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:10 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125827 s, 833 MB/s 2024-04-26T04:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-26T04:48:10.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:10 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:48:10.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:10 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:48:10.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:10 smithi077 ceph-mon[26311]: osd.4 [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] boot 2024-04-26T04:48:10.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:10 smithi077 ceph-mon[26311]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T04:48:10.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:10 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:48:11.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-04-26T04:48:11.629 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: osdmap e36: 5 total, 5 up, 5 in 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi078 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi078 to 7968M 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:11.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:11.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:11 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: osdmap e36: 5 total, 5 up, 5 in 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi078 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi078 to 7968M 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:11.751 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:11.752 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:11.752 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:11 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:12.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 -- 172.21.15.78:0/3707808248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 msgr2=0x7f5b1c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:12.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 --2- 172.21.15.78:0/3707808248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c0fd700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c004940 tx=0x7f5b0c009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 -- 172.21.15.78:0/3707808248 shutdown_connections 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 --2- 172.21.15.78:0/3707808248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 --2- 172.21.15.78:0/3707808248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c0fd700 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 -- 172.21.15.78:0/3707808248 >> 172.21.15.78:0/3707808248 conn(0x7f5b1c0f6bd0 msgr2=0x7f5b1c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 -- 172.21.15.78:0/3707808248 shutdown_connections 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.980+0000 7f5b21c80700 1 -- 172.21.15.78:0/3707808248 wait complete. 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 Processor -- start 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 -- start start 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:12.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c10e490 con 0x7f5b1c0fdc40 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c10e600 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48474/0 (socket says 172.21.15.78:48474) 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 -- 172.21.15.78:0/668701590 learned_addr learned my addr 172.21.15.78:0/668701590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b13fff700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 -- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 msgr2=0x7f5b1c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 -- 172.21.15.78:0/668701590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c005040 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b13fff700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b1b7fe700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b3b0 tx=0x7f5b0c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c00e040 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b0c00bcc0 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c01c630 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.981+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c113000 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.982+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c1135c0 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.982+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c101bf0 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.983+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5b0c013980 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.983+0000 7f5b18ff9700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 0x7f5b0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.983+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f5b0c087ae0 con 0x7f5b1c0fd2f0 2024-04-26T04:48:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.984+0000 7f5b13fff700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 0x7f5b0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.984+0000 7f5b13fff700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 0x7f5b0806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007fd0 tx=0x7f5b04009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:12.988+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5b0c057d10 con 0x7f5b1c0fd2f0 2024-04-26T04:48:13.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:13.180+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1c05e3a0 con 0x7f5b080692a0 2024-04-26T04:48:13.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:13 smithi078 ceph-mon[33746]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:13.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:48:13.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:48:13.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:13 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:13.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:13 smithi077 ceph-mon[26311]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:13.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:48:13.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:48:13.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:13 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:14.785 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:14 smithi078 ceph-mon[33746]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:14 smithi077 ceph-mon[26311]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/1198759652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]: dispatch 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]: dispatch 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]': finished 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T04:48:15.821 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1198759652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]: dispatch 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]: dispatch 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf968b1d-2324-4b14-9723-24ebb6b6c406"}]': finished 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T04:48:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:16.921 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:16 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/990561428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:48:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:16 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/990561428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:48:17.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:17 smithi077 ceph-mon[26311]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:17 smithi078 ceph-mon[33746]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:19.673 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:19 smithi078 ceph-mon[33746]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:19.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:19 smithi077 ceph-mon[26311]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:21.689 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:21 smithi078 ceph-mon[33746]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:21.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:21 smithi077 ceph-mon[26311]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:23.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:23 smithi078 ceph-mon[33746]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:23.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:48:23.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:23.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:23 smithi078 ceph-mon[33746]: Deploying daemon osd.5 on smithi078 2024-04-26T04:48:23.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:23.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:23 smithi077 ceph-mon[26311]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:23.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:48:23.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:23.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:23 smithi077 ceph-mon[26311]: Deploying daemon osd.5 on smithi078 2024-04-26T04:48:23.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:25.913 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:25 smithi078 ceph-mon[33746]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:25 smithi077 ceph-mon[26311]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:27.596 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:27 smithi078 ceph-mon[33746]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:27 smithi077 ceph-mon[26311]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:28.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:28.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:28.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:28 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:28 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:29.979 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:29 smithi078 ceph-mon[33746]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:29.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:29 smithi077 ceph-mon[26311]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:31.247 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 5 on host 'smithi078' 2024-04-26T04:48:31.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.245+0000 7f5b18ff9700 1 -- 172.21.15.78:0/668701590 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b1c05e3a0 con 0x7f5b080692a0 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 msgr2=0x7f5b0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 0x7f5b0806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007fd0 tx=0x7f5b04009040 comp rx=0 tx=0).stop 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 msgr2=0x7f5b1c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b3b0 tx=0x7f5b0c009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 shutdown_connections 2024-04-26T04:48:31.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5b080692a0 0x7f5b0806b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0fd2f0 0x7f5b1c10d950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 --2- 172.21.15.78:0/668701590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b1c0fdc40 0x7f5b1c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 >> 172.21.15.78:0/668701590 conn(0x7f5b1c0f6bd0 msgr2=0x7f5b1c0f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 shutdown_connections 2024-04-26T04:48:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:31.248+0000 7f5b21c80700 1 -- 172.21.15.78:0/668701590 wait complete. 2024-04-26T04:48:31.890 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:31.890 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.890 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.890 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.891 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.891 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:48:31.891 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:31 smithi078 ceph-mon[33746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:48:31.914 DEBUG:teuthology.orchestra.run.smithi078:osd.5> sudo journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T04:48:31.917 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-26T04:48:31.917 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T04:48:31.943 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:48:31.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:31 smithi077 ceph-mon[26311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:48:32.184 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: from='osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:33.588 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:33 smithi077 ceph-mon[26311]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:33.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:33.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:48:33.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T04:48:33.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:33.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: from='osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:33.633 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-mon[33746]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:48:33.633 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[52660]: 2024-04-26T04:48:33.477+0000 7f65c0de7700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T04:48:33.633 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:48:33 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[52660]: 2024-04-26T04:48:33.487+0000 7f65bdbe6700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.769+0000 7f86c8424700 1 -- 172.21.15.77:0/3348972101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fbcc0 msgr2=0x7f86c00fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.769+0000 7f86c8424700 1 --2- 172.21.15.77:0/3348972101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fbcc0 0x7f86c00fc0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86bc004940 tx=0x7f86bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- 172.21.15.77:0/3348972101 shutdown_connections 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 --2- 172.21.15.77:0/3348972101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fcf70 0x7f86c00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 --2- 172.21.15.77:0/3348972101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fbcc0 0x7f86c00fc0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- 172.21.15.77:0/3348972101 >> 172.21.15.77:0/3348972101 conn(0x7f86c00f7420 msgr2=0x7f86c00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:33.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- 172.21.15.77:0/3348972101 shutdown_connections 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- 172.21.15.77:0/3348972101 wait complete. 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 Processor -- start 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- start start 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fbcc0 0x7f86c0110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0066c20 con 0x7f86c00fcf70 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c8424700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0066d60 con 0x7f86c00fbcc0 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c59bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c59bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46194/0 (socket says 172.21.15.77:46194) 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c59bf700 1 -- 172.21.15.77:0/710856805 learned_addr learned my addr 172.21.15.77:0/710856805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.770+0000 7f86c61c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fbcc0 0x7f86c0110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c59bf700 1 -- 172.21.15.77:0/710856805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fbcc0 msgr2=0x7f86c0110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c59bf700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fbcc0 0x7f86c0110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c59bf700 1 -- 172.21.15.77:0/710856805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc005040 con 0x7f86c00fcf70 2024-04-26T04:48:33.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c59bf700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b790 tx=0x7f86b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:33.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b000f6f0 con 0x7f86c00fcf70 2024-04-26T04:48:33.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86b000fd20 con 0x7f86c00fcf70 2024-04-26T04:48:33.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b00103a0 con 0x7f86c00fcf70 2024-04-26T04:48:33.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c01132a0 con 0x7f86c00fcf70 2024-04-26T04:48:33.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.771+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0113860 con 0x7f86c00fcf70 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.772+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c005f5c0 con 0x7f86c00fcf70 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.772+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f86b0011400 con 0x7f86c00fcf70 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.773+0000 7f86b6ffd700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 0x7f86ac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.773+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f86b0086b20 con 0x7f86c00fcf70 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.773+0000 7f86c61c0700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 0x7f86ac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:33.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.773+0000 7f86c61c0700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 0x7f86ac06b760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:33.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.777+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86b00532a0 con 0x7f86c00fcf70 2024-04-26T04:48:33.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.967+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f86c004bed0 con 0x7f86c00fcf70 2024-04-26T04:48:33.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.968+0000 7f86b6ffd700 1 -- 172.21.15.77:0/710856805 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f86b00568c0 con 0x7f86c00fcf70 2024-04-26T04:48:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 msgr2=0x7f86ac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 0x7f86ac06b760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 msgr2=0x7f86c0112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b790 tx=0x7f86b0009250 comp rx=0 tx=0).stop 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 shutdown_connections 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f86ac0692b0 0x7f86ac06b760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c00fbcc0 0x7f86c0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 --2- 172.21.15.77:0/710856805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c00fcf70 0x7f86c0112ca0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 >> 172.21.15.77:0/710856805 conn(0x7f86c00f7420 msgr2=0x7f86c00f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 shutdown_connections 2024-04-26T04:48:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:33.970+0000 7f86c8424700 1 -- 172.21.15.77:0/710856805 wait complete. 2024-04-26T04:48:34.593 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1714106889,"num_in_osds":6,"osd_in_since":1714106895,"num_remapped_pgs":0} 2024-04-26T04:48:34.607 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:34 smithi077 ceph-mon[26311]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:48:34.607 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:34 smithi077 ceph-mon[26311]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T04:48:34.607 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:34 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:34.607 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:34 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:34.607 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/710856805' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:48:34.686 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:34 smithi078 ceph-mon[33746]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-26T04:48:34.686 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:34 smithi078 ceph-mon[33746]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T04:48:34.686 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:34 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:34.686 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:34 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:34.686 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:34 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/710856805' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:48:35.594 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: purged_snaps scrub starts 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: purged_snaps scrub ok 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] boot 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:35.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:35.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: purged_snaps scrub starts 2024-04-26T04:48:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: purged_snaps scrub ok 2024-04-26T04:48:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-26T04:48:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: osd.5 [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] boot 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:35.860 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: Detected new or changed devices on smithi078 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi078 to 5312M 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:36.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: Detected new or changed devices on smithi078 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi078 to 5312M 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:48:37.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 -- 172.21.15.77:0/3434096365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 msgr2=0x7f33240fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:37.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 --2- 172.21.15.77:0/3434096365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f33240fe820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3314004940 tx=0x7f3314009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:37.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 -- 172.21.15.77:0/3434096365 shutdown_connections 2024-04-26T04:48:37.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 --2- 172.21.15.77:0/3434096365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 0x7f33240faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 --2- 172.21.15.77:0/3434096365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f33240fe820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 -- 172.21.15.77:0/3434096365 >> 172.21.15.77:0/3434096365 conn(0x7f33240f6bd0 msgr2=0x7f33240f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:37.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 -- 172.21.15.77:0/3434096365 shutdown_connections 2024-04-26T04:48:37.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.450+0000 7f332938d700 1 -- 172.21.15.77:0/3434096365 wait complete. 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 Processor -- start 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 -- start start 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 0x7f332410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332410e450 con 0x7f33240fe410 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f332938d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332410e5c0 con 0x7f33240fed60 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f3322ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f3322ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46216/0 (socket says 172.21.15.77:46216) 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f3322ffd700 1 -- 172.21.15.77:0/3894261414 learned_addr learned my addr 172.21.15.77:0/3894261414 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.451+0000 7f33227fc700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 0x7f332410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:37.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f3322ffd700 1 -- 172.21.15.77:0/3894261414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 msgr2=0x7f332410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f3322ffd700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 0x7f332410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f3322ffd700 1 -- 172.21.15.77:0/3894261414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314005040 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f3322ffd700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f331400b3b0 tx=0x7f3314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f331400e040 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f331400bbf0 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3324112fc0 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f331401c630 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.452+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33241134f0 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.453+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332404c710 con 0x7f33240fe410 2024-04-26T04:48:37.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.454+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f331400bd60 con 0x7f33240fe410 2024-04-26T04:48:37.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.454+0000 7f330b7fe700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 0x7f330c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:37.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.454+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3314087a20 con 0x7f33240fe410 2024-04-26T04:48:37.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.454+0000 7f33227fc700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 0x7f330c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:37.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.454+0000 7f33227fc700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 0x7f330c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3318004f40 tx=0x7f331800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:37.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.458+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3314057bc0 con 0x7f33240fe410 2024-04-26T04:48:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.645+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f33241058c0 con 0x7f33240fe410 2024-04-26T04:48:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.645+0000 7f330b7fe700 1 -- 172.21.15.77:0/3894261414 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f3314057750 con 0x7f33240fe410 2024-04-26T04:48:37.646 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 msgr2=0x7f330c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 0x7f330c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3318004f40 tx=0x7f331800b040 comp rx=0 tx=0).stop 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 msgr2=0x7f332410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f331400b3b0 tx=0x7f3314009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 shutdown_connections 2024-04-26T04:48:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f330c0692a0 0x7f330c06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33240fe410 0x7f332410d910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 --2- 172.21.15.77:0/3894261414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33240fed60 0x7f332410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 >> 172.21.15.77:0/3894261414 conn(0x7f33240f6bd0 msgr2=0x7f33240f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 shutdown_connections 2024-04-26T04:48:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:37.647+0000 7f332938d700 1 -- 172.21.15.77:0/3894261414 wait complete. 2024-04-26T04:48:37.981 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:37 smithi077 ceph-mon[26311]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:37 smithi078 ceph-mon[33746]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:38.289 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1714106914,"num_in_osds":6,"osd_in_since":1714106895,"num_remapped_pgs":0} 2024-04-26T04:48:38.290 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:48:38.539 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:38.894 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:38 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3894261414' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:48:38.894 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:39.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:38 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3894261414' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:48:39.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:39.888 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:39 smithi077 ceph-mon[26311]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:39 smithi078 ceph-mon[33746]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- 172.21.15.77:0/4018981842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 msgr2=0x7f08c0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 --2- 172.21.15.77:0/4018981842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c0102160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004940 tx=0x7f08b0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- 172.21.15.77:0/4018981842 shutdown_connections 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 --2- 172.21.15.77:0/4018981842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 0x7f08c00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 --2- 172.21.15.77:0/4018981842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c0102160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- 172.21.15.77:0/4018981842 >> 172.21.15.77:0/4018981842 conn(0x7f08c0071d00 msgr2=0x7f08c0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:40.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- 172.21.15.77:0/4018981842 shutdown_connections 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- 172.21.15.77:0/4018981842 wait complete. 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 Processor -- start 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.122+0000 7f08c853d700 1 -- start start 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 0x7f08c0113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0114740 con 0x7f08c0101d90 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c010db30 con 0x7f08c00fbd40 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34954/0 (socket says 172.21.15.77:34954) 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 -- 172.21.15.77:0/2858646196 learned_addr learned my addr 172.21.15.77:0/2858646196 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c62d9700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 0x7f08c0113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 -- 172.21.15.77:0/2858646196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 msgr2=0x7f08c0113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 0x7f08c0113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 -- 172.21.15.77:0/2858646196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0005040 con 0x7f08c0101d90 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c5ad8700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00c830 tx=0x7f08bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08bc0106f0 con 0x7f08c0101d90 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08bc010d20 con 0x7f08c0101d90 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c010de10 con 0x7f08c0101d90 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08bc0113a0 con 0x7f08c0101d90 2024-04-26T04:48:40.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.123+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c010e3d0 con 0x7f08c0101d90 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.124+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c00fd430 con 0x7f08c0101d90 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.125+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f08bc012400 con 0x7f08c0101d90 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.125+0000 7f08b6ffd700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 0x7f08ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.125+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f08bc086d30 con 0x7f08c0101d90 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.125+0000 7f08c62d9700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 0x7f08ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.126+0000 7f08c62d9700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 0x7f08ac06b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004560 tx=0x7f08b0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.130+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f08bc056ed0 con 0x7f08c0101d90 2024-04-26T04:48:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.318+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08c0108df0 con 0x7f08c0101d90 2024-04-26T04:48:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.319+0000 7f08b6ffd700 1 -- 172.21.15.77:0/2858646196 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11009 (secure 0 0 0) 0x7f08bc056a60 con 0x7f08c0101d90 2024-04-26T04:48:40.320 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:40.321 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T04:48:35.480862+0000","last_up_change":"2024-04-26T04:48:34.475903+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6803","nonce":1220397746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6805","nonce":1220397746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6809","nonce":1220397746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6807","nonce":1220397746}]},"public_addr":"172.21.15.77:6803/1220397746","cluster_addr":"172.21.15.77:6805/1220397746","heartbeat_back_addr":"172.21.15.77:6809/1220397746","heartbeat_front_addr":"172.21.15.77:6807/1220397746","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6811","nonce":50899416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6813","nonce":50899416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6817","nonce":50899416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6815","nonce":50899416}]},"public_addr":"172.21.15.77:6811/50899416","cluster_addr":"172.21.15.77:6813/50899416","heartbeat_back_addr":"172.21.15.77:6817/50899416","heartbeat_front_addr":"172.21.15.77:6815/50899416","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6819","nonce":2248042345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6821","nonce":2248042345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6825","nonce":2248042345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6823","nonce":2248042345}]},"public_addr":"172.21.15.77:6819/2248042345","cluster_addr":"172.21.15.77:6821/2248042345","heartbeat_back_addr":"172.21.15.77:6825/2248042345","heartbeat_front_addr":"172.21.15.77:6823/2248042345","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6801","nonce":632729185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6803","nonce":632729185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6807","nonce":632729185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6805","nonce":632729185}]},"public_addr":"172.21.15.78:6801/632729185","cluster_addr":"172.21.15.78:6803/632729185","heartbeat_back_addr":"172.21.15.78:6807/632729185","heartbeat_front_addr":"172.21.15.78:6805/632729185","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6809","nonce":489554457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6811","nonce":489554457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6815","nonce":489554457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6813","nonce":489554457}]},"public_addr":"172.21.15.78:6809/489554457","cluster_addr":"172.21.15.78:6811/489554457","heartbeat_back_addr":"172.21.15.78:6815/489554457","heartbeat_front_addr":"172.21.15.78:6813/489554457","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6817","nonce":2307894772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6819","nonce":2307894772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6823","nonce":2307894772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6821","nonce":2307894772}]},"public_addr":"172.21.15.78:6817/2307894772","cluster_addr":"172.21.15.78:6819/2307894772","heartbeat_back_addr":"172.21.15.78:6823/2307894772","heartbeat_front_addr":"172.21.15.78:6821/2307894772","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:48:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 msgr2=0x7f08ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 0x7f08ac06b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08b0004560 tx=0x7f08b0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 msgr2=0x7f08c01140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00c830 tx=0x7f08bc004470 comp rx=0 tx=0).stop 2024-04-26T04:48:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 shutdown_connections 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f08ac0692a0 0x7f08ac06b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c00fbd40 0x7f08c0113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 --2- 172.21.15.77:0/2858646196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08c0101d90 0x7f08c01140f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 >> 172.21.15.77:0/2858646196 conn(0x7f08c0071d00 msgr2=0x7f08c00655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 shutdown_connections 2024-04-26T04:48:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:40.321+0000 7f08c853d700 1 -- 172.21.15.77:0/2858646196 wait complete. 2024-04-26T04:48:40.963 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:40 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2858646196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:40.987 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:47:22.739195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-26T04:48:40.987 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T04:48:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:40 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2858646196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:41.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:41 smithi077 ceph-mon[26311]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:41 smithi078 ceph-mon[33746]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 -- 172.21.15.77:0/3699432010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc101e20 msgr2=0x7f87fc1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 --2- 172.21.15.77:0/3699432010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc101e20 0x7f87fc1021f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87f0004940 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 -- 172.21.15.77:0/3699432010 shutdown_connections 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 --2- 172.21.15.77:0/3699432010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87fc0fc710 0x7f87fc0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 --2- 172.21.15.77:0/3699432010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc101e20 0x7f87fc1021f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 -- 172.21.15.77:0/3699432010 >> 172.21.15.77:0/3699432010 conn(0x7f87fc071460 msgr2=0x7f87fc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 -- 172.21.15.77:0/3699432010 shutdown_connections 2024-04-26T04:48:42.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.897+0000 7f8802c3f700 1 -- 172.21.15.77:0/3699432010 wait complete. 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 Processor -- start 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 -- start start 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87fc101e20 0x7f87fc10f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc10f690 con 0x7f87fc0fc710 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f8802c3f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc10f800 con 0x7f87fc101e20 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34972/0 (socket says 172.21.15.77:34972) 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 -- 172.21.15.77:0/1642791127 learned_addr learned my addr 172.21.15.77:0/1642791127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 -- 172.21.15.77:0/1642791127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87fc101e20 msgr2=0x7f87fc10f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87fc101e20 0x7f87fc10f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.898+0000 7f88009db700 1 -- 172.21.15.77:0/1642791127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0005040 con 0x7f87fc0fc710 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f88009db700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b3b0 tx=0x7f87f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:42.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f000e040 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc117b70 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f000bbc0 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f001c3f0 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.899+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc118050 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.900+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc104260 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.900+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f87f000bd30 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.901+0000 7f87f97fa700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 0x7f87ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.901+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f87f00880a0 con 0x7f87fc0fc710 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.901+0000 7f87fbfff700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 0x7f87ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.901+0000 7f87fbfff700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 0x7f87ec06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0fbfb0 tx=0x7f87e8007d40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:42.905+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f87f0054690 con 0x7f87fc0fc710 2024-04-26T04:48:43.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.092+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87fc0652f0 con 0x7f87fc0fc710 2024-04-26T04:48:43.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.093+0000 7f87f97fa700 1 -- 172.21.15.77:0/1642791127 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87fc0652f0 con 0x7f87fc0fc710 2024-04-26T04:48:43.094 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 msgr2=0x7f87ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 0x7f87ec06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0fbfb0 tx=0x7f87e8007d40 comp rx=0 tx=0).stop 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 msgr2=0x7f87fc10eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b3b0 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 shutdown_connections 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f87ec0692a0 0x7f87ec06b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87fc0fc710 0x7f87fc10eb00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:43.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 --2- 172.21.15.77:0/1642791127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87fc101e20 0x7f87fc10f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:43.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 >> 172.21.15.77:0/1642791127 conn(0x7f87fc071460 msgr2=0x7f87fc0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:43.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 shutdown_connections 2024-04-26T04:48:43.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:43.095+0000 7f8802c3f700 1 -- 172.21.15.77:0/1642791127 wait complete. 2024-04-26T04:48:43.811 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T04:48:43.812 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T04:48:43.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:43 smithi077 ceph-mon[26311]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:43.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:43 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1642791127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:48:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:43 smithi078 ceph-mon[33746]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:43 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1642791127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:48:44.062 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 -- 172.21.15.77:0/3712015929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 msgr2=0x7f2edc0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3712015929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc0fd3e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0004940 tx=0x7f2ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 -- 172.21.15.77:0/3712015929 shutdown_connections 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3712015929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc0fd3e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3712015929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 0x7f2edc0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 -- 172.21.15.77:0/3712015929 >> 172.21.15.77:0/3712015929 conn(0x7f2edc0f7440 msgr2=0x7f2edc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 -- 172.21.15.77:0/3712015929 shutdown_connections 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.600+0000 7f2ee1631700 1 -- 172.21.15.77:0/3712015929 wait complete. 2024-04-26T04:48:45.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 Processor -- start 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 -- start start 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 0x7f2edc10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc10e490 con 0x7f2edc0fcf70 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2ee1631700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc10e600 con 0x7f2edc0fbcc0 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.601+0000 7f2eda7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35004/0 (socket says 172.21.15.77:35004) 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 -- 172.21.15.77:0/3727824013 learned_addr learned my addr 172.21.15.77:0/3727824013 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2edaffd700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 0x7f2edc10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 -- 172.21.15.77:0/3727824013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 msgr2=0x7f2edc10d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 0x7f2edc10d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 -- 172.21.15.77:0/3727824013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0005040 con 0x7f2edc0fcf70 2024-04-26T04:48:45.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2eda7fc700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b3b0 tx=0x7f2ed0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:45.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed000e040 con 0x7f2edc0fcf70 2024-04-26T04:48:45.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ed000bbf0 con 0x7f2edc0fcf70 2024-04-26T04:48:45.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc112fb0 con 0x7f2edc0fcf70 2024-04-26T04:48:45.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed001c630 con 0x7f2edc0fcf70 2024-04-26T04:48:45.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.602+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc113540 con 0x7f2edc0fcf70 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.603+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc101b00 con 0x7f2edc0fcf70 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.603+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2ed0026440 con 0x7f2edc0fcf70 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.604+0000 7f2ec37fe700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 0x7f2ec406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.604+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2ed0087a40 con 0x7f2edc0fcf70 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.604+0000 7f2edaffd700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 0x7f2ec406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:45.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.604+0000 7f2edaffd700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 0x7f2ec406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0041d0 tx=0x7f2ecc0043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:45.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.608+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ed0057be0 con 0x7f2edc0fcf70 2024-04-26T04:48:45.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.870+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2edc04c710 con 0x7f2edc0fcf70 2024-04-26T04:48:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.874+0000 7f2ec37fe700 1 -- 172.21.15.77:0/3727824013 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2ed0057770 con 0x7f2edc0fcf70 2024-04-26T04:48:45.875 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2024-04-26T04:48:45.875 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQAtMitmfEb0MxAAWNO7yvutufJlKZ9t2R/nhw== 2024-04-26T04:48:45.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 msgr2=0x7f2ec406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:45.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 0x7f2ec406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0041d0 tx=0x7f2ecc0043c0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 msgr2=0x7f2edc10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:45.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b3b0 tx=0x7f2ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:45.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 shutdown_connections 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2ec40692a0 0x7f2ec406b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2edc0fbcc0 0x7f2edc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 --2- 172.21.15.77:0/3727824013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2edc0fcf70 0x7f2edc10de90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.877+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 >> 172.21.15.77:0/3727824013 conn(0x7f2edc0f7440 msgr2=0x7f2edc066730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.878+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 shutdown_connections 2024-04-26T04:48:45.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:45.878+0000 7f2ee1631700 1 -- 172.21.15.77:0/3727824013 wait complete. 2024-04-26T04:48:45.923 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:45 smithi077 ceph-mon[26311]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:46.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:45 smithi078 ceph-mon[33746]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:46.606 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:48:46.606 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T04:48:46.606 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T04:48:46.641 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T04:48:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi078/config 2024-04-26T04:48:46.933 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:46 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3727824013' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:46.933 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:46 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3727824013' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:48:46.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:46 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3727824013' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:46.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:46 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3727824013' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:48:48.155 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:47 smithi078 ceph-mon[33746]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:48.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:47 smithi077 ceph-mon[26311]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:48.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- 172.21.15.78:0/118162338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 msgr2=0x7f007c0fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:48.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 --2- 172.21.15.78:0/118162338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c0fe4c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0064004940 tx=0x7f0064009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- 172.21.15.78:0/118162338 shutdown_connections 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 --2- 172.21.15.78:0/118162338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c0fe4c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 --2- 172.21.15.78:0/118162338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 0x7f007c0fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- 172.21.15.78:0/118162338 >> 172.21.15.78:0/118162338 conn(0x7f007c0f8580 msgr2=0x7f007c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- 172.21.15.78:0/118162338 shutdown_connections 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- 172.21.15.78:0/118162338 wait complete. 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 Processor -- start 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.368+0000 7f008144b700 1 -- start start 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 0x7f007c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c10e440 con 0x7f007c0fce50 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c10e5b0 con 0x7f007c0fe050 2024-04-26T04:48:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:48.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36406/0 (socket says 172.21.15.78:36406) 2024-04-26T04:48:48.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 -- 172.21.15.78:0/1546624374 learned_addr learned my addr 172.21.15.78:0/1546624374 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-26T04:48:48.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f007affd700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 0x7f007c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:48.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 -- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 msgr2=0x7f007c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:48.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 0x7f007c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 -- 172.21.15.78:0/1546624374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064005040 con 0x7f007c0fe050 2024-04-26T04:48:48.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f00727fc700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f006400b3b0 tx=0x7f0064009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006400e040 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006400bcc0 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c112fb0 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006401c630 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.369+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c113540 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.370+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c04c710 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.371+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0064013980 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.371+0000 7f0073fff700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 0x7f006806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.371+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0064087c50 con 0x7f007c0fe050 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.371+0000 7f007affd700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 0x7f006806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.372+0000 7f007affd700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 0x7f006806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f006c009fd0 tx=0x7f006c009f00 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:48.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.376+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0064057cd0 con 0x7f007c0fe050 2024-04-26T04:48:48.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.634+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f007c1058e0 con 0x7f007c0fe050 2024-04-26T04:48:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.639+0000 7f0073fff700 1 -- 172.21.15.78:0/1546624374 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0064057860 con 0x7f007c0fe050 2024-04-26T04:48:48.640 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2024-04-26T04:48:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAwMitmQxXtJRAAFdTP+Ez7XoEi3k2CTSgwZw== 2024-04-26T04:48:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 msgr2=0x7f006806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 0x7f006806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f006c009fd0 tx=0x7f006c009f00 comp rx=0 tx=0).stop 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 msgr2=0x7f007c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f006400b3b0 tx=0x7f0064009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 shutdown_connections 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f00680692a0 0x7f006806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f007c0fce50 0x7f007c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 --2- 172.21.15.78:0/1546624374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f007c0fe050 0x7f007c10de40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 >> 172.21.15.78:0/1546624374 conn(0x7f007c0f8580 msgr2=0x7f007c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 shutdown_connections 2024-04-26T04:48:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:48:48.641+0000 7f008144b700 1 -- 172.21.15.78:0/1546624374 wait complete. 2024-04-26T04:48:48.959 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:48 smithi078 ceph-mon[33746]: from='client.? 172.21.15.78:0/1546624374' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:48.959 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:48 smithi078 ceph-mon[33746]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:48.959 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:48 smithi078 ceph-mon[33746]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:48:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:48 smithi077 ceph-mon[26311]: from='client.? 172.21.15.78:0/1546624374' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:48 smithi077 ceph-mon[26311]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:48:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:48 smithi077 ceph-mon[26311]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:48:49.260 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:48:49.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-26T04:48:49.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T04:48:49.296 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T04:48:49.297 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T04:48:49.297 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T04:48:49.559 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:49.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:49 smithi077 ceph-mon[26311]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:49 smithi078 ceph-mon[33746]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:51.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/1311932056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 msgr2=0x7f5e440fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/1311932056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e440fe1b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e34004940 tx=0x7f5e34009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/1311932056 shutdown_connections 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/1311932056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 0x7f5e44106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/1311932056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e440fe1b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/1311932056 >> 172.21.15.77:0/1311932056 conn(0x7f5e440f7850 msgr2=0x7f5e440f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/1311932056 shutdown_connections 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.214+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/1311932056 wait complete. 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 Processor -- start 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 -- start start 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 0x7f5e44114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441146e0 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e4a9b9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4410dad0 con 0x7f5e440fe780 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.215+0000 7f5e43fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51362/0 (socket says 172.21.15.77:51362) 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 -- 172.21.15.77:0/2777550776 learned_addr learned my addr 172.21.15.77:0/2777550776 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e437fe700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 0x7f5e44114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 -- 172.21.15.77:0/2777550776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 msgr2=0x7f5e44114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 0x7f5e44114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 -- 172.21.15.77:0/2777550776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34005040 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e43fff700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009380 tx=0x7f5e34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3400e040 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4410dd50 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e34009af0 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3401c3f0 con 0x7f5e440fdde0 2024-04-26T04:48:51.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.216+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4410e290 con 0x7f5e440fdde0 2024-04-26T04:48:51.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.217+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e44066750 con 0x7f5e440fdde0 2024-04-26T04:48:51.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.218+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5e34013420 con 0x7f5e440fdde0 2024-04-26T04:48:51.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.218+0000 7f5e40ff9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 0x7f5e2c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.218+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5e340881d0 con 0x7f5e440fdde0 2024-04-26T04:48:51.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.218+0000 7f5e437fe700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 0x7f5e2c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.219+0000 7f5e437fe700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 0x7f5e2c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e4410ec90 tx=0x7f5e3800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.223+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5e34058300 con 0x7f5e440fdde0 2024-04-26T04:48:51.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.468+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5e44109e70 con 0x7f5e440fdde0 2024-04-26T04:48:51.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.471+0000 7f5e40ff9700 1 -- 172.21.15.77:0/2777550776 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167178 (secure 0 0 0) 0x7f5e34057e90 con 0x7f5e440fdde0 2024-04-26T04:48:51.478 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:51.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 msgr2=0x7f5e2c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 0x7f5e2c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e4410ec90 tx=0x7f5e3800b040 comp rx=0 tx=0).stop 2024-04-26T04:48:51.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 msgr2=0x7f5e44113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009380 tx=0x7f5e34009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:51.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 shutdown_connections 2024-04-26T04:48:51.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f5e2c0692a0 0x7f5e2c06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e440fdde0 0x7f5e44113b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 --2- 172.21.15.77:0/2777550776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e440fe780 0x7f5e44114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 >> 172.21.15.77:0/2777550776 conn(0x7f5e440f7850 msgr2=0x7f5e440f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 shutdown_connections 2024-04-26T04:48:51.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:51.474+0000 7f5e4a9b9700 1 -- 172.21.15.77:0/2777550776 wait complete. 2024-04-26T04:48:52.146 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:51 smithi077 ceph-mon[26311]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:52.147 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:51 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2777550776' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T04:48:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi077.hkvsor","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1184941417},{"type":"v1","addr":"172.21.15.77:6801","nonce":1184941417}]},"active_addr":"172.21.15.77:6801/1184941417","active_change":"2024-04-26T04:45:08.425389+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi078.navvuw","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.77:8443/","prometheus":"http://172.21.15.77:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":192705865}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":4033800957}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3441382659}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3379821906}]}]} 2024-04-26T04:48:52.175 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T04:48:52.175 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T04:48:52.175 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:48:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:51 smithi078 ceph-mon[33746]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:51 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2777550776' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T04:48:52.430 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2774465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 msgr2=0x7f3e480fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2774465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e480fbd20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c004940 tx=0x7f3e3c009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2774465454 shutdown_connections 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2774465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e480fbd20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2774465454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 0x7f3e481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2774465454 >> 172.21.15.77:0/2774465454 conn(0x7f3e480f7860 msgr2=0x7f3e480f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2774465454 shutdown_connections 2024-04-26T04:48:54.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.102+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2774465454 wait complete. 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 Processor -- start 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 -- start start 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 0x7f3e481139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e481145c0 con 0x7f3e481026b0 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e4dd92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4810db50 con 0x7f3e48101d10 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e46ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e46ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51378/0 (socket says 172.21.15.77:51378) 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e46ffd700 1 -- 172.21.15.77:0/2098092812 learned_addr learned my addr 172.21.15.77:0/2098092812 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e477fe700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 0x7f3e481139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e46ffd700 1 -- 172.21.15.77:0/2098092812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 msgr2=0x7f3e481139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.103+0000 7f3e46ffd700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 0x7f3e481139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e46ffd700 1 -- 172.21.15.77:0/2098092812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c005040 con 0x7f3e481026b0 2024-04-26T04:48:54.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e46ffd700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009180 tx=0x7f3e3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:54.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c00e040 con 0x7f3e481026b0 2024-04-26T04:48:54.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e3c013420 con 0x7f3e481026b0 2024-04-26T04:48:54.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4810ddd0 con 0x7f3e481026b0 2024-04-26T04:48:54.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c01c610 con 0x7f3e481026b0 2024-04-26T04:48:54.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.104+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4810e390 con 0x7f3e481026b0 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.105+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e480fd460 con 0x7f3e481026b0 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.105+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3e3c013a00 con 0x7f3e481026b0 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.106+0000 7f3e2ffff700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.106+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3e3c087ee0 con 0x7f3e481026b0 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.106+0000 7f3e477fe700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:54.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.106+0000 7f3e477fe700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 0x7f3e3006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009bf0 tx=0x7f3e380091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:54.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.110+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3e3c0545f0 con 0x7f3e481026b0 2024-04-26T04:48:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:53 smithi077 ceph-mon[26311]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:53 smithi078 ceph-mon[33746]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:54.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.297+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e4804c710 con 0x7f3e481026b0 2024-04-26T04:48:54.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.298+0000 7f3e2ffff700 1 -- 172.21.15.77:0/2098092812 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11009 (secure 0 0 0) 0x7f3e3c057c10 con 0x7f3e481026b0 2024-04-26T04:48:54.300 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:54.300 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T04:48:35.480862+0000","last_up_change":"2024-04-26T04:48:34.475903+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6803","nonce":1220397746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6805","nonce":1220397746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6809","nonce":1220397746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6807","nonce":1220397746}]},"public_addr":"172.21.15.77:6803/1220397746","cluster_addr":"172.21.15.77:6805/1220397746","heartbeat_back_addr":"172.21.15.77:6809/1220397746","heartbeat_front_addr":"172.21.15.77:6807/1220397746","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6811","nonce":50899416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6813","nonce":50899416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6817","nonce":50899416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6815","nonce":50899416}]},"public_addr":"172.21.15.77:6811/50899416","cluster_addr":"172.21.15.77:6813/50899416","heartbeat_back_addr":"172.21.15.77:6817/50899416","heartbeat_front_addr":"172.21.15.77:6815/50899416","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6819","nonce":2248042345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6821","nonce":2248042345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6825","nonce":2248042345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6823","nonce":2248042345}]},"public_addr":"172.21.15.77:6819/2248042345","cluster_addr":"172.21.15.77:6821/2248042345","heartbeat_back_addr":"172.21.15.77:6825/2248042345","heartbeat_front_addr":"172.21.15.77:6823/2248042345","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6801","nonce":632729185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6803","nonce":632729185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6807","nonce":632729185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6805","nonce":632729185}]},"public_addr":"172.21.15.78:6801/632729185","cluster_addr":"172.21.15.78:6803/632729185","heartbeat_back_addr":"172.21.15.78:6807/632729185","heartbeat_front_addr":"172.21.15.78:6805/632729185","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6809","nonce":489554457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6811","nonce":489554457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6815","nonce":489554457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6813","nonce":489554457}]},"public_addr":"172.21.15.78:6809/489554457","cluster_addr":"172.21.15.78:6811/489554457","heartbeat_back_addr":"172.21.15.78:6815/489554457","heartbeat_front_addr":"172.21.15.78:6813/489554457","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6817","nonce":2307894772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6819","nonce":2307894772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6823","nonce":2307894772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6821","nonce":2307894772}]},"public_addr":"172.21.15.78:6817/2307894772","cluster_addr":"172.21.15.78:6819/2307894772","heartbeat_back_addr":"172.21.15.78:6823/2307894772","heartbeat_front_addr":"172.21.15.78:6821/2307894772","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 msgr2=0x7f3e3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 0x7f3e3006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009bf0 tx=0x7f3e380091a0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 msgr2=0x7f3e48113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009180 tx=0x7f3e3c009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 shutdown_connections 2024-04-26T04:48:54.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e48101d10 0x7f3e481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 --2- 172.21.15.77:0/2098092812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e481026b0 0x7f3e48113ee0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 >> 172.21.15.77:0/2098092812 conn(0x7f3e480f7860 msgr2=0x7f3e480f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 shutdown_connections 2024-04-26T04:48:54.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:54.300+0000 7f3e4dd92700 1 -- 172.21.15.77:0/2098092812 wait complete. 2024-04-26T04:48:55.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:54 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2098092812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:55.138 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T04:48:55.139 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:48:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:54 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2098092812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:55.393 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:56.142 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:55 smithi077 ceph-mon[26311]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:55 smithi078 ceph-mon[33746]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 -- 172.21.15.77:0/3934429650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fe7d0 msgr2=0x7f834c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 --2- 172.21.15.77:0/3934429650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fe7d0 0x7f834c106ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8340004940 tx=0x7f8340009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 -- 172.21.15.77:0/3934429650 shutdown_connections 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 --2- 172.21.15.77:0/3934429650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fe7d0 0x7f834c106ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 --2- 172.21.15.77:0/3934429650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fde30 0x7f834c0fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 -- 172.21.15.77:0/3934429650 >> 172.21.15.77:0/3934429650 conn(0x7f834c0f7870 msgr2=0x7f834c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 -- 172.21.15.77:0/3934429650 shutdown_connections 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.083+0000 7f83528f5700 1 -- 172.21.15.77:0/3934429650 wait complete. 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 Processor -- start 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 -- start start 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:57.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fe7d0 0x7f834c113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1145a0 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f83528f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c10dba0 con 0x7f834c0fe7d0 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f834bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f834bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51396/0 (socket says 172.21.15.77:51396) 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f834bfff700 1 -- 172.21.15.77:0/16883476 learned_addr learned my addr 172.21.15.77:0/16883476 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.084+0000 7f834b7fe700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fe7d0 0x7f834c113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f834bfff700 1 -- 172.21.15.77:0/16883476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fe7d0 msgr2=0x7f834c113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f834bfff700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fe7d0 0x7f834c113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f834bfff700 1 -- 172.21.15.77:0/16883476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340005040 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f834bfff700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f833c002a50 tx=0x7f833c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f833c007cc0 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f833c00f040 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f833c009640 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c10de80 con 0x7f834c0fde30 2024-04-26T04:48:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.085+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c10e390 con 0x7f834c0fde30 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.086+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c066a70 con 0x7f834c0fde30 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.086+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f833c0097a0 con 0x7f834c0fde30 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.087+0000 7f8348ff9700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 0x7f833406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.087+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f833c086b80 con 0x7f834c0fde30 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.087+0000 7f834b7fe700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 0x7f833406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.087+0000 7f834b7fe700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 0x7f833406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f834c10ee80 tx=0x7f8340009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:57.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.091+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f833c056c80 con 0x7f834c0fde30 2024-04-26T04:48:57.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.279+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f834c10eff0 con 0x7f834c0fde30 2024-04-26T04:48:57.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.279+0000 7f8348ff9700 1 -- 172.21.15.77:0/16883476 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11009 (secure 0 0 0) 0x7f833c056810 con 0x7f834c0fde30 2024-04-26T04:48:57.281 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:48:57.281 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T04:48:35.480862+0000","last_up_change":"2024-04-26T04:48:34.475903+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6803","nonce":1220397746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6805","nonce":1220397746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6809","nonce":1220397746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6807","nonce":1220397746}]},"public_addr":"172.21.15.77:6803/1220397746","cluster_addr":"172.21.15.77:6805/1220397746","heartbeat_back_addr":"172.21.15.77:6809/1220397746","heartbeat_front_addr":"172.21.15.77:6807/1220397746","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6811","nonce":50899416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6813","nonce":50899416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6817","nonce":50899416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6815","nonce":50899416}]},"public_addr":"172.21.15.77:6811/50899416","cluster_addr":"172.21.15.77:6813/50899416","heartbeat_back_addr":"172.21.15.77:6817/50899416","heartbeat_front_addr":"172.21.15.77:6815/50899416","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6819","nonce":2248042345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6821","nonce":2248042345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6825","nonce":2248042345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6823","nonce":2248042345}]},"public_addr":"172.21.15.77:6819/2248042345","cluster_addr":"172.21.15.77:6821/2248042345","heartbeat_back_addr":"172.21.15.77:6825/2248042345","heartbeat_front_addr":"172.21.15.77:6823/2248042345","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6801","nonce":632729185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6803","nonce":632729185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6807","nonce":632729185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6805","nonce":632729185}]},"public_addr":"172.21.15.78:6801/632729185","cluster_addr":"172.21.15.78:6803/632729185","heartbeat_back_addr":"172.21.15.78:6807/632729185","heartbeat_front_addr":"172.21.15.78:6805/632729185","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6809","nonce":489554457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6811","nonce":489554457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6815","nonce":489554457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6813","nonce":489554457}]},"public_addr":"172.21.15.78:6809/489554457","cluster_addr":"172.21.15.78:6811/489554457","heartbeat_back_addr":"172.21.15.78:6815/489554457","heartbeat_front_addr":"172.21.15.78:6813/489554457","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6817","nonce":2307894772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6819","nonce":2307894772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6823","nonce":2307894772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6821","nonce":2307894772}]},"public_addr":"172.21.15.78:6817/2307894772","cluster_addr":"172.21.15.78:6819/2307894772","heartbeat_back_addr":"172.21.15.78:6823/2307894772","heartbeat_front_addr":"172.21.15.78:6821/2307894772","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 msgr2=0x7f833406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 0x7f833406b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f834c10ee80 tx=0x7f8340009e30 comp rx=0 tx=0).stop 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 msgr2=0x7f834c113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f833c002a50 tx=0x7f833c0046b0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 shutdown_connections 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f83340692a0 0x7f833406b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f834c0fde30 0x7f834c113a10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 --2- 172.21.15.77:0/16883476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f834c0fe7d0 0x7f834c113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:57.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 >> 172.21.15.77:0/16883476 conn(0x7f834c0f7870 msgr2=0x7f834c0f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:57.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 shutdown_connections 2024-04-26T04:48:57.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:48:57.281+0000 7f83528f5700 1 -- 172.21.15.77:0/16883476 wait complete. 2024-04-26T04:48:57.975 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T04:48:57.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T04:48:57.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T04:48:57.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T04:48:57.977 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T04:48:57.977 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T04:48:58.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:57 smithi077 ceph-mon[26311]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:57 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/16883476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:57 smithi078 ceph-mon[33746]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:57 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/16883476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:48:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:48:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:00.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:48:59 smithi077 ceph-mon[26311]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:48:59 smithi078 ceph-mon[33746]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 -- 172.21.15.77:0/2496812652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144102e50 msgr2=0x7fc144103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/2496812652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144102e50 0x7fc144103220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc140004940 tx=0x7fc140009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 -- 172.21.15.77:0/2496812652 shutdown_connections 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/2496812652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc144065640 0x7fc144065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/2496812652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144102e50 0x7fc144103220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 -- 172.21.15.77:0/2496812652 >> 172.21.15.77:0/2496812652 conn(0x7fc144071d00 msgr2=0x7fc144072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 -- 172.21.15.77:0/2496812652 shutdown_connections 2024-04-26T04:49:01.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.462+0000 7fc14c3b7700 1 -- 172.21.15.77:0/2496812652 wait complete. 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 Processor -- start 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 -- start start 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc144102e50 0x7fc1441140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1441147a0 con 0x7fc144065640 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14c3b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14410db00 con 0x7fc144102e50 2024-04-26T04:49:01.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14a153700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14a153700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51962/0 (socket says 172.21.15.77:51962) 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14a153700 1 -- 172.21.15.77:0/184378917 learned_addr learned my addr 172.21.15.77:0/184378917 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14a153700 1 -- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc144102e50 msgr2=0x7fc1441140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.463+0000 7fc14a153700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc144102e50 0x7fc1441140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc14a153700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140005040 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc14a153700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc14000b3b0 tx=0x7fc140009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14000e040 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc14000bbc0 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14410dd80 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14001c3f0 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.464+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14410e340 con 0x7fc144065640 2024-04-26T04:49:01.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.465+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc14410e860 con 0x7fc144065640 2024-04-26T04:49:01.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.465+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc14000bd30 con 0x7fc144065640 2024-04-26T04:49:01.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.465+0000 7fc13affd700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 0x7fc13006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:01.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc1400880f0 con 0x7fc144065640 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc149952700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 0x7fc13006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc13affd700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 0x7fc130071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc130071a30 con 0x7fc13006ef70 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14008a6e0 con 0x7fc144065640 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc149952700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 0x7fc13006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc134006fd0 tx=0x7fc134006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc14a954700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 0x7fc130071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:01.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.466+0000 7fc14a954700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 0x7fc130071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:01.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.467+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== osd.3 v2:172.21.15.78:6800/632729185 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc130071a30 con 0x7fc13006ef70 2024-04-26T04:49:01.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.482+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 --> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1440faac0 con 0x7fc13006ef70 2024-04-26T04:49:01.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.482+0000 7fc13affd700 1 -- 172.21.15.77:0/184378917 <== osd.3 v2:172.21.15.78:6800/632729185 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc1440faac0 con 0x7fc13006ef70 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 msgr2=0x7fc130071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 0x7fc130071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 msgr2=0x7fc13006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 0x7fc13006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc134006fd0 tx=0x7fc134006db0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 msgr2=0x7fc144113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc14000b3b0 tx=0x7fc140009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 shutdown_connections 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:6800/632729185,v1:172.21.15.78:6801/632729185] conn(0x7fc13006ef70 0x7fc130071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc1300692a0 0x7fc13006b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc144065640 0x7fc144113b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 --2- 172.21.15.77:0/184378917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc144102e50 0x7fc1441140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:01.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 >> 172.21.15.77:0/184378917 conn(0x7fc144071d00 msgr2=0x7fc1440fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:01.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 shutdown_connections 2024-04-26T04:49:01.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:01.483+0000 7fc14c3b7700 1 -- 172.21.15.77:0/184378917 wait complete. 2024-04-26T04:49:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:01 smithi077 ceph-mon[26311]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:02.207 INFO:teuthology.orchestra.run.smithi077.stdout:115964117011 2024-04-26T04:49:02.208 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T04:49:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:01 smithi078 ceph-mon[33746]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:02.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 -- 172.21.15.77:0/1990950934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 msgr2=0x7f89040fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:02.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 --2- 172.21.15.77:0/1990950934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f89040fee20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f88f4004940 tx=0x7f88f4009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:02.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 -- 172.21.15.77:0/1990950934 shutdown_connections 2024-04-26T04:49:02.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 --2- 172.21.15.77:0/1990950934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f89040fee20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 --2- 172.21.15.77:0/1990950934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 0x7f89040fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.946+0000 7f890bdd4700 1 -- 172.21.15.77:0/1990950934 >> 172.21.15.77:0/1990950934 conn(0x7f89040f89b0 msgr2=0x7f89040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 -- 172.21.15.77:0/1990950934 shutdown_connections 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 -- 172.21.15.77:0/1990950934 wait complete. 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 Processor -- start 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 -- start start 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 0x7f8904113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89041144e0 con 0x7f89040fe9b0 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890bdd4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890410dae0 con 0x7f89040fd1b0 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890936f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890936f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51978/0 (socket says 172.21.15.77:51978) 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f890936f700 1 -- 172.21.15.77:0/3110333058 learned_addr learned my addr 172.21.15.77:0/3110333058 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:02.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.947+0000 7f8909b70700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 0x7f8904113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890936f700 1 -- 172.21.15.77:0/3110333058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 msgr2=0x7f8904113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890936f700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 0x7f8904113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890936f700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4005040 con 0x7f89040fe9b0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890936f700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009b30 tx=0x7f88f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f4021070 con 0x7f89040fe9b0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f4013c00 con 0x7f89040fe9b0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890410dd60 con 0x7f89040fe9b0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f400e070 con 0x7f89040fe9b0 2024-04-26T04:49:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.948+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890410e1f0 con 0x7f89040fe9b0 2024-04-26T04:49:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.949+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f890410e760 con 0x7f89040fe9b0 2024-04-26T04:49:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.949+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f88f401c400 con 0x7f89040fe9b0 2024-04-26T04:49:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f88fa7fc700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 0x7f88f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f88f4088230 con 0x7f89040fe9b0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f8909b70700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 0x7f88f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f88fa7fc700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 0x7f88f0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88f0071a30 con 0x7f88f006ef70 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f408a880 con 0x7f89040fe9b0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.950+0000 7f890a371700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 0x7f88f0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.951+0000 7f8909b70700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 0x7f88f006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8900009b80 tx=0x7f890000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.951+0000 7f890a371700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 0x7f88f0071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.951+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== osd.2 v2:172.21.15.77:6818/2248042345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f88f0071a30 con 0x7f88f006ef70 2024-04-26T04:49:02.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.966+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 --> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89040faad0 con 0x7f88f006ef70 2024-04-26T04:49:02.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.966+0000 7f88fa7fc700 1 -- 172.21.15.77:0/3110333058 <== osd.2 v2:172.21.15.77:6818/2248042345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f89040faad0 con 0x7f88f006ef70 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 msgr2=0x7f88f0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 0x7f88f0071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 msgr2=0x7f88f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 0x7f88f006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8900009b80 tx=0x7f890000b040 comp rx=0 tx=0).stop 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 msgr2=0x7f8904113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009b30 tx=0x7f88f4009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 shutdown_connections 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6818/2248042345,v1:172.21.15.77:6819/2248042345] conn(0x7f88f006ef70 0x7f88f0071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f88f00692a0 0x7f88f006b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89040fd1b0 0x7f8904113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 --2- 172.21.15.77:0/3110333058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89040fe9b0 0x7f8904113e90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:02.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 >> 172.21.15.77:0/3110333058 conn(0x7f89040f89b0 msgr2=0x7f89040fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:02.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 shutdown_connections 2024-04-26T04:49:02.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:02.967+0000 7f890bdd4700 1 -- 172.21.15.77:0/3110333058 wait complete. 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- 172.21.15.77:0/818549788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 msgr2=0x7fe2941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 --2- 172.21.15.77:0/818549788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe2941031b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe290004940 tx=0x7fe290009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- 172.21.15.77:0/818549788 shutdown_connections 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 --2- 172.21.15.77:0/818549788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 0x7fe2940fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 --2- 172.21.15.77:0/818549788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe2941031b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- 172.21.15.77:0/818549788 >> 172.21.15.77:0/818549788 conn(0x7fe2940f8990 msgr2=0x7fe2940fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- 172.21.15.77:0/818549788 shutdown_connections 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- 172.21.15.77:0/818549788 wait complete. 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 Processor -- start 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.565+0000 7fe29b80e700 1 -- start start 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:03.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 0x7fe294113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe294114500 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29410db50 con 0x7fe294103780 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52008/0 (socket says 172.21.15.77:52008) 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 -- 172.21.15.77:0/4100414758 learned_addr learned my addr 172.21.15.77:0/4100414758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe298da9700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 0x7fe294113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 -- 172.21.15.77:0/4100414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 msgr2=0x7fe294113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 0x7fe294113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe290005040 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe2995aa700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe29000b3b0 tx=0x7fe290009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29000e040 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29410ddd0 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe29000bbc0 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29001c3f0 con 0x7fe294102de0 2024-04-26T04:49:03.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.566+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29410e390 con 0x7fe294102de0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.567+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe29410e870 con 0x7fe294102de0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe29000bd30 con 0x7fe294102de0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 0x7fe28006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe298da9700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 0x7fe28006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe290088bc0 con 0x7fe294102de0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 0x7fe280071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe280071a30 con 0x7fe28006ef70 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.568+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe290088fa0 con 0x7fe294102de0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.569+0000 7fe299dab700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 0x7fe280071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.569+0000 7fe298da9700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 0x7fe28006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe284007fd0 tx=0x7fe284009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.569+0000 7fe299dab700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 0x7fe280071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.569+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== osd.1 v2:172.21.15.77:6810/50899416 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe280071a30 con 0x7fe28006ef70 2024-04-26T04:49:03.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.586+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 --> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe29405e660 con 0x7fe28006ef70 2024-04-26T04:49:03.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.586+0000 7fe289ffb700 1 -- 172.21.15.77:0/4100414758 <== osd.1 v2:172.21.15.77:6810/50899416 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe29405e660 con 0x7fe28006ef70 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.586+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 msgr2=0x7fe280071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.586+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 0x7fe280071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 msgr2=0x7fe28006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 0x7fe28006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe284007fd0 tx=0x7fe284009040 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 msgr2=0x7fe294113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe29000b3b0 tx=0x7fe290009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 shutdown_connections 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6810/50899416,v1:172.21.15.77:6811/50899416] conn(0x7fe28006ef70 0x7fe280071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe2800692a0 0x7fe28006b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe294102de0 0x7fe294113970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 --2- 172.21.15.77:0/4100414758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe294103780 0x7fe294113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 >> 172.21.15.77:0/4100414758 conn(0x7fe2940f8990 msgr2=0x7fe294107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:03.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 shutdown_connections 2024-04-26T04:49:03.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:03.587+0000 7fe29b80e700 1 -- 172.21.15.77:0/4100414758 wait complete. 2024-04-26T04:49:04.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:03 smithi077 ceph-mon[26311]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:04.204 INFO:teuthology.orchestra.run.smithi077.stdout:60129542172 2024-04-26T04:49:04.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T04:49:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:03 smithi078 ceph-mon[33746]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:04.442 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:06.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:05 smithi077 ceph-mon[26311]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:05 smithi078 ceph-mon[33746]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:06.643 INFO:teuthology.orchestra.run.smithi077.stdout:81604378648 2024-04-26T04:49:06.643 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T04:49:07.113 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:07 smithi077 ceph-mon[26311]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:08.102 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:07 smithi078 ceph-mon[33746]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:08.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 -- 172.21.15.77:0/2770549644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fde40 msgr2=0x7fbebc0fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:08.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 --2- 172.21.15.77:0/2770549644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fde40 0x7fbebc0fe210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbeac004940 tx=0x7fbeac009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 -- 172.21.15.77:0/2770549644 shutdown_connections 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 --2- 172.21.15.77:0/2770549644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 --2- 172.21.15.77:0/2770549644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fde40 0x7fbebc0fe210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 -- 172.21.15.77:0/2770549644 >> 172.21.15.77:0/2770549644 conn(0x7fbebc0f7850 msgr2=0x7fbebc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 -- 172.21.15.77:0/2770549644 shutdown_connections 2024-04-26T04:49:08.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.130+0000 7fbec2728700 1 -- 172.21.15.77:0/2770549644 wait complete. 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 Processor -- start 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 -- start start 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fde40 0x7fbebc113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc114690 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbec2728700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc10da80 con 0x7fbebc0fde40 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41384/0 (socket says 172.21.15.77:41384) 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 -- 172.21.15.77:0/1934332934 learned_addr learned my addr 172.21.15.77:0/1934332934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebbfff700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fde40 0x7fbebc113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 -- 172.21.15.77:0/1934332934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fde40 msgr2=0x7fbebc113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:08.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fde40 0x7fbebc113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac005040 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.131+0000 7fbebb7fe700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c830 tx=0x7fbeb0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb000a760 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbeb000ad90 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc10dd60 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb00113a0 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc10e2a0 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.132+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbebc10e820 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.133+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fbeb0012400 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.133+0000 7fbeb8ff9700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 0x7fbea406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.133+0000 7fbebbfff700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 0x7fbea406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.133+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbeb0086f20 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbeb8ff9700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 0x7fbea4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:08.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbea4071a30 con 0x7fbea406ef70 2024-04-26T04:49:08.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb0089570 con 0x7fbebc0fe7e0 2024-04-26T04:49:08.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbebbfff700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 0x7fbea406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbeac004f90 tx=0x7fbeac009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbec0cc5700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 0x7fbea4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.134+0000 7fbec0cc5700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 0x7fbea4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.135+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== osd.0 v2:172.21.15.77:6802/1220397746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbea4071a30 con 0x7fbea406ef70 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 --> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbebc0f9820 con 0x7fbea406ef70 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbeb8ff9700 1 -- 172.21.15.77:0/1934332934 <== osd.0 v2:172.21.15.77:6802/1220397746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbebc0f9820 con 0x7fbea406ef70 2024-04-26T04:49:08.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 msgr2=0x7fbea4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:08.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 0x7fbea4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 msgr2=0x7fbea406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:08.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 0x7fbea406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbeac004f90 tx=0x7fbeac009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:08.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 msgr2=0x7fbebc114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:08.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c830 tx=0x7fbeb0004470 comp rx=0 tx=0).stop 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 shutdown_connections 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6802/1220397746,v1:172.21.15.77:6803/1220397746] conn(0x7fbea406ef70 0x7fbea4071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbea40692a0 0x7fbea406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc0fde40 0x7fbebc113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 --2- 172.21.15.77:0/1934332934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbebc0fe7e0 0x7fbebc114040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:08.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 >> 172.21.15.77:0/1934332934 conn(0x7fbebc0f7850 msgr2=0x7fbebc0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:08.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 shutdown_connections 2024-04-26T04:49:08.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:08.150+0000 7fbec2728700 1 -- 172.21.15.77:0/1934332934 wait complete. 2024-04-26T04:49:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:09.524 INFO:teuthology.orchestra.run.smithi077.stdout:38654705699 2024-04-26T04:49:09.524 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T04:49:10.068 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:09 smithi077 ceph-mon[26311]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:09 smithi078 ceph-mon[33746]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:10.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 -- 172.21.15.77:0/2981531762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fde70 msgr2=0x7f4b480fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:10.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 --2- 172.21.15.77:0/2981531762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fde70 0x7f4b480fe240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b38004940 tx=0x7f4b38009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:10.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 -- 172.21.15.77:0/2981531762 shutdown_connections 2024-04-26T04:49:10.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 --2- 172.21.15.77:0/2981531762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fe810 0x7f4b48106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 --2- 172.21.15.77:0/2981531762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fde70 0x7f4b480fe240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 -- 172.21.15.77:0/2981531762 >> 172.21.15.77:0/2981531762 conn(0x7f4b480f7860 msgr2=0x7f4b480f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 -- 172.21.15.77:0/2981531762 shutdown_connections 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.484+0000 7f4b4d950700 1 -- 172.21.15.77:0/2981531762 wait complete. 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 Processor -- start 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 -- start start 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 0x7f4b480fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480fc270 con 0x7f4b480fe810 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b4d950700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480fc3e0 con 0x7f4b480fde70 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b467fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b467fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41396/0 (socket says 172.21.15.77:41396) 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.485+0000 7f4b467fc700 1 -- 172.21.15.77:0/1226320487 learned_addr learned my addr 172.21.15.77:0/1226320487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b46ffd700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 0x7f4b480fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b467fc700 1 -- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 msgr2=0x7f4b480fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b467fc700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 0x7f4b480fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b467fc700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b38005040 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b46ffd700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 0x7f4b480fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b467fc700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b790 tx=0x7f4b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3c00f770 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b480fc6c0 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b3c00fda0 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3c00d310 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.486+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4811e6a0 con 0x7f4b480fe810 2024-04-26T04:49:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.487+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b480f96a0 con 0x7f4b480fe810 2024-04-26T04:49:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.487+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4b3c010420 con 0x7f4b480fe810 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b2f7fe700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 0x7f4b3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b46ffd700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 0x7f4b3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4b3c086e70 con 0x7f4b480fe810 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b2f7fe700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 0x7f4b30071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b30071a30 con 0x7f4b3006ef70 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b3c08c470 con 0x7f4b480fe810 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b46ffd700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 0x7f4b3006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800b3b0 tx=0x7f4b38009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.488+0000 7f4b477fe700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 0x7f4b30071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.489+0000 7f4b477fe700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 0x7f4b30071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:10.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.489+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== osd.5 v2:172.21.15.78:6816/2307894772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4b30071a30 con 0x7f4b3006ef70 2024-04-26T04:49:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.507+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 --> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b48066af0 con 0x7f4b3006ef70 2024-04-26T04:49:10.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.507+0000 7f4b2f7fe700 1 -- 172.21.15.77:0/1226320487 <== osd.5 v2:172.21.15.78:6816/2307894772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b48066af0 con 0x7f4b3006ef70 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 msgr2=0x7f4b30071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 0x7f4b30071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 msgr2=0x7f4b3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 0x7f4b3006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800b3b0 tx=0x7f4b38009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 msgr2=0x7f4b480fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b790 tx=0x7f4b3c009250 comp rx=0 tx=0).stop 2024-04-26T04:49:10.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 shutdown_connections 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:6816/2307894772,v1:172.21.15.78:6817/2307894772] conn(0x7f4b3006ef70 0x7f4b30071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4b300692a0 0x7f4b3006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b480fde70 0x7f4b480fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 --2- 172.21.15.77:0/1226320487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b480fe810 0x7f4b480fbca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 >> 172.21.15.77:0/1226320487 conn(0x7f4b480f7860 msgr2=0x7f4b480f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 shutdown_connections 2024-04-26T04:49:10.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:10.508+0000 7f4b4d950700 1 -- 172.21.15.77:0/1226320487 wait complete. 2024-04-26T04:49:11.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:11 smithi077 ceph-mon[26311]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:12.051 INFO:teuthology.orchestra.run.smithi077.stdout:171798691850 2024-04-26T04:49:12.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T04:49:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:11 smithi078 ceph-mon[33746]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:13.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- 172.21.15.77:0/1808176575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 msgr2=0x7f6fdc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 --2- 172.21.15.77:0/1808176575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0fcdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc004940 tx=0x7f6fcc009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- 172.21.15.77:0/1808176575 shutdown_connections 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 --2- 172.21.15.77:0/1808176575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0fcdf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 --2- 172.21.15.77:0/1808176575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 0x7f6fdc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- 172.21.15.77:0/1808176575 >> 172.21.15.77:0/1808176575 conn(0x7f6fdc0f8990 msgr2=0x7f6fdc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- 172.21.15.77:0/1808176575 shutdown_connections 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- 172.21.15.77:0/1808176575 wait complete. 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 Processor -- start 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.245+0000 7f6fe4739700 1 -- start start 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 0x7f6fdc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc072240 con 0x7f6fdc103780 2024-04-26T04:49:13.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc070ce0 con 0x7f6fdc102de0 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41420/0 (socket says 172.21.15.77:41420) 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 -- 172.21.15.77:0/486894940 learned_addr learned my addr 172.21.15.77:0/486894940 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 -- 172.21.15.77:0/486894940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 msgr2=0x7f6fdc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe24d5700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 0x7f6fdc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 0x7f6fdc070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 -- 172.21.15.77:0/486894940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc005040 con 0x7f6fdc103780 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe1cd4700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009100 tx=0x7f6fcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc00e040 con 0x7f6fdc103780 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc070f60 con 0x7f6fdc103780 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fcc013420 con 0x7f6fdc103780 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc01c610 con 0x7f6fdc103780 2024-04-26T04:49:13.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.246+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc11a1b0 con 0x7f6fdc103780 2024-04-26T04:49:13.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.247+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc1000f0 con 0x7f6fdc103780 2024-04-26T04:49:13.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.248+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6fcc013590 con 0x7f6fdc103780 2024-04-26T04:49:13.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.248+0000 7f6fd2ffd700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.248+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6fcc088090 con 0x7f6fdc103780 2024-04-26T04:49:13.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.248+0000 7f6fe24d5700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:13.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.249+0000 7f6fe24d5700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 0x7f6fc806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a900 tx=0x7f6fd8008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:13.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.253+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6fcc054700 con 0x7f6fdc103780 2024-04-26T04:49:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:13.318 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:13.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.448+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6fdc0fe530 con 0x7f6fdc103780 2024-04-26T04:49:13.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.449+0000 7f6fd2ffd700 1 -- 172.21.15.77:0/486894940 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fcc057d20 con 0x7f6fdc103780 2024-04-26T04:49:13.450 INFO:teuthology.orchestra.run.smithi077.stdout:60129542174 2024-04-26T04:49:13.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 msgr2=0x7f6fc806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 0x7f6fc806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800a900 tx=0x7f6fd8008040 comp rx=0 tx=0).stop 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 msgr2=0x7f6fdc0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009100 tx=0x7f6fcc009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 shutdown_connections 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6fc80692a0 0x7f6fc806b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc102de0 0x7f6fdc070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 --2- 172.21.15.77:0/486894940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fdc103780 0x7f6fdc0707a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 >> 172.21.15.77:0/486894940 conn(0x7f6fdc0f8990 msgr2=0x7f6fdc0fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 shutdown_connections 2024-04-26T04:49:13.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.451+0000 7f6fe4739700 1 -- 172.21.15.77:0/486894940 wait complete. 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- 172.21.15.77:0/2963620997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 msgr2=0x7f4bd01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/2963620997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd01020d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0004940 tx=0x7f4bc0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- 172.21.15.77:0/2963620997 shutdown_connections 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/2963620997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd01026a0 0x7f4bd00fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/2963620997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd01020d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- 172.21.15.77:0/2963620997 >> 172.21.15.77:0/2963620997 conn(0x7f4bd00f7870 msgr2=0x7f4bd00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- 172.21.15.77:0/2963620997 shutdown_connections 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- 172.21.15.77:0/2963620997 wait complete. 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 Processor -- start 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 -- start start 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.555+0000 7f4bd69a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bd69a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd01026a0 0x7f4bd010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bd69a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd010f5d0 con 0x7f4bd0101d00 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bd69a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd010f710 con 0x7f4bd01026a0 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41424/0 (socket says 172.21.15.77:41424) 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 -- 172.21.15.77:0/75880552 learned_addr learned my addr 172.21.15.77:0/75880552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 -- 172.21.15.77:0/75880552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd01026a0 msgr2=0x7f4bd010f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd01026a0 0x7f4bd010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 -- 172.21.15.77:0/75880552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0005040 con 0x7f4bd0101d00 2024-04-26T04:49:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bcffff700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b3b0 tx=0x7f4bc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc000e040 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bc000bbc0 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd010f990 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc001c3f0 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.556+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0117f60 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.557+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd00fd450 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.557+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4bc000bd30 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.558+0000 7f4bccff9700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 0x7f4bb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.558+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4bc00880a0 con 0x7f4bd0101d00 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.558+0000 7f4bcf7fe700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 0x7f4bb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.558+0000 7f4bcf7fe700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 0x7f4bb806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.562+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4bc0054690 con 0x7f4bd0101d00 2024-04-26T04:49:13.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.750+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4bd0108e70 con 0x7f4bd0101d00 2024-04-26T04:49:13.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.751+0000 7f4bccff9700 1 -- 172.21.15.77:0/75880552 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bc0057cb0 con 0x7f4bd0101d00 2024-04-26T04:49:13.752 INFO:teuthology.orchestra.run.smithi077.stdout:115964117013 2024-04-26T04:49:13.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 msgr2=0x7f4bb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 0x7f4bb806b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4006db0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 msgr2=0x7f4bd010eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b3b0 tx=0x7f4bc0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 shutdown_connections 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4bb80692a0 0x7f4bb806b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0101d00 0x7f4bd010eb50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 --2- 172.21.15.77:0/75880552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bd01026a0 0x7f4bd010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 >> 172.21.15.77:0/75880552 conn(0x7f4bd00f7870 msgr2=0x7f4bd00f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 shutdown_connections 2024-04-26T04:49:13.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:13.753+0000 7f4bd69a7700 1 -- 172.21.15.77:0/75880552 wait complete. 2024-04-26T04:49:14.070 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:13 smithi077 ceph-mon[26311]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:14.070 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:13 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/486894940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T04:49:14.070 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:13 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/75880552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T04:49:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:13 smithi078 ceph-mon[33746]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:13 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/486894940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T04:49:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:13 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/75880552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T04:49:14.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117013 for osd.3 2024-04-26T04:49:14.403 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:14.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542174 for osd.1 2024-04-26T04:49:14.823 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:16.098 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:15 smithi077 ceph-mon[26311]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:15 smithi078 ceph-mon[33746]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:17.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 -- 172.21.15.77:0/2188801809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fde10 msgr2=0x7f577c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 --2- 172.21.15.77:0/2188801809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fde10 0x7f577c0fe1e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f576c004940 tx=0x7f576c009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 -- 172.21.15.77:0/2188801809 shutdown_connections 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 --2- 172.21.15.77:0/2188801809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fe7b0 0x7f577c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 --2- 172.21.15.77:0/2188801809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fde10 0x7f577c0fe1e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 -- 172.21.15.77:0/2188801809 >> 172.21.15.77:0/2188801809 conn(0x7f577c0f7870 msgr2=0x7f577c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 -- 172.21.15.77:0/2188801809 shutdown_connections 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 -- 172.21.15.77:0/2188801809 wait complete. 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.521+0000 7f5781d13700 1 Processor -- start 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5781d13700 1 -- start start 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5781d13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fde10 0x7f577c1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5781d13700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:17.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5781d13700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c114580 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5781d13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c10db80 con 0x7f577c0fde10 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49172/0 (socket says 172.21.15.77:49172) 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 -- 172.21.15.77:0/1238850353 learned_addr learned my addr 172.21.15.77:0/1238850353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577b7fe700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fde10 0x7f577c1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 -- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fde10 msgr2=0x7f577c1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fde10 0x7f577c1139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c005040 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f577affd700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f577000c830 tx=0x7f5770004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.522+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57700106f0 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.523+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5770010d20 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.523+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57700113a0 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.523+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c10de60 con 0x7f577c0fe7b0 2024-04-26T04:49:17.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.523+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c10e350 con 0x7f577c0fe7b0 2024-04-26T04:49:17.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.523+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f577c10e920 con 0x7f577c0fe7b0 2024-04-26T04:49:17.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.524+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5770012400 con 0x7f577c0fe7b0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f5763fff700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 0x7f576406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f577b7fe700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 0x7f576406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5770087130 con 0x7f577c0fe7b0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f5763fff700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 0x7f5764071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5764071a30 con 0x7f576406ef70 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5770087470 con 0x7f577c0fe7b0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f577b7fe700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 0x7f576406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b3b0 tx=0x7f576c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f577bfff700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 0x7f5764071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.525+0000 7f577bfff700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 0x7f5764071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:17.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.526+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== osd.4 v2:172.21.15.78:6808/489554457 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5764071a30 con 0x7f576406ef70 2024-04-26T04:49:17.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.541+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 --> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f577c0f96e0 con 0x7f576406ef70 2024-04-26T04:49:17.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5763fff700 1 -- 172.21.15.77:0/1238850353 <== osd.4 v2:172.21.15.78:6808/489554457 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f577c0f96e0 con 0x7f576406ef70 2024-04-26T04:49:17.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 msgr2=0x7f5764071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:17.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 0x7f5764071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 msgr2=0x7f576406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 0x7f576406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b3b0 tx=0x7f576c009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 msgr2=0x7f577c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f577000c830 tx=0x7f5770004470 comp rx=0 tx=0).stop 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 shutdown_connections 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:6808/489554457,v1:172.21.15.78:6809/489554457] conn(0x7f576406ef70 0x7f5764071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f57640692a0 0x7f576406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f577c0fde10 0x7f577c1139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 --2- 172.21.15.77:0/1238850353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0fe7b0 0x7f577c113f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:17.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.542+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 >> 172.21.15.77:0/1238850353 conn(0x7f577c0f7870 msgr2=0x7f577c0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:17.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.543+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 shutdown_connections 2024-04-26T04:49:17.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:17.543+0000 7f5781d13700 1 -- 172.21.15.77:0/1238850353 wait complete. 2024-04-26T04:49:18.077 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:17 smithi077 ceph-mon[26311]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:17 smithi078 ceph-mon[33746]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:18.494 INFO:teuthology.orchestra.run.smithi077.stdout:150323855376 2024-04-26T04:49:18.494 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T04:49:18.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 -- 172.21.15.77:0/2032847534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e000fbd40 msgr2=0x7f2e000fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:18.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 --2- 172.21.15.77:0/2032847534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e000fbd40 0x7f2e000fc1b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2de8004940 tx=0x7f2de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:18.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 -- 172.21.15.77:0/2032847534 shutdown_connections 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 --2- 172.21.15.77:0/2032847534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e000fbd40 0x7f2e000fc1b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 --2- 172.21.15.77:0/2032847534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e00101d90 0x7f2e00102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 -- 172.21.15.77:0/2032847534 >> 172.21.15.77:0/2032847534 conn(0x7f2e00071d00 msgr2=0x7f2e00072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 -- 172.21.15.77:0/2032847534 shutdown_connections 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 -- 172.21.15.77:0/2032847534 wait complete. 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.746+0000 7f2e05c38700 1 Processor -- start 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2e05c38700 1 -- start start 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2e05c38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e000fbd40 0x7f2e00115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2e05c38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2e05c38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00116950 con 0x7f2e00101d90 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2e05c38700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0010fd20 con 0x7f2e000fbd40 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49188/0 (socket says 172.21.15.77:49188) 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 -- 172.21.15.77:0/3095916308 learned_addr learned my addr 172.21.15.77:0/3095916308 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2dff7fe700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e000fbd40 0x7f2e00115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 -- 172.21.15.77:0/3095916308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e000fbd40 msgr2=0x7f2e00115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e000fbd40 0x7f2e00115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 -- 172.21.15.77:0/3095916308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8005040 con 0x7f2e00101d90 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.747+0000 7f2df6ffd700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2de800b8b0 tx=0x7f2de8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.748+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de800e040 con 0x7f2e00101d90 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.748+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2de8004290 con 0x7f2e00101d90 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.748+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de801c3f0 con 0x7f2e00101d90 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.748+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0010ffa0 con 0x7f2e00101d90 2024-04-26T04:49:18.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.748+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e00110510 con 0x7f2e00101d90 2024-04-26T04:49:18.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.749+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e000fd430 con 0x7f2e00101d90 2024-04-26T04:49:18.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.749+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2de8013820 con 0x7f2e00101d90 2024-04-26T04:49:18.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.749+0000 7f2dfcff9700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 0x7f2dec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:18.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.750+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2de8088110 con 0x7f2e00101d90 2024-04-26T04:49:18.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.750+0000 7f2dff7fe700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 0x7f2dec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:18.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.750+0000 7f2dff7fe700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 0x7f2dec06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004f40 tx=0x7f2df000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:18.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.754+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2de8058210 con 0x7f2e00101d90 2024-04-26T04:49:18.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.953+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e0005f5c0 con 0x7f2e00101d90 2024-04-26T04:49:18.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.954+0000 7f2dfcff9700 1 -- 172.21.15.77:0/3095916308 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2de8057da0 con 0x7f2e00101d90 2024-04-26T04:49:18.955 INFO:teuthology.orchestra.run.smithi077.stdout:81604378651 2024-04-26T04:49:18.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 msgr2=0x7f2dec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:18.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 0x7f2dec06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004f40 tx=0x7f2df000b040 comp rx=0 tx=0).stop 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 msgr2=0x7f2e00116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2de800b8b0 tx=0x7f2de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 shutdown_connections 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2dec0692a0 0x7f2dec06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e000fbd40 0x7f2e00115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 --2- 172.21.15.77:0/3095916308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e00101d90 0x7f2e00116300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.956+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 >> 172.21.15.77:0/3095916308 conn(0x7f2e00071d00 msgr2=0x7f2e000fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.957+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 shutdown_connections 2024-04-26T04:49:18.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:18.957+0000 7f2e05c38700 1 -- 172.21.15.77:0/3095916308 wait complete. 2024-04-26T04:49:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.623+0000 7fa58a3b9700 1 -- 172.21.15.77:0/549726889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 msgr2=0x7fa5841020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.623+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/549726889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa5841020b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa578004940 tx=0x7fa578009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 -- 172.21.15.77:0/549726889 shutdown_connections 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/549726889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 0x7fa5840fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/549726889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa5841020b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 -- 172.21.15.77:0/549726889 >> 172.21.15.77:0/549726889 conn(0x7fa5840f7890 msgr2=0x7fa5840f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 -- 172.21.15.77:0/549726889 shutdown_connections 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 -- 172.21.15.77:0/549726889 wait complete. 2024-04-26T04:49:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 Processor -- start 2024-04-26T04:49:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.624+0000 7fa58a3b9700 1 -- start start 2024-04-26T04:49:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 0x7fa584112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa584113600 con 0x7fa584101ce0 2024-04-26T04:49:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58410c9f0 con 0x7fa584102680 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49200/0 (socket says 172.21.15.77:49200) 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 -- 172.21.15.77:0/2848664864 learned_addr learned my addr 172.21.15.77:0/2848664864 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa5837fe700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 0x7fa584112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 -- 172.21.15.77:0/2848664864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 msgr2=0x7fa584112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 0x7fa584112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 -- 172.21.15.77:0/2848664864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578005040 con 0x7fa584101ce0 2024-04-26T04:49:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa583fff700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b3b0 tx=0x7fa578009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57800e040 con 0x7fa584101ce0 2024-04-26T04:49:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa578013420 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58410cc70 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57801c630 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.625+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58410d230 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.626+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5840fd430 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.627+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa578013590 con 0x7fa584101ce0 2024-04-26T04:49:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.627+0000 7fa580ff9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 0x7fa56c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.627+0000 7fa5837fe700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 0x7fa56c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.627+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa578088040 con 0x7fa584101ce0 2024-04-26T04:49:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.628+0000 7fa5837fe700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 0x7fa56c06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa58410dbb0 tx=0x7fa57400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.632+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa578058140 con 0x7fa584101ce0 2024-04-26T04:49:19.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.821+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa584108de0 con 0x7fa584101ce0 2024-04-26T04:49:19.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.821+0000 7fa580ff9700 1 -- 172.21.15.77:0/2848664864 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa578057cd0 con 0x7fa584101ce0 2024-04-26T04:49:19.823 INFO:teuthology.orchestra.run.smithi077.stdout:38654705701 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 msgr2=0x7fa56c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 0x7fa56c06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa58410dbb0 tx=0x7fa57400b040 comp rx=0 tx=0).stop 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 msgr2=0x7fa584112a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b3b0 tx=0x7fa578009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 shutdown_connections 2024-04-26T04:49:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa56c0692a0 0x7fa56c06b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa584101ce0 0x7fa584112a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 --2- 172.21.15.77:0/2848664864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa584102680 0x7fa584112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.823+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 >> 172.21.15.77:0/2848664864 conn(0x7fa5840f7890 msgr2=0x7fa5840f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.824+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 shutdown_connections 2024-04-26T04:49:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:19.824+0000 7fa58a3b9700 1 -- 172.21.15.77:0/2848664864 wait complete. 2024-04-26T04:49:20.197 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:19 smithi077 ceph-mon[26311]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:20.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:19 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3095916308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T04:49:20.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:19 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2848664864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T04:49:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:19 smithi078 ceph-mon[33746]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:19 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3095916308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T04:49:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:19 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2848664864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T04:49:20.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-04-26T04:49:20.682 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:21.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378651 for osd.2 2024-04-26T04:49:21.076 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:21.515 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:22.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:21 smithi077 ceph-mon[26311]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:21 smithi078 ceph-mon[33746]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 -- 172.21.15.77:0/1595611496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e541037c0 msgr2=0x7f2e540fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 --2- 172.21.15.77:0/1595611496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e541037c0 0x7f2e540fb3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e48004940 tx=0x7f2e48009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 -- 172.21.15.77:0/1595611496 shutdown_connections 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 --2- 172.21.15.77:0/1595611496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e541037c0 0x7f2e540fb3c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 --2- 172.21.15.77:0/1595611496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e54102e20 0x7f2e541031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 -- 172.21.15.77:0/1595611496 >> 172.21.15.77:0/1595611496 conn(0x7f2e54072380 msgr2=0x7f2e54072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 -- 172.21.15.77:0/1595611496 shutdown_connections 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.171+0000 7f2e58b32700 1 -- 172.21.15.77:0/1595611496 wait complete. 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 Processor -- start 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 -- start start 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 0x7f2e540fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e540fe540 con 0x7f2e54102e20 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e58b32700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e540fd020 con 0x7f2e541037c0 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e5259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e5259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49212/0 (socket says 172.21.15.77:49212) 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e5259c700 1 -- 172.21.15.77:0/47176628 learned_addr learned my addr 172.21.15.77:0/47176628 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.172+0000 7f2e51d9b700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 0x7f2e540fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e5259c700 1 -- 172.21.15.77:0/47176628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 msgr2=0x7f2e540fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e5259c700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 0x7f2e540fcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e5259c700 1 -- 172.21.15.77:0/47176628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48005040 con 0x7f2e54102e20 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e51d9b700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 0x7f2e540fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e5259c700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00ca90 tx=0x7f2e4c007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4c00aa50 con 0x7f2e54102e20 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e4c00abb0 con 0x7f2e54102e20 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e540fd2d0 con 0x7f2e54102e20 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4c0115e0 con 0x7f2e54102e20 2024-04-26T04:49:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.173+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5411a1f0 con 0x7f2e54102e20 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.174+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e54100160 con 0x7f2e54102e20 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.174+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2e4c00ad20 con 0x7f2e54102e20 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.175+0000 7f2e42ffd700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 0x7f2e3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.175+0000 7f2e51d9b700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 0x7f2e3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.175+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2e4c0874f0 con 0x7f2e54102e20 2024-04-26T04:49:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.175+0000 7f2e51d9b700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 0x7f2e3c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e540fdcd0 tx=0x7f2e48009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.179+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2e4c053b60 con 0x7f2e54102e20 2024-04-26T04:49:23.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.369+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2e540fe6e0 con 0x7f2e54102e20 2024-04-26T04:49:23.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.369+0000 7f2e42ffd700 1 -- 172.21.15.77:0/47176628 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e4c057180 con 0x7f2e54102e20 2024-04-26T04:49:23.370 INFO:teuthology.orchestra.run.smithi077.stdout:171798691852 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 msgr2=0x7f2e3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 0x7f2e3c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e540fdcd0 tx=0x7f2e48009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 msgr2=0x7f2e540fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00ca90 tx=0x7f2e4c007f00 comp rx=0 tx=0).stop 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 shutdown_connections 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2e3c0692a0 0x7f2e3c06b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e54102e20 0x7f2e540fc5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 --2- 172.21.15.77:0/47176628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e541037c0 0x7f2e540fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 >> 172.21.15.77:0/47176628 conn(0x7f2e54072380 msgr2=0x7f2e540fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 shutdown_connections 2024-04-26T04:49:23.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:23.371+0000 7f2e58b32700 1 -- 172.21.15.77:0/47176628 wait complete. 2024-04-26T04:49:24.186 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:23 smithi077 ceph-mon[26311]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:24.186 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:23 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/47176628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T04:49:24.186 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:24.213 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691850 got 171798691852 for osd.5 2024-04-26T04:49:24.213 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:23 smithi078 ceph-mon[33746]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:23 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/47176628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T04:49:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:24.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 -- 172.21.15.77:0/3006781642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 msgr2=0x7f47e00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 --2- 172.21.15.77:0/3006781642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e00fcdf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004940 tx=0x7f47d0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:24.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 -- 172.21.15.77:0/3006781642 shutdown_connections 2024-04-26T04:49:24.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 --2- 172.21.15.77:0/3006781642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e00fcdf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 --2- 172.21.15.77:0/3006781642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 -- 172.21.15.77:0/3006781642 >> 172.21.15.77:0/3006781642 conn(0x7f47e00f8990 msgr2=0x7f47e00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 -- 172.21.15.77:0/3006781642 shutdown_connections 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.772+0000 7f47e81c7700 1 -- 172.21.15.77:0/3006781642 wait complete. 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 Processor -- start 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 -- start start 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e0113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01144e0 con 0x7f47e0103780 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e81c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e010dae0 con 0x7f47e0102de0 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e5762700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e5762700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49232/0 (socket says 172.21.15.77:49232) 2024-04-26T04:49:24.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.773+0000 7f47e5762700 1 -- 172.21.15.77:0/1391200266 learned_addr learned my addr 172.21.15.77:0/1391200266 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5f63700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e0113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5762700 1 -- 172.21.15.77:0/1391200266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 msgr2=0x7f47e0113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5762700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e0113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5762700 1 -- 172.21.15.77:0/1391200266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0005040 con 0x7f47e0103780 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5f63700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e0113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e5762700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009100 tx=0x7f47d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000e040 con 0x7f47e0103780 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d0013420 con 0x7f47e0103780 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e010dd60 con 0x7f47e0103780 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d001c610 con 0x7f47e0103780 2024-04-26T04:49:24.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.774+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e010e320 con 0x7f47e0103780 2024-04-26T04:49:24.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.775+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e00fe530 con 0x7f47e0103780 2024-04-26T04:49:24.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.776+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f47d0013590 con 0x7f47e0103780 2024-04-26T04:49:24.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.776+0000 7f47d67fc700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.776+0000 7f47e5f63700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.776+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f47d0088000 con 0x7f47e0103780 2024-04-26T04:49:24.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.776+0000 7f47e5f63700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 0x7f47cc06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009fd0 tx=0x7f47dc009e50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.780+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f47d0058250 con 0x7f47e0103780 2024-04-26T04:49:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.965+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f47e0109ec0 con 0x7f47e0103780 2024-04-26T04:49:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.965+0000 7f47d67fc700 1 -- 172.21.15.77:0/1391200266 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47d0057de0 con 0x7f47e0103780 2024-04-26T04:49:24.966 INFO:teuthology.orchestra.run.smithi077.stdout:150323855377 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 msgr2=0x7f47cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 0x7f47cc06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009fd0 tx=0x7f47dc009e50 comp rx=0 tx=0).stop 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 msgr2=0x7f47e0113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009100 tx=0x7f47d0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 shutdown_connections 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f47cc0692a0 0x7f47cc06b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e0102de0 0x7f47e0113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 --2- 172.21.15.77:0/1391200266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47e0103780 0x7f47e0113e90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 >> 172.21.15.77:0/1391200266 conn(0x7f47e00f8990 msgr2=0x7f47e00f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 shutdown_connections 2024-04-26T04:49:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:24.967+0000 7f47e81c7700 1 -- 172.21.15.77:0/1391200266 wait complete. 2024-04-26T04:49:25.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855377 for osd.4 2024-04-26T04:49:25.668 DEBUG:teuthology.parallel:result is None 2024-04-26T04:49:25.668 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T04:49:25.668 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T04:49:25.919 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:25 smithi077 ceph-mon[26311]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:25 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1391200266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T04:49:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:25 smithi078 ceph-mon[33746]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:25 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1391200266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 -- 172.21.15.77:0/3867854180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc1026a0 msgr2=0x7ff7cc0fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 --2- 172.21.15.77:0/3867854180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc0fbd10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0004940 tx=0x7ff7c0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 -- 172.21.15.77:0/3867854180 shutdown_connections 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 --2- 172.21.15.77:0/3867854180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc0fbd10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 --2- 172.21.15.77:0/3867854180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.684+0000 7ff7d1798700 1 -- 172.21.15.77:0/3867854180 >> 172.21.15.77:0/3867854180 conn(0x7ff7cc0f7870 msgr2=0x7ff7cc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:27.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 -- 172.21.15.77:0/3867854180 shutdown_connections 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 -- 172.21.15.77:0/3867854180 wait complete. 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 Processor -- start 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 -- start start 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc112df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc113440 con 0x7ff7cc101d00 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.685+0000 7ff7d1798700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc10ca40 con 0x7ff7cc1026a0 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43116/0 (socket says 172.21.15.77:43116) 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 -- 172.21.15.77:0/4002992703 learned_addr learned my addr 172.21.15.77:0/4002992703 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7ca7fc700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 -- 172.21.15.77:0/4002992703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc1026a0 msgr2=0x7ff7cc112df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc112df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 -- 172.21.15.77:0/4002992703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c0005040 con 0x7ff7cc101d00 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7caffd700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc002a50 tx=0x7ff7bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7bc007cc0 con 0x7ff7cc101d00 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7bc00f040 con 0x7ff7cc101d00 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc10cd20 con 0x7ff7cc101d00 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7bc00da30 con 0x7ff7cc101d00 2024-04-26T04:49:27.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.686+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc10d2b0 con 0x7ff7cc101d00 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.687+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc0fd450 con 0x7ff7cc101d00 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.688+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff7bc00dbb0 con 0x7ff7cc101d00 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.688+0000 7ff7b37fe700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 0x7ff7b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.688+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff7bc086c10 con 0x7ff7cc101d00 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.688+0000 7ff7ca7fc700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 0x7ff7b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:27.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.689+0000 7ff7ca7fc700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 0x7ff7b406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0004890 tx=0x7ff7c0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:27.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.693+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff7bc056c90 con 0x7ff7cc101d00 2024-04-26T04:49:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.874+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7cc108de0 con 0x7ff7b40692a0 2024-04-26T04:49:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.876+0000 7ff7b37fe700 1 -- 172.21.15.77:0/4002992703 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27968 (secure 0 0 0) 0x7ff7cc108de0 con 0x7ff7b40692a0 2024-04-26T04:49:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 msgr2=0x7ff7b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 0x7ff7b406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0004890 tx=0x7ff7c0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 msgr2=0x7ff7cc1128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc002a50 tx=0x7ff7bc0046b0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 shutdown_connections 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff7b40692a0 0x7ff7b406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7cc101d00 0x7ff7cc1128b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 --2- 172.21.15.77:0/4002992703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7cc1026a0 0x7ff7cc112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 >> 172.21.15.77:0/4002992703 conn(0x7ff7cc0f7870 msgr2=0x7ff7cc0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 shutdown_connections 2024-04-26T04:49:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:27.878+0000 7ff7d1798700 1 -- 172.21.15.77:0/4002992703 wait complete. 2024-04-26T04:49:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-04-26T04:49:28.231 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:27 smithi077 ceph-mon[26311]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:27 smithi078 ceph-mon[33746]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:28.637 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-04-26T04:49:26.472209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788080,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682832,"statfs":{"total":575970213888,"available":574139219968,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002339"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T04:48:34.479000+0000","last_change":"2024-04-26T04:48:09.390489+0000","last_active":"2024-04-26T04:48:34.479000+0000","last_peered":"2024-04-26T04:48:34.479000+0000","last_clean":"2024-04-26T04:48:34.479000+0000","last_became_active":"2024-04-26T04:48:09.390223+0000","last_became_peered":"2024-04-26T04:48:09.390223+0000","last_unstale":"2024-04-26T04:48:34.479000+0000","last_undegraded":"2024-04-26T04:48:34.479000+0000","last_fullsized":"2024-04-26T04:48:34.479000+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:47:23.007192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:47:23.007192+0000","last_clean_scrub_stamp":"2024-04-26T04:47:23.007192+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:20.343766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.621},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.40799999999999997}]},{"osd":1,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.374}]},{"osd":2,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Apr 26 04:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.26100000000000001}]},{"osd":1,"last update":"Fri Apr 26 04:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.36799999999999999}]},{"osd":3,"last update":"Fri Apr 26 04:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Apr 26 04:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 26 04:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32300000000000001,"15min":0.32000000000000001},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Apr 26 04:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Apr 26 04:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Apr 26 04:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.318,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.255,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Apr 26 04:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.248},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27300000000000002}]},{"osd":3,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Apr 26 04:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:49:28.638 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T04:49:28.896 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:28 smithi077 ceph-mon[26311]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:49:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:28 smithi078 ceph-mon[33746]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:49:30.172 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:29 smithi077 ceph-mon[26311]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:29 smithi078 ceph-mon[33746]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:30.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.529+0000 7fb2cb352700 1 -- 172.21.15.77:0/1805366978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fdde0 msgr2=0x7fb2c40fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:30.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 --2- 172.21.15.77:0/1805366978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fdde0 0x7fb2c40fe1b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0004940 tx=0x7fb2c0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:30.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- 172.21.15.77:0/1805366978 shutdown_connections 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 --2- 172.21.15.77:0/1805366978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fe780 0x7fb2c4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 --2- 172.21.15.77:0/1805366978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fdde0 0x7fb2c40fe1b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- 172.21.15.77:0/1805366978 >> 172.21.15.77:0/1805366978 conn(0x7fb2c40f7860 msgr2=0x7fb2c40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- 172.21.15.77:0/1805366978 shutdown_connections 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- 172.21.15.77:0/1805366978 wait complete. 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 Processor -- start 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- start start 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fdde0 0x7fb2c4113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c4114570 con 0x7fb2c40fe780 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2cb352700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c410dae0 con 0x7fb2c40fdde0 2024-04-26T04:49:30.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.530+0000 7fb2c88ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43136/0 (socket says 172.21.15.77:43136) 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 -- 172.21.15.77:0/3919474420 learned_addr learned my addr 172.21.15.77:0/3919474420 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c90ee700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fdde0 0x7fb2c4113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 -- 172.21.15.77:0/3919474420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fdde0 msgr2=0x7fb2c4113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fdde0 0x7fb2c4113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 -- 172.21.15.77:0/3919474420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0005040 con 0x7fb2c40fe780 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2c88ed700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400b790 tx=0x7fb2b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b400f6f0 con 0x7fb2c40fe780 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c410ddc0 con 0x7fb2c40fe780 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2b400fd20 con 0x7fb2c40fe780 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b40103a0 con 0x7fb2c40fe780 2024-04-26T04:49:30.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.531+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c410e300 con 0x7fb2c40fe780 2024-04-26T04:49:30.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.532+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c4101810 con 0x7fb2c40fe780 2024-04-26T04:49:30.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.533+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb2b4011400 con 0x7fb2c40fe780 2024-04-26T04:49:30.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.533+0000 7fb2b9ffb700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 0x7fb2b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:30.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.533+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb2b4086c40 con 0x7fb2c40fe780 2024-04-26T04:49:30.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.533+0000 7fb2c90ee700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 0x7fb2b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:30.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.533+0000 7fb2c90ee700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 0x7fb2b006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0004f90 tx=0x7fb2c0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:30.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.537+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb2b4056d40 con 0x7fb2c40fe780 2024-04-26T04:49:30.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.718+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2c410efa0 con 0x7fb2b00692a0 2024-04-26T04:49:30.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.719+0000 7fb2b9ffb700 1 -- 172.21.15.77:0/3919474420 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27852 (secure 0 0 0) 0x7fb2c410efa0 con 0x7fb2b00692a0 2024-04-26T04:49:30.721 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 msgr2=0x7fb2b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 0x7fb2b006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0004f90 tx=0x7fb2c0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 msgr2=0x7fb2c4113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400b790 tx=0x7fb2b4009250 comp rx=0 tx=0).stop 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 shutdown_connections 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb2b00692a0 0x7fb2b006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c40fdde0 0x7fb2c4113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 --2- 172.21.15.77:0/3919474420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2c40fe780 0x7fb2c4113e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 >> 172.21.15.77:0/3919474420 conn(0x7fb2c40f7860 msgr2=0x7fb2c40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 shutdown_connections 2024-04-26T04:49:30.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:30.722+0000 7fb2cb352700 1 -- 172.21.15.77:0/3919474420 wait complete. 2024-04-26T04:49:30.724 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-04-26T04:49:31.505 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-26T04:49:30.473054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788080,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682832,"statfs":{"total":575970213888,"available":574139219968,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002452"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-26T04:48:34.479000+0000","last_change":"2024-04-26T04:48:09.390489+0000","last_active":"2024-04-26T04:48:34.479000+0000","last_peered":"2024-04-26T04:48:34.479000+0000","last_clean":"2024-04-26T04:48:34.479000+0000","last_became_active":"2024-04-26T04:48:09.390223+0000","last_became_peered":"2024-04-26T04:48:09.390223+0000","last_unstale":"2024-04-26T04:48:34.479000+0000","last_undegraded":"2024-04-26T04:48:34.479000+0000","last_fullsized":"2024-04-26T04:48:34.479000+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:47:23.007192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:47:23.007192+0000","last_clean_scrub_stamp":"2024-04-26T04:47:23.007192+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:20.343766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Apr 26 04:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":3,"up_from":27,"seq":115964117017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Apr 26 04:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.35699999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.41199999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.377,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Apr 26 04:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.374,"15min":0.38600000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.379,"5min":0.498,"15min":0.498},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.39200000000000002,"15min":0.40300000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.40400000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Apr 26 04:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48699999999999999}]},{"osd":4,"last update":"Fri Apr 26 04:49:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.432},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 26 04:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32300000000000001,"15min":0.32000000000000001},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.50800000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Apr 26 04:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.34699999999999998,"15min":0.35499999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38,"15min":0.38700000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.51500000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Apr 26 04:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Apr 26 04:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:48:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.318,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.255},{"interface":"front","average":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.255,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Apr 26 04:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47499999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.219},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.48499999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27900000000000003}]},{"osd":3,"last update":"Fri Apr 26 04:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Apr 26 04:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:49:31.506 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T04:49:31.506 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T04:49:31.506 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T04:49:31.506 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T04:49:31.759 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:32.022 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:31 smithi077 ceph-mon[26311]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:32.022 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:31 smithi077 ceph-mon[26311]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:49:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:31 smithi078 ceph-mon[33746]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:31 smithi078 ceph-mon[33746]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:49:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- 172.21.15.77:0/1932098858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 msgr2=0x7f0c601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 --2- 172.21.15.77:0/1932098858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c601031b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c58004940 tx=0x7f0c58009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- 172.21.15.77:0/1932098858 shutdown_connections 2024-04-26T04:49:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 --2- 172.21.15.77:0/1932098858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 0x7f0c600fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 --2- 172.21.15.77:0/1932098858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c601031b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- 172.21.15.77:0/1932098858 >> 172.21.15.77:0/1932098858 conn(0x7f0c600f8990 msgr2=0x7f0c600fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- 172.21.15.77:0/1932098858 shutdown_connections 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- 172.21.15.77:0/1932098858 wait complete. 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 Processor -- start 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.342+0000 7f0c64959700 1 -- start start 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c64959700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c64959700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 0x7f0c60113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c64959700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60114550 con 0x7f0c60102de0 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c64959700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6010dac0 con 0x7f0c60103780 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43158/0 (socket says 172.21.15.77:43158) 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 -- 172.21.15.77:0/1289639331 learned_addr learned my addr 172.21.15.77:0/1289639331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5dd9b700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 0x7f0c60113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 -- 172.21.15.77:0/1289639331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 msgr2=0x7f0c60113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 0x7f0c60113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 -- 172.21.15.77:0/1289639331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58005040 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c5e59c700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800b3b0 tx=0x7f0c58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c5800e040 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c58013420 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c5801c630 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.343+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6010dd40 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.344+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6010e300 con 0x7f0c60102de0 2024-04-26T04:49:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.344+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c601000f0 con 0x7f0c60102de0 2024-04-26T04:49:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.345+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0c58013590 con 0x7f0c60102de0 2024-04-26T04:49:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.345+0000 7f0c4effd700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 0x7f0c4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.345+0000 7f0c5dd9b700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 0x7f0c4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.345+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0c58088040 con 0x7f0c60102de0 2024-04-26T04:49:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.346+0000 7f0c5dd9b700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 0x7f0c4806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c6010ec80 tx=0x7f0c54009780 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.350+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c58058140 con 0x7f0c60102de0 2024-04-26T04:49:33.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.578+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0c6010ef80 con 0x7f0c60102de0 2024-04-26T04:49:33.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.579+0000 7f0c4effd700 1 -- 172.21.15.77:0/1289639331 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0c58057cd0 con 0x7f0c60102de0 2024-04-26T04:49:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T04:49:33.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 msgr2=0x7f0c4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 0x7f0c4806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c6010ec80 tx=0x7f0c54009780 comp rx=0 tx=0).stop 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 msgr2=0x7f0c60113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800b3b0 tx=0x7f0c58009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 shutdown_connections 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0c480692a0 0x7f0c4806b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c60102de0 0x7f0c60113930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 --2- 172.21.15.77:0/1289639331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c60103780 0x7f0c60113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 >> 172.21.15.77:0/1289639331 conn(0x7f0c600f8990 msgr2=0x7f0c600fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 shutdown_connections 2024-04-26T04:49:33.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:33.581+0000 7f0c64959700 1 -- 172.21.15.77:0/1289639331 wait complete. 2024-04-26T04:49:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:33 smithi077 ceph-mon[26311]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:34.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:33 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1289639331' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T04:49:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:33 smithi078 ceph-mon[33746]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:33 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1289639331' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T04:49:34.294 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T04:49:34.294 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T04:49:34.295 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:49:34.304 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-26T04:49:34.304 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:49:34.333 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:49:34.333 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-26T04:49:34.595 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:36.215 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:35 smithi077 ceph-mon[26311]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:36.215 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:35 smithi078 ceph-mon[33746]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 -- 172.21.15.77:0/2881352849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 msgr2=0x7f93580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 --2- 172.21.15.77:0/2881352849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580fe440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f934c004940 tx=0x7f934c009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 -- 172.21.15.77:0/2881352849 shutdown_connections 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 --2- 172.21.15.77:0/2881352849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580fe440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 --2- 172.21.15.77:0/2881352849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 0x7f93580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 -- 172.21.15.77:0/2881352849 >> 172.21.15.77:0/2881352849 conn(0x7f93580f8540 msgr2=0x7f93580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 -- 172.21.15.77:0/2881352849 shutdown_connections 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.462+0000 7f935d0ab700 1 -- 172.21.15.77:0/2881352849 wait complete. 2024-04-26T04:49:36.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 Processor -- start 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 -- start start 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 0x7f9358071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358072200 con 0x7f93580fdfd0 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935d0ab700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580707f0 con 0x7f93580fce60 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43166/0 (socket says 172.21.15.77:43166) 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 -- 172.21.15.77:0/1056867091 learned_addr learned my addr 172.21.15.77:0/1056867091 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f9356d9d700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 0x7f9358071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 -- 172.21.15.77:0/1056867091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 msgr2=0x7f9358071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 0x7f9358071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.463+0000 7f935659c700 1 -- 172.21.15.77:0/1056867091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c005040 con 0x7f93580fdfd0 2024-04-26T04:49:36.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f935659c700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b3b0 tx=0x7f934c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:36.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c00e040 con 0x7f93580fdfd0 2024-04-26T04:49:36.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f934c004290 con 0x7f93580fdfd0 2024-04-26T04:49:36.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9358070a40 con 0x7f93580fdfd0 2024-04-26T04:49:36.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c01c3f0 con 0x7f93580fdfd0 2024-04-26T04:49:36.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.464+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9358071000 con 0x7f93580fdfd0 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.465+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935804c710 con 0x7f93580fdfd0 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.465+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f934c00bbf0 con 0x7f93580fdfd0 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.466+0000 7f933f7fe700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 0x7f934006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.466+0000 7f9356d9d700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 0x7f934006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.466+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f934c088160 con 0x7f93580fdfd0 2024-04-26T04:49:36.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.466+0000 7f9356d9d700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 0x7f934006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93480041d0 tx=0x7f93480043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.470+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f934c058260 con 0x7f93580fdfd0 2024-04-26T04:49:36.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.655+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f93581058c0 con 0x7f93580fdfd0 2024-04-26T04:49:36.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.660+0000 7f933f7fe700 1 -- 172.21.15.77:0/1056867091 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f934c057df0 con 0x7f93580fdfd0 2024-04-26T04:49:36.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 msgr2=0x7f934006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:36.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 0x7f934006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93480041d0 tx=0x7f93480043c0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 msgr2=0x7f93580702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:36.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b3b0 tx=0x7f934c009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 shutdown_connections 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f93400692a0 0x7f934006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93580fce60 0x7f9358071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 --2- 172.21.15.77:0/1056867091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93580fdfd0 0x7f93580702b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 >> 172.21.15.77:0/1056867091 conn(0x7f93580f8540 msgr2=0x7f93580f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 shutdown_connections 2024-04-26T04:49:36.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:36.662+0000 7f935d0ab700 1 -- 172.21.15.77:0/1056867091 wait complete. 2024-04-26T04:49:37.039 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1056867091' entity='client.admin' 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:37.040 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:36 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1056867091' entity='client.admin' 2024-04-26T04:49:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:37.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:37.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:37.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:36 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:37.408 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:49:37.418 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-26T04:49:37.419 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:49:37.457 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:49:37.458 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T04:49:37.707 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:38.102 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:37 smithi077 ceph-mon[26311]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:37 smithi078 ceph-mon[33746]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:39.112 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 -- 172.21.15.77:0/1867889690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc066ef0 msgr2=0x7f6bbc0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/1867889690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc0ffa30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004940 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 -- 172.21.15.77:0/1867889690 shutdown_connections 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/1867889690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/1867889690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc0ffa30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.321+0000 7f6bc2c33700 1 -- 172.21.15.77:0/1867889690 >> 172.21.15.77:0/1867889690 conn(0x7f6bbc0f7480 msgr2=0x7f6bbc0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 -- 172.21.15.77:0/1867889690 shutdown_connections 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 -- 172.21.15.77:0/1867889690 wait complete. 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 Processor -- start 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 -- start start 2024-04-26T04:49:39.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc10e4d0 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.322+0000 7f6bc2c33700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc10e640 con 0x7f6bbc066ef0 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37872/0 (socket says 172.21.15.77:37872) 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 -- 172.21.15.77:0/556636631 learned_addr learned my addr 172.21.15.77:0/556636631 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bc09cf700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 -- 172.21.15.77:0/556636631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc066ef0 msgr2=0x7f6bbc10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 -- 172.21.15.77:0/556636631 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0005040 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bbbfff700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c810 tx=0x7f6ba8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba8012810 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc1130a0 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ba8012e40 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba800a370 con 0x7f6bbc0fff70 2024-04-26T04:49:39.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.323+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc113660 con 0x7f6bbc0fff70 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.324+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc04c710 con 0x7f6bbc0fff70 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.325+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6ba8011810 con 0x7f6bbc0fff70 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.325+0000 7f6bb97fa700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 0x7f6bac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.325+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6ba80872f0 con 0x7f6bbc0fff70 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.325+0000 7f6bc09cf700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 0x7f6bac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.325+0000 7f6bc09cf700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 0x7f6bac06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b970 tx=0x7f6bb0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:39.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.329+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6ba8057370 con 0x7f6bbc0fff70 2024-04-26T04:49:39.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.533+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bbc0f8f70 con 0x7f6bac0692a0 2024-04-26T04:49:39.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.534+0000 7f6bb97fa700 1 -- 172.21.15.77:0/556636631 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6bbc0f8f70 con 0x7f6bac0692a0 2024-04-26T04:49:39.536 INFO:teuthology.orchestra.run.smithi077.stdout:Backend: cephadm 2024-04-26T04:49:39.536 INFO:teuthology.orchestra.run.smithi077.stdout:Available: Yes 2024-04-26T04:49:39.536 INFO:teuthology.orchestra.run.smithi077.stdout:Paused: No 2024-04-26T04:49:39.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.536+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 msgr2=0x7f6bac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:39.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.536+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 0x7f6bac06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b970 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:39.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.536+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 msgr2=0x7f6bbc10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:39.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.536+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c810 tx=0x7f6ba8004470 comp rx=0 tx=0).stop 2024-04-26T04:49:39.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 shutdown_connections 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f6bac0692a0 0x7f6bac06b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bbc066ef0 0x7f6bbc10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 --2- 172.21.15.77:0/556636631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bbc0fff70 0x7f6bbc10ded0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 >> 172.21.15.77:0/556636631 conn(0x7f6bbc0f7480 msgr2=0x7f6bbc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 shutdown_connections 2024-04-26T04:49:39.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:39.537+0000 7f6bc2c33700 1 -- 172.21.15.77:0/556636631 wait complete. 2024-04-26T04:49:40.212 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:39 smithi077 ceph-mon[26311]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:40.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T04:49:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:39 smithi078 ceph-mon[33746]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:40 smithi077 ceph-mon[26311]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:40 smithi078 ceph-mon[33746]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:42.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 -- 172.21.15.77:0/2487177702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fbcb0 msgr2=0x7fd9540fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:42.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/2487177702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fbcb0 0x7fd9540fc0c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd948004940 tx=0x7fd948009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:42.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 -- 172.21.15.77:0/2487177702 shutdown_connections 2024-04-26T04:49:42.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/2487177702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fcf60 0x7fd9540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/2487177702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fbcb0 0x7fd9540fc0c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 -- 172.21.15.77:0/2487177702 >> 172.21.15.77:0/2487177702 conn(0x7fd9540f7430 msgr2=0x7fd9540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:42.104 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:41 smithi077 ceph-mon[26311]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.101+0000 7fd95a7f4700 1 -- 172.21.15.77:0/2487177702 shutdown_connections 2024-04-26T04:49:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.102+0000 7fd95a7f4700 1 -- 172.21.15.77:0/2487177702 wait complete. 2024-04-26T04:49:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.102+0000 7fd95a7f4700 1 Processor -- start 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.102+0000 7fd95a7f4700 1 -- start start 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.102+0000 7fd95a7f4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fbcb0 0x7fd954110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd95a7f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd95a7f4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954066330 con 0x7fd9540fcf60 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd95a7f4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954066470 con 0x7fd9540fbcb0 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37890/0 (socket says 172.21.15.77:37890) 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 -- 172.21.15.77:0/1260089897 learned_addr learned my addr 172.21.15.77:0/1260089897 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd953fff700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fbcb0 0x7fd954110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 -- 172.21.15.77:0/1260089897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fbcb0 msgr2=0x7fd954110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fbcb0 0x7fd954110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 -- 172.21.15.77:0/1260089897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948005040 con 0x7fd9540fcf60 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd9537fe700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c8c0 tx=0x7fd9440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd94400a980 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954113210 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd94400ce60 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9440115e0 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.103+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9541137d0 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.104+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95405f5c0 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.105+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd944012400 con 0x7fd9540fcf60 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.105+0000 7fd950ff9700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 0x7fd93c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.105+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd944086fc0 con 0x7fd9540fcf60 2024-04-26T04:49:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.105+0000 7fd953fff700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 0x7fd93c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.105+0000 7fd953fff700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 0x7fd93c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b970 tx=0x7fd948009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:42.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.109+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd9440535d0 con 0x7fd9540fcf60 2024-04-26T04:49:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:41 smithi078 ceph-mon[33746]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:42.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.307+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd954101320 con 0x7fd93c0692a0 2024-04-26T04:49:42.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.315+0000 7fd950ff9700 1 -- 172.21.15.77:0/1260089897 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2700 (secure 0 0 0) 0x7fd954101320 con 0x7fd93c0692a0 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (3m) 2m ago 5m 26.6M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (5m) 2m ago 5m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (4m) 70s ago 4m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (3m) 2m ago 4m 80.6M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (6m) 2m ago 6m 470M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (4m) 70s ago 4m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 7b0ffd125c63 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (6m) 2m ago 6m 40.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (4m) 70s ago 4m 38.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (5m) 2m ago 5m 14.3M - 1.5.0 0da6a335fe13 e44a82d54ec7 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (4m) 70s ago 4m 16.3M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (3m) 2m ago 3m 34.8M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:49:42.317 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (2m) 2m ago 2m 34.6M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:49:42.318 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (2m) 2m ago 2m 35.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:49:42.318 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (2m) 70s ago 2m 40.4M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:49:42.318 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (100s) 70s ago 99s 37.3M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:49:42.318 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (75s) 70s ago 74s 13.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:49:42.318 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (3m) 2m ago 4m 38.2M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 msgr2=0x7fd93c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 0x7fd93c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd94800b970 tx=0x7fd948009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 msgr2=0x7fd954112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c8c0 tx=0x7fd9440044c0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 shutdown_connections 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd93c0692a0 0x7fd93c06b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9540fbcb0 0x7fd954110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 --2- 172.21.15.77:0/1260089897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9540fcf60 0x7fd954112c70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:42.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 >> 172.21.15.77:0/1260089897 conn(0x7fd9540f7430 msgr2=0x7fd9540f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:42.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 shutdown_connections 2024-04-26T04:49:42.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:42.317+0000 7fd95a7f4700 1 -- 172.21.15.77:0/1260089897 wait complete. 2024-04-26T04:49:42.891 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T04:49:43.141 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:43 smithi077 ceph-mon[26311]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:43 smithi077 ceph-mon[26311]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:43 smithi078 ceph-mon[33746]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:43 smithi078 ceph-mon[33746]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:44.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 -- 172.21.15.77:0/790205573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 msgr2=0x7fb9340fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:44.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/790205573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9340fc0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb924004940 tx=0x7fb924009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 -- 172.21.15.77:0/790205573 shutdown_connections 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/790205573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 0x7fb9340fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/790205573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9340fc0c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 -- 172.21.15.77:0/790205573 >> 172.21.15.77:0/790205573 conn(0x7fb9340f7430 msgr2=0x7fb9340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 -- 172.21.15.77:0/790205573 shutdown_connections 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.798+0000 7fb93c4f4700 1 -- 172.21.15.77:0/790205573 wait complete. 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 Processor -- start 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 -- start start 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 0x7fb934112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb934066390 con 0x7fb9340fbcb0 2024-04-26T04:49:44.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93c4f4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340664d0 con 0x7fb9340fcf60 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37914/0 (socket says 172.21.15.77:37914) 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 -- 172.21.15.77:0/3836539568 learned_addr learned my addr 172.21.15.77:0/3836539568 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb939a8f700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 0x7fb934112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 -- 172.21.15.77:0/3836539568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 msgr2=0x7fb934112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 0x7fb934112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 -- 172.21.15.77:0/3836539568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924005040 con 0x7fb9340fbcb0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb93a290700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb92400b3b0 tx=0x7fb924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.799+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92400e040 con 0x7fb9340fbcb0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.800+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb92400bcc0 con 0x7fb9340fbcb0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.800+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9341131b0 con 0x7fb9340fbcb0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.800+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92401c630 con 0x7fb9340fbcb0 2024-04-26T04:49:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.800+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb934113770 con 0x7fb9340fbcb0 2024-04-26T04:49:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.800+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93405f5c0 con 0x7fb9340fbcb0 2024-04-26T04:49:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.801+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb924013980 con 0x7fb9340fbcb0 2024-04-26T04:49:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.801+0000 7fb92affd700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 0x7fb92006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.801+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb924087bb0 con 0x7fb9340fbcb0 2024-04-26T04:49:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.801+0000 7fb939a8f700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 0x7fb92006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:44.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.802+0000 7fb939a8f700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 0x7fb92006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb930004990 tx=0x7fb930009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:44.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:44.806+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb92408e080 con 0x7fb9340fbcb0 2024-04-26T04:49:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.002+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb934101320 con 0x7fb9200692a0 2024-04-26T04:49:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.006+0000 7fb92affd700 1 -- 172.21.15.77:0/3836539568 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7fb934101320 con 0x7fb9200692a0 2024-04-26T04:49:45.007 INFO:teuthology.orchestra.run.smithi077.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:crash 2/2 2m ago 5m * 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:mgr 2/2 2m ago 5m count:2 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:mon 2/2 2m ago 5m smithi077:172.21.15.77=smithi077;smithi078:172.21.15.78=smithi078;count:2 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:osd 6 2m ago - 2024-04-26T04:49:45.008 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 msgr2=0x7fb92006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 0x7fb92006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb930004990 tx=0x7fb930009ee0 comp rx=0 tx=0).stop 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 msgr2=0x7fb9341106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb92400b3b0 tx=0x7fb924009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 shutdown_connections 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fb9200692a0 0x7fb92006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9340fbcb0 0x7fb9341106c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:45.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 --2- 172.21.15.77:0/3836539568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9340fcf60 0x7fb934112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:45.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 >> 172.21.15.77:0/3836539568 conn(0x7fb9340f7430 msgr2=0x7fb9340f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:45.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 shutdown_connections 2024-04-26T04:49:45.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:45.008+0000 7fb93c4f4700 1 -- 172.21.15.77:0/3836539568 wait complete. 2024-04-26T04:49:45.629 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T04:49:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:46.232 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:45 smithi077 ceph-mon[26311]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:46.232 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:45 smithi077 ceph-mon[26311]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:45 smithi078 ceph-mon[33746]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:45 smithi078 ceph-mon[33746]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:47.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- 172.21.15.77:0/1848877856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c066fb0 msgr2=0x7fd51c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:47.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 --2- 172.21.15.77:0/1848877856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c066fb0 0x7fd51c0ff9b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd518004940 tx=0x7fd518009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:47.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- 172.21.15.77:0/1848877856 shutdown_connections 2024-04-26T04:49:47.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 --2- 172.21.15.77:0/1848877856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c0ffef0 0x7fd51c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 --2- 172.21.15.77:0/1848877856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c066fb0 0x7fd51c0ff9b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- 172.21.15.77:0/1848877856 >> 172.21.15.77:0/1848877856 conn(0x7fd51c0f7420 msgr2=0x7fd51c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- 172.21.15.77:0/1848877856 shutdown_connections 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- 172.21.15.77:0/1848877856 wait complete. 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 Processor -- start 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- start start 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c066fb0 0x7fd51c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c10e4d0 con 0x7fd51c0ffef0 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.528+0000 7fd523519700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c10e640 con 0x7fd51c066fb0 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49270/0 (socket says 172.21.15.77:49270) 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 -- 172.21.15.77:0/1750899869 learned_addr learned my addr 172.21.15.77:0/1750899869 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd5212b5700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c066fb0 0x7fd51c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 -- 172.21.15.77:0/1750899869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c066fb0 msgr2=0x7fd51c10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c066fb0 0x7fd51c10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 -- 172.21.15.77:0/1750899869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518005040 con 0x7fd51c0ffef0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd520ab4700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd50c002a50 tx=0x7fd50c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50c007cc0 con 0x7fd51c0ffef0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd50c00f040 con 0x7fd51c0ffef0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c113050 con 0x7fd51c0ffef0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.529+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50c00da30 con 0x7fd51c0ffef0 2024-04-26T04:49:47.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.530+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c1135e0 con 0x7fd51c0ffef0 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.531+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c04c710 con 0x7fd51c0ffef0 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.531+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd50c00db90 con 0x7fd51c0ffef0 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.531+0000 7fd511ffb700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 0x7fd50806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.531+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd50c086c10 con 0x7fd51c0ffef0 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.531+0000 7fd5212b5700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 0x7fd50806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:47.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.532+0000 7fd5212b5700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 0x7fd50806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd518004890 tx=0x7fd518009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:47.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.536+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd50c056c90 con 0x7fd51c0ffef0 2024-04-26T04:49:47.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.730+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd51c0f8f60 con 0x7fd5080692a0 2024-04-26T04:49:47.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.731+0000 7fd511ffb700 1 -- 172.21.15.77:0/1750899869 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7fd51c0f8f60 con 0x7fd5080692a0 2024-04-26T04:49:47.733 INFO:teuthology.orchestra.run.smithi077.stdout:HOST ADDR LABELS STATUS 2024-04-26T04:49:47.733 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 172.21.15.77 2024-04-26T04:49:47.733 INFO:teuthology.orchestra.run.smithi077.stdout:smithi078 172.21.15.78 2024-04-26T04:49:47.733 INFO:teuthology.orchestra.run.smithi077.stdout:2 hosts in cluster 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 msgr2=0x7fd50806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 0x7fd50806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd518004890 tx=0x7fd518009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 msgr2=0x7fd51c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd50c002a50 tx=0x7fd50c0046b0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 shutdown_connections 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fd5080692a0 0x7fd50806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd51c066fb0 0x7fd51c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 --2- 172.21.15.77:0/1750899869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd51c0ffef0 0x7fd51c10ded0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:47.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 >> 172.21.15.77:0/1750899869 conn(0x7fd51c0f7420 msgr2=0x7fd51c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:47.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 shutdown_connections 2024-04-26T04:49:47.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:47.733+0000 7fd523519700 1 -- 172.21.15.77:0/1750899869 wait complete. 2024-04-26T04:49:48.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:47 smithi077 ceph-mon[26311]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:47 smithi078 ceph-mon[33746]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:48.411 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T04:49:48.664 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:49.112 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:48 smithi077 ceph-mon[26311]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:48 smithi078 ceph-mon[33746]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:50.178 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:49 smithi077 ceph-mon[26311]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:49 smithi078 ceph-mon[33746]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 -- 172.21.15.77:0/1907699463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fce60 msgr2=0x7feed80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 --2- 172.21.15.77:0/1907699463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fce60 0x7feed80fd270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feec8004940 tx=0x7feec8009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 -- 172.21.15.77:0/1907699463 shutdown_connections 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 --2- 172.21.15.77:0/1907699463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fdfd0 0x7feed80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 --2- 172.21.15.77:0/1907699463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fce60 0x7feed80fd270 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 -- 172.21.15.77:0/1907699463 >> 172.21.15.77:0/1907699463 conn(0x7feed80f8560 msgr2=0x7feed80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 -- 172.21.15.77:0/1907699463 shutdown_connections 2024-04-26T04:49:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 -- 172.21.15.77:0/1907699463 wait complete. 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.280+0000 7feede57c700 1 Processor -- start 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 -- start start 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fce60 0x7feed810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed81107f0 con 0x7feed80fdfd0 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8110fe0 con 0x7feed80fce60 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49284/0 (socket says 172.21.15.77:49284) 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 -- 172.21.15.77:0/267556935 learned_addr learned my addr 172.21.15.77:0/267556935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed7fff700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fce60 0x7feed810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 -- 172.21.15.77:0/267556935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fce60 msgr2=0x7feed810fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fce60 0x7feed810fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 -- 172.21.15.77:0/267556935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8005040 con 0x7feed80fdfd0 2024-04-26T04:49:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed77fe700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b790 tx=0x7feecc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:50.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feecc00f6f0 con 0x7feed80fdfd0 2024-04-26T04:49:50.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feecc00fd20 con 0x7feed80fdfd0 2024-04-26T04:49:50.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.281+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed81112c0 con 0x7feed80fdfd0 2024-04-26T04:49:50.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.282+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feecc0103a0 con 0x7feed80fdfd0 2024-04-26T04:49:50.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.282+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8111880 con 0x7feed80fdfd0 2024-04-26T04:49:50.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.282+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed8101960 con 0x7feed80fdfd0 2024-04-26T04:49:50.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.283+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feecc011400 con 0x7feed80fdfd0 2024-04-26T04:49:50.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.283+0000 7feed4ff9700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 0x7feec006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:50.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.284+0000 7feed7fff700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 0x7feec006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:50.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.284+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feecc086b60 con 0x7feed80fdfd0 2024-04-26T04:49:50.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.284+0000 7feed7fff700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 0x7feec006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feec800b970 tx=0x7feec8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:50.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.288+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feecc056d00 con 0x7feed80fdfd0 2024-04-26T04:49:50.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.479+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feed8105a50 con 0x7feec00692a0 2024-04-26T04:49:50.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.481+0000 7feed4ff9700 1 -- 172.21.15.77:0/267556935 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7feed8105a50 con 0x7feec00692a0 2024-04-26T04:49:50.483 INFO:teuthology.orchestra.run.smithi077.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T04:49:50.483 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-26T04:49:50.483 INFO:teuthology.orchestra.run.smithi077.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 75s ago Has a FileSystem, LVM detected 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 msgr2=0x7feec006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 0x7feec006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feec800b970 tx=0x7feec8009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 msgr2=0x7feed8110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b790 tx=0x7feecc009250 comp rx=0 tx=0).stop 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 shutdown_connections 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7feec00692a0 0x7feec006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed80fce60 0x7feed810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 --2- 172.21.15.77:0/267556935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feed80fdfd0 0x7feed8110160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:50.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 >> 172.21.15.77:0/267556935 conn(0x7feed80f8560 msgr2=0x7feed80f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:50.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 shutdown_connections 2024-04-26T04:49:50.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:50.484+0000 7feede57c700 1 -- 172.21.15.77:0/267556935 wait complete. 2024-04-26T04:49:51.056 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:49:51.066 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:49:51.066 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-26T04:49:51.331 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:51 smithi077 ceph-mon[26311]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:52 smithi077 ceph-mon[26311]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:51 smithi078 ceph-mon[33746]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:51 smithi078 ceph-mon[33746]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:53.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 -- 172.21.15.77:0/521677353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 msgr2=0x7f64b8104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:53.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 --2- 172.21.15.77:0/521677353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8104720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f64ac004940 tx=0x7f64ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:53.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 -- 172.21.15.77:0/521677353 shutdown_connections 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 --2- 172.21.15.77:0/521677353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8104720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 --2- 172.21.15.77:0/521677353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 0x7f64b8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 -- 172.21.15.77:0/521677353 >> 172.21.15.77:0/521677353 conn(0x7f64b80f7430 msgr2=0x7f64b80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 -- 172.21.15.77:0/521677353 shutdown_connections 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.074+0000 7f64be0dd700 1 -- 172.21.15.77:0/521677353 wait complete. 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 Processor -- start 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 -- start start 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 0x7f64b810fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8110750 con 0x7f64b8102340 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64be0dd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8110fa0 con 0x7f64b8066260 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:53.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49306/0 (socket says 172.21.15.77:49306) 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 -- 172.21.15.77:0/3974199036 learned_addr learned my addr 172.21.15.77:0/3974199036 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b77fe700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 0x7f64b810fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 -- 172.21.15.77:0/3974199036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 msgr2=0x7f64b810fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 0x7f64b810fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 -- 172.21.15.77:0/3974199036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac005040 con 0x7f64b8102340 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.075+0000 7f64b6ffd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00b3b0 tx=0x7f64ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.076+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64ac00e040 con 0x7f64b8102340 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.076+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64ac00bcc0 con 0x7f64b8102340 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.076+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b8111220 con 0x7f64b8102340 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.076+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64ac01c630 con 0x7f64b8102340 2024-04-26T04:49:53.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.076+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b8111760 con 0x7f64b8102340 2024-04-26T04:49:53.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.077+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b805f5c0 con 0x7f64b8102340 2024-04-26T04:49:53.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.077+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f64ac013980 con 0x7f64b8102340 2024-04-26T04:49:53.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.077+0000 7f649ffff700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 0x7f64a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:53.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.077+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f64ac087bd0 con 0x7f64b8102340 2024-04-26T04:49:53.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.077+0000 7f64b77fe700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 0x7f64a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:53.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.078+0000 7f64b77fe700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 0x7f64a006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007fd0 tx=0x7f64a8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:53.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.082+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f64ac057cd0 con 0x7f64b8102340 2024-04-26T04:49:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:53.298+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f64b8111ae0 con 0x7f64a00692a0 2024-04-26T04:49:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:53 smithi077 ceph-mon[26311]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-26T04:49:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:53 smithi078 ceph-mon[33746]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-26T04:49:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:55.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.021+0000 7f649ffff700 1 -- 172.21.15.77:0/3974199036 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f64b8111ae0 con 0x7f64a00692a0 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 msgr2=0x7f64a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 0x7f64a006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007fd0 tx=0x7f64a8009040 comp rx=0 tx=0).stop 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 msgr2=0x7f64b8110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00b3b0 tx=0x7f64ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 shutdown_connections 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f64a00692a0 0x7f64a006b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b8066260 0x7f64b810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 --2- 172.21.15.77:0/3974199036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64b8102340 0x7f64b8110150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 >> 172.21.15.77:0/3974199036 conn(0x7f64b80f7430 msgr2=0x7f64b80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 shutdown_connections 2024-04-26T04:49:55.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:55.025+0000 7f64be0dd700 1 -- 172.21.15.77:0/3974199036 wait complete. 2024-04-26T04:49:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-26T04:49:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T04:49:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:49:55.378 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:54 smithi077 ceph-mon[26311]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:55.378 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-26T04:49:55.378 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:54 smithi077 ceph-mon[26311]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T04:49:55.378 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:54 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:49:55.378 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:55.002+0000 7ff93db88700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:49:55.711 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: pgmap v173: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-26T04:49:56.256 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: fsmap cephfs:0 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: Saving service mds.cephfs spec with placement count:4 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:49:56.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:55 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: pgmap v173: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T04:49:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-26T04:49:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T04:49:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: fsmap cephfs:0 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: Saving service mds.cephfs spec with placement count:4 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:55 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:56 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:56 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:49:56.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:56 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:56.949 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:49:57.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:57 smithi077 ceph-mon[26311]: Deploying daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:49:57.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:57 smithi077 ceph-mon[26311]: osdmap e45: 6 total, 6 up, 6 in 2024-04-26T04:49:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:57 smithi078 ceph-mon[33746]: Deploying daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:49:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:57 smithi078 ceph-mon[33746]: osdmap e45: 6 total, 6 up, 6 in 2024-04-26T04:49:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:58 smithi078 ceph-mon[33746]: pgmap v177: 65 pgs: 18 active+clean, 27 creating+peering, 20 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:58 smithi078 ceph-mon[33746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T04:49:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:58 smithi078 ceph-mon[33746]: osdmap e46: 6 total, 6 up, 6 in 2024-04-26T04:49:58.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:58 smithi077 ceph-mon[26311]: pgmap v177: 65 pgs: 18 active+clean, 27 creating+peering, 20 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:58.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:58 smithi077 ceph-mon[26311]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T04:49:58.312 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:58 smithi077 ceph-mon[26311]: osdmap e46: 6 total, 6 up, 6 in 2024-04-26T04:49:58.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.699+0000 7fa926eb5700 1 -- 172.21.15.77:0/897849229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fbce0 msgr2=0x7fa9200fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.699+0000 7fa926eb5700 1 --2- 172.21.15.77:0/897849229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fbce0 0x7fa9200fc0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa914004940 tx=0x7fa914009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- 172.21.15.77:0/897849229 shutdown_connections 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 --2- 172.21.15.77:0/897849229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fcf90 0x7fa9200fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 --2- 172.21.15.77:0/897849229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fbce0 0x7fa9200fc0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- 172.21.15.77:0/897849229 >> 172.21.15.77:0/897849229 conn(0x7fa9200f7480 msgr2=0x7fa9200f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- 172.21.15.77:0/897849229 shutdown_connections 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- 172.21.15.77:0/897849229 wait complete. 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 Processor -- start 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- start start 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 0x7fa920066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920112c70 con 0x7fa9200fcf90 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.700+0000 7fa926eb5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920112db0 con 0x7fa9200fbce0 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49242/0 (socket says 172.21.15.77:49242) 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 -- 172.21.15.77:0/3590313310 learned_addr learned my addr 172.21.15.77:0/3590313310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa924c51700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 0x7fa920066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 -- 172.21.15.77:0/3590313310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 msgr2=0x7fa920066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 0x7fa920066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 -- 172.21.15.77:0/3590313310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914005040 con 0x7fa9200fcf90 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa924c51700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 0x7fa920066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91ffff700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c830 tx=0x7fa90c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90c00a760 con 0x7fa9200fcf90 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa90c00ad90 con 0x7fa9200fcf90 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa920113010 con 0x7fa9200fcf90 2024-04-26T04:49:58.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90c0113a0 con 0x7fa9200fcf90 2024-04-26T04:49:58.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.701+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9201135a0 con 0x7fa9200fcf90 2024-04-26T04:49:58.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.702+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92004c710 con 0x7fa9200fcf90 2024-04-26T04:49:58.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.703+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa90c012400 con 0x7fa9200fcf90 2024-04-26T04:49:58.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.703+0000 7fa91d7fa700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 0x7fa91006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.703+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa90c087200 con 0x7fa9200fcf90 2024-04-26T04:49:58.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.703+0000 7fa924c51700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 0x7fa91006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.703+0000 7fa924c51700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 0x7fa91006b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b7a0 tx=0x7fa914009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.707+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa90c056fc0 con 0x7fa9200fcf90 2024-04-26T04:49:58.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.943+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa9201047d0 con 0x7fa9200fcf90 2024-04-26T04:49:58.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.944+0000 7fa91d7fa700 1 -- 172.21.15.77:0/3590313310 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fa90c056b50 con 0x7fa9200fcf90 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:e2 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 2 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:49:58.946 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:49:55.003744+0000 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:in 2024-04-26T04:49:58.947 INFO:teuthology.orchestra.run.smithi077.stdout:up {} 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data disabled 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 0 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:58.948 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 msgr2=0x7fa91006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 0x7fa91006b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa91400b7a0 tx=0x7fa914009e30 comp rx=0 tx=0).stop 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 msgr2=0x7fa920112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00c830 tx=0x7fa90c004470 comp rx=0 tx=0).stop 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 shutdown_connections 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fa9100692a0 0x7fa91006b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9200fbce0 0x7fa920066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 --2- 172.21.15.77:0/3590313310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9200fcf90 0x7fa920112730 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 >> 172.21.15.77:0/3590313310 conn(0x7fa9200f7480 msgr2=0x7fa9200f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 shutdown_connections 2024-04-26T04:49:58.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:49:58.946+0000 7fa926eb5700 1 -- 172.21.15.77:0/3590313310 wait complete. 2024-04-26T04:49:58.950 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 2 2024-04-26T04:49:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:49:59 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3590313310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:49:59 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3590313310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:59.930 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:49:59.940 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:49:59.940 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-26T04:50:00.206 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: pgmap v179: 65 pgs: 27 active+clean, 27 creating+peering, 11 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:00.210 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: fs cephfs is offline because no MDS is active for it. 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.211 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:00 smithi078 ceph-mon[33746]: fs cephfs has 0 MDS online, but wants 1 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:59.998+0000 7ff94038d700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:59.998+0000 7ff94038d700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:59.998+0000 7ff94038d700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:59.998+0000 7ff94038d700 -1 log_channel(cluster) log [ERR] : [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:49:59.998+0000 7ff94038d700 -1 log_channel(cluster) log [ERR] : fs cephfs has 0 MDS online, but wants 1 2024-04-26T04:50:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: pgmap v179: 65 pgs: 27 active+clean, 27 creating+peering, 11 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: fs cephfs is offline because no MDS is active for it. 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-26T04:50:00.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:00 smithi077 ceph-mon[26311]: fs cephfs has 0 MDS online, but wants 1 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: Deploying daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:boot 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: daemon mds.cephfs.smithi077.quaiyg assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: Cluster is now healthy 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: fsmap cephfs:0 1 up:standby 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:50:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:creating} 2024-04-26T04:50:01.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:01 smithi078 ceph-mon[33746]: daemon mds.cephfs.smithi077.quaiyg is now active in filesystem cephfs as rank 0 2024-04-26T04:50:01.300 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: Deploying daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T04:50:01.300 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:boot 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: daemon mds.cephfs.smithi077.quaiyg assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: Cluster is now healthy 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: fsmap cephfs:0 1 up:standby 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:creating} 2024-04-26T04:50:01.301 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:01 smithi077 ceph-mon[26311]: daemon mds.cephfs.smithi077.quaiyg is now active in filesystem cephfs as rank 0 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 -- 172.21.15.77:0/305220917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 msgr2=0x7f891c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 --2- 172.21.15.77:0/305220917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c102300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f890c004940 tx=0x7f890c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 -- 172.21.15.77:0/305220917 shutdown_connections 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 --2- 172.21.15.77:0/305220917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c102300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 --2- 172.21.15.77:0/305220917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 0x7f891c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:01.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 -- 172.21.15.77:0/305220917 >> 172.21.15.77:0/305220917 conn(0x7f891c0f7430 msgr2=0x7f891c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 -- 172.21.15.77:0/305220917 shutdown_connections 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.827+0000 7f892178a700 1 -- 172.21.15.77:0/305220917 wait complete. 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 Processor -- start 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 -- start start 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 0x7f891c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c10e540 con 0x7f891c0ffee0 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f892178a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c10ed90 con 0x7f891c0662c0 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f891a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f891a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49286/0 (socket says 172.21.15.77:49286) 2024-04-26T04:50:01.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f891a7fc700 1 -- 172.21.15.77:0/1138167617 learned_addr learned my addr 172.21.15.77:0/1138167617 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.828+0000 7f891affd700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 0x7f891c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f891a7fc700 1 -- 172.21.15.77:0/1138167617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 msgr2=0x7f891c10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f891a7fc700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 0x7f891c10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f891a7fc700 1 -- 172.21.15.77:0/1138167617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c005040 con 0x7f891c0ffee0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f891a7fc700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890c00b3b0 tx=0x7f890c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c00e040 con 0x7f891c0ffee0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f890c00bbf0 con 0x7f891c0ffee0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c10f010 con 0x7f891c0ffee0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c01c630 con 0x7f891c0ffee0 2024-04-26T04:50:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.829+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c10f550 con 0x7f891c0ffee0 2024-04-26T04:50:01.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.830+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c05f5c0 con 0x7f891c0ffee0 2024-04-26T04:50:01.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.830+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f890c00bd60 con 0x7f891c0ffee0 2024-04-26T04:50:01.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.831+0000 7f89037fe700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 0x7f890406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:01.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.831+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f890c087e50 con 0x7f891c0ffee0 2024-04-26T04:50:01.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.831+0000 7f891affd700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 0x7f890406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:01.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.831+0000 7f891affd700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 0x7f890406b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8910007fd0 tx=0x7f8910009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:01.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:01.835+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f890c057c10 con 0x7f891c0ffee0 2024-04-26T04:50:02.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:02.089+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f891c04bed0 con 0x7f891c0ffee0 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: pgmap v180: 65 pgs: 57 active+clean, 8 creating+peering; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 746 B/s wr, 3 op/s 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:active 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:02.316 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:02 smithi077 ceph-mon[26311]: Deploying daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: pgmap v180: 65 pgs: 57 active+clean, 8 creating+peering; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 746 B/s wr, 3 op/s 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:active 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:02.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:02.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:02.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:02.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:02 smithi078 ceph-mon[33746]: Deploying daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T04:50:03.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.058+0000 7f89037fe700 1 -- 172.21.15.77:0/1138167617 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f891c04bed0 con 0x7f891c0ffee0 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 msgr2=0x7f890406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 0x7f890406b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8910007fd0 tx=0x7f8910009040 comp rx=0 tx=0).stop 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 msgr2=0x7f891c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f890c00b3b0 tx=0x7f890c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 shutdown_connections 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f89040692a0 0x7f890406b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891c0662c0 0x7f891c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 --2- 172.21.15.77:0/1138167617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f891c0ffee0 0x7f891c10df40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 >> 172.21.15.77:0/1138167617 conn(0x7f891c0f7430 msgr2=0x7f891c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:03.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 shutdown_connections 2024-04-26T04:50:03.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:03.060+0000 7f892178a700 1 -- 172.21.15.77:0/1138167617 wait complete. 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:boot 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1138167617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-26T04:50:03.176 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:03 smithi077 ceph-mon[26311]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 548 B/s wr, 4 op/s 2024-04-26T04:50:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:boot 2024-04-26T04:50:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:50:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1138167617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-26T04:50:03.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:03 smithi078 ceph-mon[33746]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 548 B/s wr, 4 op/s 2024-04-26T04:50:03.955 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:50:03.966 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:03.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-26T04:50:04.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:04 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1138167617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-26T04:50:04.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:04 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:04 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1138167617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-26T04:50:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:04 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 1 up:standby 2024-04-26T04:50:04.476 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: pgmap v182: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 725 B/s wr, 4 op/s 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:05.544 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:05.545 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: Deploying daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T04:50:05.545 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:boot 2024-04-26T04:50:05.545 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2 up:standby 2024-04-26T04:50:05.545 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:50:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: pgmap v182: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 725 B/s wr, 4 op/s 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: Deploying daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:boot 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2 up:standby 2024-04-26T04:50:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:50:06.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- 172.21.15.77:0/992620608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 msgr2=0x7fda700fe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:06.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 --2- 172.21.15.77:0/992620608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda700fe890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda60004940 tx=0x7fda60009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:06.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- 172.21.15.77:0/992620608 shutdown_connections 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 --2- 172.21.15.77:0/992620608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 0x7fda700faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 --2- 172.21.15.77:0/992620608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda700fe890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- 172.21.15.77:0/992620608 >> 172.21.15.77:0/992620608 conn(0x7fda700f6b80 msgr2=0x7fda700f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- 172.21.15.77:0/992620608 shutdown_connections 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- 172.21.15.77:0/992620608 wait complete. 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 Processor -- start 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 -- start start 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.008+0000 7fda74b07700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda74b07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 0x7fda7010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda74b07700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7010e4b0 con 0x7fda700fe480 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda74b07700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7010ed00 con 0x7fda700fedd0 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49322/0 (socket says 172.21.15.77:49322) 2024-04-26T04:50:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 -- 172.21.15.77:0/1748178681 learned_addr learned my addr 172.21.15.77:0/1748178681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6dd9b700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 0x7fda7010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 -- 172.21.15.77:0/1748178681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 msgr2=0x7fda7010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 0x7fda7010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 -- 172.21.15.77:0/1748178681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60005040 con 0x7fda700fe480 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda6e59c700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda60013870 tx=0x7fda60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda60021070 con 0x7fda700fe480 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7010ef80 con 0x7fda700fe480 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda60009430 con 0x7fda700fe480 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda6000e070 con 0x7fda700fe480 2024-04-26T04:50:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.009+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7010f4c0 con 0x7fda700fe480 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.010+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7005f5c0 con 0x7fda700fe480 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.011+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fda6001c750 con 0x7fda700fe480 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.011+0000 7fda5effd700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 0x7fda5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.011+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fda60088370 con 0x7fda700fe480 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.011+0000 7fda6dd9b700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 0x7fda5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.012+0000 7fda6dd9b700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 0x7fda5806b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda64009b80 tx=0x7fda6400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.016+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fda600581d0 con 0x7fda700fe480 2024-04-26T04:50:06.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.256+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fda7004bed0 con 0x7fda700fe480 2024-04-26T04:50:06.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.492+0000 7fda5effd700 1 -- 172.21.15.77:0/1748178681 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fda60004490 con 0x7fda700fe480 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.494+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 msgr2=0x7fda5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.494+0000 7fda74b07700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 0x7fda5806b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda64009b80 tx=0x7fda6400b040 comp rx=0 tx=0).stop 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.494+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 msgr2=0x7fda7010d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.494+0000 7fda74b07700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda60013870 tx=0x7fda60009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 shutdown_connections 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fda580692a0 0x7fda5806b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700fe480 0x7fda7010d970 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 --2- 172.21.15.77:0/1748178681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda700fedd0 0x7fda7010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 >> 172.21.15.77:0/1748178681 conn(0x7fda700f6b80 msgr2=0x7fda700f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 shutdown_connections 2024-04-26T04:50:06.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:06.495+0000 7fda74b07700 1 -- 172.21.15.77:0/1748178681 wait complete. 2024-04-26T04:50:06.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:06 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1748178681' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-26T04:50:06.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:06 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1748178681' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-26T04:50:07.173 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:50:07.184 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:07.184 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-26T04:50:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T04:50:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1748178681' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-26T04:50:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2 up:standby 2024-04-26T04:50:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:07 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1748178681' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 2 up:standby 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:07.788 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:07 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:08.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:08 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:boot 2024-04-26T04:50:08.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:08 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:08.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:50:08.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:08 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:boot 2024-04-26T04:50:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:08 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:50:08.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:09.571 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-26T04:50:09.571 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:active 2024-04-26T04:50:09.571 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:standby 2024-04-26T04:50:09.571 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:09.572 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.572 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.572 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:09 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] up:active 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:standby 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:09 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 -- 172.21.15.77:0/2504846208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 msgr2=0x7fe0300fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 --2- 172.21.15.77:0/2504846208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300fc0c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe020004940 tx=0x7fe020009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 -- 172.21.15.77:0/2504846208 shutdown_connections 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 --2- 172.21.15.77:0/2504846208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 0x7fe0300fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 --2- 172.21.15.77:0/2504846208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300fc0c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:09.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 -- 172.21.15.77:0/2504846208 >> 172.21.15.77:0/2504846208 conn(0x7fe0300f7430 msgr2=0x7fe0300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 -- 172.21.15.77:0/2504846208 shutdown_connections 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.796+0000 7fe036a12700 1 -- 172.21.15.77:0/2504846208 wait complete. 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 Processor -- start 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 -- start start 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 0x7fe0300fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300fe7e0 con 0x7fe0300fbcb0 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe036a12700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300fe920 con 0x7fe0300fcf60 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55074/0 (socket says 172.21.15.77:55074) 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 -- 172.21.15.77:0/3293459473 learned_addr learned my addr 172.21.15.77:0/3293459473 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:09.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02f7fe700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 0x7fe0300fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:09.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 -- 172.21.15.77:0/3293459473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 msgr2=0x7fe0300fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:09.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 0x7fe0300fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:09.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.797+0000 7fe02ffff700 1 -- 172.21.15.77:0/3293459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020005040 con 0x7fe0300fbcb0 2024-04-26T04:50:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.798+0000 7fe02ffff700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b3b0 tx=0x7fe020009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.798+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe02000e040 con 0x7fe0300fbcb0 2024-04-26T04:50:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.798+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300feba0 con 0x7fe0300fbcb0 2024-04-26T04:50:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.798+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030115b00 con 0x7fe0300fbcb0 2024-04-26T04:50:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.798+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe02000bbf0 con 0x7fe0300fbcb0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.799+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe02001c630 con 0x7fe0300fbcb0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.799+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030101af0 con 0x7fe0300fbcb0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.800+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe020026440 con 0x7fe0300fbcb0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.800+0000 7fe02cff9700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 0x7fe01806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.800+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe020087da0 con 0x7fe0300fbcb0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.800+0000 7fe02f7fe700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 0x7fe01806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.801+0000 7fe02f7fe700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 0x7fe01806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe024004f60 tx=0x7fe02400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:09.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:09.808+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe0200542c0 con 0x7fe0300fbcb0 2024-04-26T04:50:10.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.044+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe03005f5c0 con 0x7fe0300fbcb0 2024-04-26T04:50:10.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.510+0000 7fe02cff9700 1 -- 172.21.15.77:0/3293459473 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v13) v1 ==== 168+0+0 (secure 0 0 0) 0x7fe0200578e0 con 0x7fe0300fbcb0 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 msgr2=0x7fe01806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 0x7fe01806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe024004f60 tx=0x7fe02400b040 comp rx=0 tx=0).stop 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 msgr2=0x7fe0300ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b3b0 tx=0x7fe020009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 shutdown_connections 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe0180692a0 0x7fe01806b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0300fbcb0 0x7fe0300ffd30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 --2- 172.21.15.77:0/3293459473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0300fcf60 0x7fe0300fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:10.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 >> 172.21.15.77:0/3293459473 conn(0x7fe0300f7430 msgr2=0x7fe030065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:10.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 shutdown_connections 2024-04-26T04:50:10.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:10.513+0000 7fe036a12700 1 -- 172.21.15.77:0/3293459473 wait complete. 2024-04-26T04:50:10.515 INFO:teuthology.orchestra.run.smithi077.stderr:inline data enabled 2024-04-26T04:50:10.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:10 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3293459473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T04:50:10.933 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:10 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3293459473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T04:50:11.250 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:50:11.260 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:11.260 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T04:50:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:11 smithi078 ceph-mon[33746]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-26T04:50:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:11 smithi078 ceph-mon[33746]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-26T04:50:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:11 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:standby 2024-04-26T04:50:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:11 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3293459473' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T04:50:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:11 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:11.988 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:11 smithi077 ceph-mon[26311]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-26T04:50:11.988 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:11 smithi077 ceph-mon[26311]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-26T04:50:11.988 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:11 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:standby 2024-04-26T04:50:11.988 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:11 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3293459473' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T04:50:11.988 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:11 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:12.133 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:12.843 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:12 smithi077 ceph-mon[26311]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:standby 2024-04-26T04:50:12.843 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:12 smithi077 ceph-mon[26311]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:12 smithi078 ceph-mon[33746]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:standby 2024-04-26T04:50:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:12 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:50:13.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:13 smithi077 ceph-mon[26311]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 2 op/s 2024-04-26T04:50:14.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:13 smithi078 ceph-mon[33746]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 2 op/s 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 -- 172.21.15.77:0/4238683411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fbcb0 msgr2=0x7fc05c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 --2- 172.21.15.77:0/4238683411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c0fc0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc050004940 tx=0x7fc050009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 -- 172.21.15.77:0/4238683411 shutdown_connections 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 --2- 172.21.15.77:0/4238683411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fcf60 0x7fc05c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 --2- 172.21.15.77:0/4238683411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c0fc0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 -- 172.21.15.77:0/4238683411 >> 172.21.15.77:0/4238683411 conn(0x7fc05c0f7430 msgr2=0x7fc05c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 -- 172.21.15.77:0/4238683411 shutdown_connections 2024-04-26T04:50:14.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.683+0000 7fc062146700 1 -- 172.21.15.77:0/4238683411 wait complete. 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 Processor -- start 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 -- start start 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c10e620 con 0x7fc05c0fcf60 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc062146700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c10edd0 con 0x7fc05c0fbcb0 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55090/0 (socket says 172.21.15.77:55090) 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 -- 172.21.15.77:0/776771238 learned_addr learned my addr 172.21.15.77:0/776771238 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05b7fe700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 -- 172.21.15.77:0/776771238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fbcb0 msgr2=0x7fc05c10dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:14.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c10dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 -- 172.21.15.77:0/776771238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050005040 con 0x7fc05c0fcf60 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.684+0000 7fc05affd700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc04c002a50 tx=0x7fc04c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.685+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04c007cc0 con 0x7fc05c0fcf60 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.685+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c10f0b0 con 0x7fc05c0fcf60 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.685+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc04c00f040 con 0x7fc05c0fcf60 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.685+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04c00da30 con 0x7fc05c0fcf60 2024-04-26T04:50:14.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.685+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c10f5f0 con 0x7fc05c0fcf60 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.686+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c101af0 con 0x7fc05c0fcf60 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.686+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc04c00db90 con 0x7fc05c0fcf60 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.687+0000 7fc043fff700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 0x7fc04406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.687+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc04c086f50 con 0x7fc05c0fcf60 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.687+0000 7fc05b7fe700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 0x7fc04406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:14.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.687+0000 7fc05b7fe700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 0x7fc04406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc05000b970 tx=0x7fc050009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:14.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.691+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc04c056c90 con 0x7fc05c0fcf60 2024-04-26T04:50:14.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.956+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc05c05f5c0 con 0x7fc05c0fcf60 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.957+0000 7fc043fff700 1 -- 172.21.15.77:0/776771238 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7fc04c056820 con 0x7fc05c0fcf60 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:50:14.958 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:50:14.959 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:14.960 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:14.961 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:14.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.960+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 msgr2=0x7fc04406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:14.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.960+0000 7fc062146700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 0x7fc04406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc05000b970 tx=0x7fc050009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.960+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 msgr2=0x7fc05c10e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.960+0000 7fc062146700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc04c002a50 tx=0x7fc04c0046b0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 shutdown_connections 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fc0440692a0 0x7fc04406b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc05c0fbcb0 0x7fc05c10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 --2- 172.21.15.77:0/776771238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc05c0fcf60 0x7fc05c10e020 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 >> 172.21.15.77:0/776771238 conn(0x7fc05c0f7430 msgr2=0x7fc05c065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 shutdown_connections 2024-04-26T04:50:14.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:14.961+0000 7fc062146700 1 -- 172.21.15.77:0/776771238 wait complete. 2024-04-26T04:50:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:50:15.664 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 1 op/s 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:15.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:15 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/776771238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 1 op/s 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:16.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:15 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/776771238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:17 smithi077 ceph-mon[26311]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:50:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:17.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:17 smithi078 ceph-mon[33746]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:50:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:18.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 -- 172.21.15.77:0/2352600488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 msgr2=0x7f1f940fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 --2- 172.21.15.77:0/2352600488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f940fd270 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009ce0 tx=0x7f1f8c004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 -- 172.21.15.77:0/2352600488 shutdown_connections 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 --2- 172.21.15.77:0/2352600488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f940fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 --2- 172.21.15.77:0/2352600488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f940fd270 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 -- 172.21.15.77:0/2352600488 >> 172.21.15.77:0/2352600488 conn(0x7f1f940f8540 msgr2=0x7f1f940fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 -- 172.21.15.77:0/2352600488 shutdown_connections 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.922+0000 7f1f99430700 1 -- 172.21.15.77:0/2352600488 wait complete. 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 Processor -- start 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 -- start start 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f9410dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9410e5d0 con 0x7f1f940fce60 2024-04-26T04:50:18.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f99430700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9410e740 con 0x7f1f940fdfd0 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59164/0 (socket says 172.21.15.77:59164) 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 -- 172.21.15.77:0/3145239846 learned_addr learned my addr 172.21.15.77:0/3145239846 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f927fc700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f9410dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 -- 172.21.15.77:0/3145239846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 msgr2=0x7f1f9410dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f9410dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f92ffd700 1 -- 172.21.15.77:0/3145239846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c009940 con 0x7f1f940fce60 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.923+0000 7f1f927fc700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f9410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f92ffd700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004740 tx=0x7f1f8c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8c00e040 con 0x7f1f940fce60 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f8c0128e0 con 0x7f1f940fce60 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f94113140 con 0x7f1f940fce60 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8c01ca80 con 0x7f1f940fce60 2024-04-26T04:50:18.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.924+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f94113700 con 0x7f1f940fce60 2024-04-26T04:50:18.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.925+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9404c710 con 0x7f1f940fce60 2024-04-26T04:50:18.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.925+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1f8c012a50 con 0x7f1f940fce60 2024-04-26T04:50:18.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.926+0000 7f1f8b7fe700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 0x7f1f8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:18.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.926+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1f8c088db0 con 0x7f1f940fce60 2024-04-26T04:50:18.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.926+0000 7f1f927fc700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 0x7f1f8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:18.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.926+0000 7f1f927fc700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 0x7f1f8006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c007fd0 tx=0x7f1f7c009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:18.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:18.930+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1f8c058b70 con 0x7f1f940fce60 2024-04-26T04:50:19.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.171+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f941058c0 con 0x7f1f940fce60 2024-04-26T04:50:19.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.172+0000 7f1f8b7fe700 1 -- 172.21.15.77:0/3145239846 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f1f8c058700 con 0x7f1f940fce60 2024-04-26T04:50:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 msgr2=0x7f1f8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 0x7f1f8006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c007fd0 tx=0x7f1f7c009040 comp rx=0 tx=0).stop 2024-04-26T04:50:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 msgr2=0x7f1f9410da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004740 tx=0x7f1f8c004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 shutdown_connections 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1f800692a0 0x7f1f8006b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f940fce60 0x7f1f9410da90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 --2- 172.21.15.77:0/3145239846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f940fdfd0 0x7f1f9410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 >> 172.21.15.77:0/3145239846 conn(0x7f1f940f8540 msgr2=0x7f1f940f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 shutdown_connections 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:19.174+0000 7f1f99430700 1 -- 172.21.15.77:0/3145239846 wait complete. 2024-04-26T04:50:19.175 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:50:19.185 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:50:19.838 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-26T04:50:20.228 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:20.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:19 smithi077 ceph-mon[26311]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:20.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:19 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3145239846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:50:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:19 smithi078 ceph-mon[33746]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:20.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:19 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3145239846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:50:21.962 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:21 smithi077 ceph-mon[26311]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:21 smithi078 ceph-mon[33746]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/3123114490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fcf70 msgr2=0x7f4fb40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/3123114490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fcf70 0x7f4fb40fd3e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c004940 tx=0x7f4f9c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/3123114490 shutdown_connections 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/3123114490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fcf70 0x7f4fb40fd3e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/3123114490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/3123114490 >> 172.21.15.77:0/3123114490 conn(0x7f4fb40f7420 msgr2=0x7f4fb40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/3123114490 shutdown_connections 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.250+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/3123114490 wait complete. 2024-04-26T04:50:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 Processor -- start 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 -- start start 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 0x7f4fb410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410e520 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb8ab8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410ed70 con 0x7f4fb40fcf70 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59172/0 (socket says 172.21.15.77:59172) 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 -- 172.21.15.77:0/432956137 learned_addr learned my addr 172.21.15.77:0/432956137 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb1d9b700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 0x7f4fb410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 -- 172.21.15.77:0/432956137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 msgr2=0x7f4fb410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 0x7f4fb410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 -- 172.21.15.77:0/432956137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c005040 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb1d9b700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 0x7f4fb410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.251+0000 7f4fb259c700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fac001d80 tx=0x7f4fac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.252+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fac007d10 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.252+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fac007e70 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.252+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb410f050 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.252+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fac014640 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.252+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb410f590 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.253+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb4101b00 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.253+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4fac00e960 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.253+0000 7f4faaffd700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 0x7f4fa006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.253+0000 7f4fb1d9b700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 0x7f4fa006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.253+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4fac086f80 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.254+0000 7f4fb1d9b700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 0x7f4fa006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00b970 tx=0x7f4f9c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.258+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4fac056d70 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.578+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f4fb405f5c0 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.579+0000 7f4faaffd700 1 -- 172.21.15.77:0/432956137 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v14) v1 ==== 78+0+99 (secure 0 0 0) 0x7f4fac056900 con 0x7f4fb40fbcc0 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 msgr2=0x7f4fa006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 0x7f4fa006b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00b970 tx=0x7f4f9c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 msgr2=0x7f4fb410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fac001d80 tx=0x7f4fac009ec0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 shutdown_connections 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4fa00692a0 0x7f4fa006b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fb40fbcc0 0x7f4fb410d9e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 --2- 172.21.15.77:0/432956137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb40fcf70 0x7f4fb410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 >> 172.21.15.77:0/432956137 conn(0x7f4fb40f7420 msgr2=0x7f4fb4066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 shutdown_connections 2024-04-26T04:50:23.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:23.581+0000 7f4fb8ab8700 1 -- 172.21.15.77:0/432956137 wait complete. 2024-04-26T04:50:23.592 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:50:24.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:23 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/432956137' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-26T04:50:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:23 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/432956137' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-26T04:50:24.279 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-26T04:50:24.293 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 -- 172.21.15.77:0/3500316288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 msgr2=0x7f41180fed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 --2- 172.21.15.77:0/3500316288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f41180fed90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4114009c50 tx=0x7f4114004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 -- 172.21.15.77:0/3500316288 shutdown_connections 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 --2- 172.21.15.77:0/3500316288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41180ff2d0 0x7f41181016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 --2- 172.21.15.77:0/3500316288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f41180fed90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.394+0000 7f411f310640 1 -- 172.21.15.77:0/3500316288 >> 172.21.15.77:0/3500316288 conn(0x7f41180f6840 msgr2=0x7f41180f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 -- 172.21.15.77:0/3500316288 shutdown_connections 2024-04-26T04:50:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 -- 172.21.15.77:0/3500316288 wait complete. 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 Processor -- start 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 -- start start 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41180ff2d0 0x7f411810d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411810d9d0 con 0x7f41180661b0 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411f310640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181122f0 con 0x7f41180ff2d0 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59184/0 (socket says 172.21.15.77:59184) 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 -- 172.21.15.77:0/2148102582 learned_addr learned my addr 172.21.15.77:0/2148102582 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 -- 172.21.15.77:0/2148102582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41180ff2d0 msgr2=0x7f411810d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41180ff2d0 0x7f411810d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.395+0000 7f411d0ac640 1 -- 172.21.15.77:0/2148102582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140098b0 con 0x7f41180661b0 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f411d0ac640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4114000f80 tx=0x7f4114004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411403a070 con 0x7f41180661b0 2024-04-26T04:50:24.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41181124f0 con 0x7f41180661b0 2024-04-26T04:50:24.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4114035d30 con 0x7f41180661b0 2024-04-26T04:50:24.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411403eb20 con 0x7f41180661b0 2024-04-26T04:50:24.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.396+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4118112a10 con 0x7f41180661b0 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.397+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411804bcc0 con 0x7f41180661b0 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.397+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4114032070 con 0x7f41180661b0 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.398+0000 7f410dffb640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 0x7f410406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.398+0000 7f411c8ab640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 0x7f410406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.398+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f41140a9660 con 0x7f41180661b0 2024-04-26T04:50:24.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.398+0000 7f411c8ab640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 0x7f410406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f411810e260 tx=0x7f4108006d20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.402+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4114075910 con 0x7f41180661b0 2024-04-26T04:50:24.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.569+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4118112e90 con 0x7f41180661b0 2024-04-26T04:50:24.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.570+0000 7f410dffb640 1 -- 172.21.15.77:0/2148102582 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14194 (secure 0 0 0) 0x7f4114078f30 con 0x7f41180661b0 2024-04-26T04:50:24.571 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:24.571 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":46,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T04:49:57.010081+0000","last_up_change":"2024-04-26T04:48:34.475903+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:49:53.533004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:49:54.229975+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6803","nonce":1220397746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6805","nonce":1220397746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6809","nonce":1220397746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1220397746},{"type":"v1","addr":"172.21.15.77:6807","nonce":1220397746}]},"public_addr":"172.21.15.77:6803/1220397746","cluster_addr":"172.21.15.77:6805/1220397746","heartbeat_back_addr":"172.21.15.77:6809/1220397746","heartbeat_front_addr":"172.21.15.77:6807/1220397746","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6811","nonce":50899416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6813","nonce":50899416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6817","nonce":50899416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":50899416},{"type":"v1","addr":"172.21.15.77:6815","nonce":50899416}]},"public_addr":"172.21.15.77:6811/50899416","cluster_addr":"172.21.15.77:6813/50899416","heartbeat_back_addr":"172.21.15.77:6817/50899416","heartbeat_front_addr":"172.21.15.77:6815/50899416","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6819","nonce":2248042345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6821","nonce":2248042345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6825","nonce":2248042345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2248042345},{"type":"v1","addr":"172.21.15.77:6823","nonce":2248042345}]},"public_addr":"172.21.15.77:6819/2248042345","cluster_addr":"172.21.15.77:6821/2248042345","heartbeat_back_addr":"172.21.15.77:6825/2248042345","heartbeat_front_addr":"172.21.15.77:6823/2248042345","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6801","nonce":632729185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6803","nonce":632729185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6807","nonce":632729185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":632729185},{"type":"v1","addr":"172.21.15.78:6805","nonce":632729185}]},"public_addr":"172.21.15.78:6801/632729185","cluster_addr":"172.21.15.78:6803/632729185","heartbeat_back_addr":"172.21.15.78:6807/632729185","heartbeat_front_addr":"172.21.15.78:6805/632729185","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6809","nonce":489554457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6811","nonce":489554457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6815","nonce":489554457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":489554457},{"type":"v1","addr":"172.21.15.78:6813","nonce":489554457}]},"public_addr":"172.21.15.78:6809/489554457","cluster_addr":"172.21.15.78:6811/489554457","heartbeat_back_addr":"172.21.15.78:6815/489554457","heartbeat_front_addr":"172.21.15.78:6813/489554457","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6817","nonce":2307894772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6819","nonce":2307894772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6823","nonce":2307894772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2307894772},{"type":"v1","addr":"172.21.15.78:6821","nonce":2307894772}]},"public_addr":"172.21.15.78:6817/2307894772","cluster_addr":"172.21.15.78:6819/2307894772","heartbeat_back_addr":"172.21.15.78:6823/2307894772","heartbeat_front_addr":"172.21.15.78:6821/2307894772","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 msgr2=0x7f410406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 0x7f410406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f411810e260 tx=0x7f4108006d20 comp rx=0 tx=0).stop 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 msgr2=0x7f411810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4114000f80 tx=0x7f4114004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 shutdown_connections 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f4104069210 0x7f410406b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41180ff2d0 0x7f411810d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 --2- 172.21.15.77:0/2148102582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41180661b0 0x7f411810cdb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 >> 172.21.15.77:0/2148102582 conn(0x7f41180f6840 msgr2=0x7f41180f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:24.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 shutdown_connections 2024-04-26T04:50:24.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.572+0000 7f411f310640 1 -- 172.21.15.77:0/2148102582 wait complete. 2024-04-26T04:50:24.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:47:22.739195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:49:53.533004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:49:54.229975+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T04:50:24.584 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 -- 172.21.15.77:0/2555982887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fb070 msgr2=0x7f00780fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 --2- 172.21.15.77:0/2555982887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fb070 0x7f00780fb480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0070009c50 tx=0x7f0070004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 -- 172.21.15.77:0/2555982887 shutdown_connections 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 --2- 172.21.15.77:0/2555982887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fc340 0x7f00780fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 --2- 172.21.15.77:0/2555982887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fb070 0x7f00780fb480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 -- 172.21.15.77:0/2555982887 >> 172.21.15.77:0/2555982887 conn(0x7f00780f6830 msgr2=0x7f00780f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 -- 172.21.15.77:0/2555982887 shutdown_connections 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.679+0000 7f007cc2e640 1 -- 172.21.15.77:0/2555982887 wait complete. 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 Processor -- start 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 -- start start 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 0x7f0078065f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078112040 con 0x7f00780fc340 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007cc2e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078112180 con 0x7f00780fb070 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59210/0 (socket says 172.21.15.77:59210) 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 -- 172.21.15.77:0/1266607853 learned_addr learned my addr 172.21.15.77:0/1266607853 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007659c640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 0x7f0078065f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 -- 172.21.15.77:0/1266607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 msgr2=0x7f0078065f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 0x7f0078065f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f0075d9b640 1 -- 172.21.15.77:0/1266607853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700098b0 con 0x7f00780fc340 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.680+0000 7f007659c640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 0x7f0078065f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f0075d9b640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f006c00eb70 tx=0x7f006c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006c007c30 con 0x7f00780fc340 2024-04-26T04:50:24.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006c00f040 con 0x7f00780fc340 2024-04-26T04:50:24.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0078112380 con 0x7f00780fc340 2024-04-26T04:50:24.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006c0095b0 con 0x7f00780fc340 2024-04-26T04:50:24.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.681+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078112920 con 0x7f00780fc340 2024-04-26T04:50:24.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.682+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0078105570 con 0x7f00780fc340 2024-04-26T04:50:24.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.682+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f006c009730 con 0x7f00780fc340 2024-04-26T04:50:24.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.682+0000 7f0066ffd640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 0x7f006006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.682+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f006c086f10 con 0x7f00780fc340 2024-04-26T04:50:24.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.682+0000 7f007659c640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 0x7f006006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.683+0000 7f007659c640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 0x7f006006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0070004f50 tx=0x7f007002e040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.687+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f006c053280 con 0x7f00780fc340 2024-04-26T04:50:24.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.852+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0078112de0 con 0x7f00780fc340 2024-04-26T04:50:24.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.852+0000 7f0066ffd640 1 -- 172.21.15.77:0/1266607853 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f006c0568a0 con 0x7f00780fc340 2024-04-26T04:50:24.853 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 msgr2=0x7f006006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 0x7f006006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0070004f50 tx=0x7f007002e040 comp rx=0 tx=0).stop 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 msgr2=0x7f0078111b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f006c00eb70 tx=0x7f006c00b040 comp rx=0 tx=0).stop 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 shutdown_connections 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f0060069210 0x7f006006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00780fc340 0x7f0078111b00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 --2- 172.21.15.77:0/1266607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00780fb070 0x7f0078065f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 >> 172.21.15.77:0/1266607853 conn(0x7f00780f6830 msgr2=0x7f00780ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:24.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 shutdown_connections 2024-04-26T04:50:24.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.854+0000 7f007cc2e640 1 -- 172.21.15.77:0/1266607853 wait complete. 2024-04-26T04:50:24.865 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 -- 172.21.15.77:0/630938917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 msgr2=0x7fe9900fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 --2- 172.21.15.77:0/630938917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe9900fc7b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9840048d0 tx=0x7fe984009da0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 -- 172.21.15.77:0/630938917 shutdown_connections 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 --2- 172.21.15.77:0/630938917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe9900fc7b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 --2- 172.21.15.77:0/630938917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe9900fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 -- 172.21.15.77:0/630938917 >> 172.21.15.77:0/630938917 conn(0x7fe9900f6830 msgr2=0x7fe9900f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.960+0000 7fe99591d640 1 -- 172.21.15.77:0/630938917 shutdown_connections 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 -- 172.21.15.77:0/630938917 wait complete. 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 Processor -- start 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 -- start start 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe99010cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99010d9d0 con 0x7fe9900fc340 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe99591d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901122f0 con 0x7fe9900fb070 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe98e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe98e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59230/0 (socket says 172.21.15.77:59230) 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.961+0000 7fe98e7fc640 1 -- 172.21.15.77:0/2954793778 learned_addr learned my addr 172.21.15.77:0/2954793778 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98effd640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe99010cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98e7fc640 1 -- 172.21.15.77:0/2954793778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 msgr2=0x7fe99010cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98e7fc640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe99010cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98e7fc640 1 -- 172.21.15.77:0/2954793778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984005040 con 0x7fe9900fc340 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98effd640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe99010cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe98e7fc640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe984007da0 tx=0x7fe984009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9840094c0 con 0x7fe9900fc340 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe984009620 con 0x7fe9900fc340 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901124f0 con 0x7fe9900fc340 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe98403b6d0 con 0x7fe9900fc340 2024-04-26T04:50:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.962+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe990112a10 con 0x7fe9900fc340 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.963+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99004bcc0 con 0x7fe9900fc340 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.963+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe98402f070 con 0x7fe9900fc340 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.964+0000 7fe9777fe640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 0x7fe97806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.964+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe9840a9770 con 0x7fe9900fc340 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.964+0000 7fe98effd640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 0x7fe97806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.964+0000 7fe98effd640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 0x7fe97806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe980006fd0 tx=0x7fe980008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:24.968+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe984075a20 con 0x7fe9900fc340 2024-04-26T04:50:25.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.136+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe990100e40 con 0x7fe9900fc340 2024-04-26T04:50:25.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.136+0000 7fe9777fe640 1 -- 172.21.15.77:0/2954793778 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe984079040 con 0x7fe9900fc340 2024-04-26T04:50:25.137 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 msgr2=0x7fe97806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 0x7fe97806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe980006fd0 tx=0x7fe980008040 comp rx=0 tx=0).stop 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 msgr2=0x7fe99010d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe984007da0 tx=0x7fe984009da0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 shutdown_connections 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe978069210 0x7fe97806b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9900fc340 0x7fe99010d2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 --2- 172.21.15.77:0/2954793778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9900fb070 0x7fe99010cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 >> 172.21.15.77:0/2954793778 conn(0x7fe9900f6830 msgr2=0x7fe990065ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:25.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 shutdown_connections 2024-04-26T04:50:25.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.138+0000 7fe99591d640 1 -- 172.21.15.77:0/2954793778 wait complete. 2024-04-26T04:50:25.148 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:24 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2148102582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:50:25.148 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:24 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1266607853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:50:25.150 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T04:50:25.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- 172.21.15.77:0/1054789041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fb060 msgr2=0x7f30600fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 --2- 172.21.15.77:0/1054789041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fb060 0x7f30600fb470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f30500048d0 tx=0x7f3050009da0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- 172.21.15.77:0/1054789041 shutdown_connections 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 --2- 172.21.15.77:0/1054789041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fc330 0x7f30600fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 --2- 172.21.15.77:0/1054789041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fb060 0x7f30600fb470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- 172.21.15.77:0/1054789041 >> 172.21.15.77:0/1054789041 conn(0x7f30600f6820 msgr2=0x7f30600f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- 172.21.15.77:0/1054789041 shutdown_connections 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- 172.21.15.77:0/1054789041 wait complete. 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 Processor -- start 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 -- start start 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 0x7f306010cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.245+0000 7f3065cae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3065cae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306010d850 con 0x7f30600fc330 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3065cae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060112210 con 0x7f30600fb060 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59248/0 (socket says 172.21.15.77:59248) 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 -- 172.21.15.77:0/3471497278 learned_addr learned my addr 172.21.15.77:0/3471497278 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305f7fe640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 0x7f306010cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 -- 172.21.15.77:0/3471497278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 msgr2=0x7f306010cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 0x7f306010cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 -- 172.21.15.77:0/3471497278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050005040 con 0x7f30600fc330 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305f7fe640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 0x7f306010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f305effd640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f305400c840 tx=0x7f30540043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305400a820 con 0x7f30600fc330 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f305400ae50 con 0x7f30600fc330 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30601123b0 con 0x7f30600fc330 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30540115c0 con 0x7f30600fc330 2024-04-26T04:50:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.246+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30601128a0 con 0x7f30600fc330 2024-04-26T04:50:25.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.247+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306004bcc0 con 0x7f30600fc330 2024-04-26T04:50:25.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.248+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3054012420 con 0x7f30600fc330 2024-04-26T04:50:25.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.248+0000 7f3047fff640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 0x7f304806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.248+0000 7f305f7fe640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 0x7f304806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.248+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3054087740 con 0x7f30600fc330 2024-04-26T04:50:25.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.248+0000 7f305f7fe640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 0x7f304806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3050004540 tx=0x7f3050009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:25.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.252+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f30540539f0 con 0x7f30600fc330 2024-04-26T04:50:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:24 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2148102582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:50:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:24 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1266607853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:50:25.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.419+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3060103d00 con 0x7f30600fc330 2024-04-26T04:50:25.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.419+0000 7f3047fff640 1 -- 172.21.15.77:0/3471497278 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3054016020 con 0x7f30600fc330 2024-04-26T04:50:25.420 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 msgr2=0x7f304806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 0x7f304806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3050004540 tx=0x7f3050009da0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 msgr2=0x7f306010d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f305400c840 tx=0x7f30540043c0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 shutdown_connections 2024-04-26T04:50:25.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f3048069210 0x7f304806b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30600fc330 0x7f306010d1d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 --2- 172.21.15.77:0/3471497278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30600fb060 0x7f306010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 >> 172.21.15.77:0/3471497278 conn(0x7f30600f6820 msgr2=0x7f30600f7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:25.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 shutdown_connections 2024-04-26T04:50:25.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.421+0000 7f3065cae640 1 -- 172.21.15.77:0/3471497278 wait complete. 2024-04-26T04:50:25.433 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T04:50:25.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 -- 172.21.15.77:0/93431222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 msgr2=0x7fe1d00fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 --2- 172.21.15.77:0/93431222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d00fb470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009c50 tx=0x7fe1c0004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:25.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 -- 172.21.15.77:0/93431222 shutdown_connections 2024-04-26T04:50:25.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 --2- 172.21.15.77:0/93431222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d00fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 --2- 172.21.15.77:0/93431222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d00fb470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 -- 172.21.15.77:0/93431222 >> 172.21.15.77:0/93431222 conn(0x7fe1d00f6820 msgr2=0x7fe1d00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 -- 172.21.15.77:0/93431222 shutdown_connections 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.525+0000 7fe1d555d640 1 -- 172.21.15.77:0/93431222 wait complete. 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 Processor -- start 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 -- start start 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d010c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d010c760 con 0x7fe1d00fb060 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1d555d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0111110 con 0x7fe1d00fc330 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59262/0 (socket says 172.21.15.77:59262) 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 -- 172.21.15.77:0/672481120 learned_addr learned my addr 172.21.15.77:0/672481120 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ce7fc640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 -- 172.21.15.77:0/672481120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 msgr2=0x7fe1d010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d010c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ceffd640 1 -- 172.21.15.77:0/672481120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00098b0 con 0x7fe1d00fb060 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.526+0000 7fe1ce7fc640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1ceffd640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0004a30 tx=0x7fe1c0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c003a070 con 0x7fe1d00fb060 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1c0035b20 con 0x7fe1d00fb060 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c003eb30 con 0x7fe1d00fb060 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d0111250 con 0x7fe1d00fb060 2024-04-26T04:50:25.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.527+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d01117c0 con 0x7fe1d00fb060 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.528+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0105560 con 0x7fe1d00fb060 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.528+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe1c0032070 con 0x7fe1d00fb060 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.529+0000 7fe1b77fe640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 0x7fe1b806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.529+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe1c00a9b30 con 0x7fe1d00fb060 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.529+0000 7fe1ce7fc640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 0x7fe1b806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:25.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.529+0000 7fe1ce7fc640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 0x7fe1b806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1d010d080 tx=0x7fe1c400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:25.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.533+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe1c0075de0 con 0x7fe1d00fb060 2024-04-26T04:50:25.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.732+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d0103bd0 con 0x7fe1d00fb060 2024-04-26T04:50:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.732+0000 7fe1b77fe640 1 -- 172.21.15.77:0/672481120 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4683 (secure 0 0 0) 0x7fe1c0079400 con 0x7fe1d00fb060 2024-04-26T04:50:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:25.733 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T04:50:10.502841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6827/132419939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":132419939},{"type":"v1","addr":"172.21.15.77:6827","nonce":132419939}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-26T04:50:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 msgr2=0x7fe1b806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 0x7fe1b806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1d010d080 tx=0x7fe1c400b040 comp rx=0 tx=0).stop 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 msgr2=0x7fe1d010bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0004a30 tx=0x7fe1c0004d10 comp rx=0 tx=0).stop 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 shutdown_connections 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe1b8069210 0x7fe1b806b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d00fc330 0x7fe1d010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 --2- 172.21.15.77:0/672481120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d00fb060 0x7fe1d010bbd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 >> 172.21.15.77:0/672481120 conn(0x7fe1d00f6820 msgr2=0x7fe1d00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 shutdown_connections 2024-04-26T04:50:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:25.734+0000 7fe1d555d640 1 -- 172.21.15.77:0/672481120 wait complete. 2024-04-26T04:50:25.745 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-26T04:50:25.745 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T04:50:25.775 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T04:50:25.775 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-26T04:50:25.775 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-26T04:50:25.775 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T04:50:25.775 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:25.775 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-26T04:50:25.776 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T04:50:25.776 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:25.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:25.776 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-04-26T04:50:25.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:25.792 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-04-26T04:50:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:Cannot find device "ceph-brx" 2024-04-26T04:50:25.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:50:25.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:25.858 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-26T04:50:25.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:25.877 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-04-26T04:50:25.948 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-04-26T04:50:25.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:50:25.950 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T04:50:25.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T04:50:26.203 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:26.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:25 smithi077 ceph-mon[26311]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:26.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:25 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2954793778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:50:26.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:25 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3471497278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:50:26.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:25 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/672481120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:50:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:25 smithi078 ceph-mon[33746]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:25 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2954793778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:50:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:25 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3471497278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:50:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:25 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/672481120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 -- 172.21.15.77:0/2242248634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 msgr2=0x7f39c4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/2242248634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4065ab0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39ac004940 tx=0x7f39ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 -- 172.21.15.77:0/2242248634 shutdown_connections 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/2242248634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4065ab0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/2242248634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:27.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 -- 172.21.15.77:0/2242248634 >> 172.21.15.77:0/2242248634 conn(0x7f39c4071d00 msgr2=0x7f39c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 -- 172.21.15.77:0/2242248634 shutdown_connections 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.854+0000 7f39c9ac4700 1 -- 172.21.15.77:0/2242248634 wait complete. 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 Processor -- start 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 -- start start 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4114700 con 0x7f39c4065640 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c9ac4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c410daf0 con 0x7f39c4102e50 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:27.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40834/0 (socket says 172.21.15.77:40834) 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 -- 172.21.15.77:0/4201484388 learned_addr learned my addr 172.21.15.77:0/4201484388 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39baffd700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 -- 172.21.15.77:0/4201484388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 msgr2=0x7f39c41140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c41140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39c37fe700 1 -- 172.21.15.77:0/4201484388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac005040 con 0x7f39c4065640 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.855+0000 7f39baffd700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c41140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:50:27.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c37fe700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39b4002a50 tx=0x7f39b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b4007cc0 con 0x7f39c4065640 2024-04-26T04:50:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c410ddd0 con 0x7f39c4065640 2024-04-26T04:50:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39b400f040 con 0x7f39c4065640 2024-04-26T04:50:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b4009640 con 0x7f39c4065640 2024-04-26T04:50:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.856+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c410e390 con 0x7f39c4065640 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.857+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c4100100 con 0x7f39c4065640 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.857+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f39b40097a0 con 0x7f39c4065640 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.858+0000 7f39c0ff9700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.858+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f39b4086fd0 con 0x7f39c4065640 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.858+0000 7f39baffd700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.858+0000 7f39baffd700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 0x7f39b006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009fd0 tx=0x7f39ac009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:27.862+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f39b4056d10 con 0x7f39c4065640 2024-04-26T04:50:28.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.097+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39c410f1c0 con 0x7f39c4065640 2024-04-26T04:50:28.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.097+0000 7f39c0ff9700 1 -- 172.21.15.77:0/4201484388 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f39b40568a0 con 0x7f39c4065640 2024-04-26T04:50:28.098 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 msgr2=0x7f39b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 0x7f39b006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009fd0 tx=0x7f39ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 msgr2=0x7f39c4113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39b4002a50 tx=0x7f39b40046b0 comp rx=0 tx=0).stop 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 shutdown_connections 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f39b00692a0 0x7f39b006b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c4065640 0x7f39c4113b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 --2- 172.21.15.77:0/4201484388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c4102e50 0x7f39c41140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 >> 172.21.15.77:0/4201484388 conn(0x7f39c4071d00 msgr2=0x7f39c40fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 shutdown_connections 2024-04-26T04:50:28.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28.099+0000 7f39c9ac4700 1 -- 172.21.15.77:0/4201484388 wait complete. 2024-04-26T04:50:28.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:27 smithi077 ceph-mon[26311]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:27 smithi078 ceph-mon[33746]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T04:50:28.804 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T04:50:28.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:28.805 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-04-26T04:50:28.821 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T04:50:28.821 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T04:50:28.821 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T04:50:28.821 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:28.821 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:28.822 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:28.826 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T04:50:28.826 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:28.826 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T04:50:28.827 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-04-26T04:50:28.827 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::3e39:a3d5:549e:8707/64 scope link noprefixroute 2024-04-26T04:50:28.827 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:28.827 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T04:50:28.827 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:28.827 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-04-26T04:50:28.827 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-04-26T04:50:28.827 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-04-26T04:50:28.827 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-04-26T04:50:28.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T04:50:28.828 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-04-26T04:50:28.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:29.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:29.010 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T04:50:29.030 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:28 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/4201484388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:50:29.034 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-04-26T04:50:29.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:29.034 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-04-26T04:50:29.108 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-04-26T04:50:29.109 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T04:50:29.109 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T04:50:29.109 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T04:50:29.109 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T04:50:29.109 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:29.109 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-04-26T04:50:29.109 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T04:50:29.109 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T04:50:29.110 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T04:50:29.110 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-04-26T04:50:29.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:28 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/4201484388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:29.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:29.378 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-04-26T04:50:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:29.393 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-04-26T04:50:29.447 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:29.447 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-04-26T04:50:29.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:50:29.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T04:50:29.447 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-04-26T04:50:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:29.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:29.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T04:50:29.562 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-04-26T04:50:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:29.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:29.690 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:29.690 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-04-26T04:50:29.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-04-26T04:50:29.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T04:50:29.690 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-04-26T04:50:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:29.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:29.764 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T04:50:29.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:50:29.764 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:29.778 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T04:50:29.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:50:29.779 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:29.834 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-04-26T04:50:29.898 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-04-26T04:50:29.954 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T04:50:29.955 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/054286f2f00b88d6ea9492305d7f10d33ca13719a8f30b1158ddbb91af142785/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/5defdf45072449154856f319dad2aa36e976c276e52ca0b4987d1087a8eebdcc/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/7c83bb6a3300a51a6cf28cd54489b98e6a0cd9c9258aab3425552bdfdb1924fc/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8f8214980584ef47d1f4631cc63767e269247d56ed916c184951f7ab178d235f/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1e5800d7303e050709fd08a0308d6294fcd422add6107bf6933105356d13900f/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8dfe07a8a894271d4646bc036daca12efb67d14bbdfa429c30b2a5ba7f53e627/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/e86f32b60cdf51c25500623de6db3903ce74e31e034b844cea6caff659989861/merged 2024-04-26T04:50:29.956 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/29d55cd1f9db09cff030bd8ceaf2b53044fcf4cfe1beeb67b696d5b99830d153/merged 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1dd1edbf9f4162a6142c8f3b35931d6a01cc4237adcebbda0a6075b2640862c4/merged 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8b21d56139978dd97f33d15127c3769039e695fe76a7fb298714d862e7350a29/merged 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/6a5ae1104dff906b4d8d08d1bbc42daea09e3c6b6b87f43662b1d2a145038127/merged 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/ae0cedc41fd89b590dc77158ad469cfc7e9bec24e8e3e667252e592c85fa976f/merged 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:50:29.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:29.957 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-04-26T04:50:30.009 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T04:50:30.009 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T04:50:30.054 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-04-26T04:50:30.076 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T04:50:30.131 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-04-26T04:50:30.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T04:50:30.133 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/054286f2f00b88d6ea9492305d7f10d33ca13719a8f30b1158ddbb91af142785/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/5defdf45072449154856f319dad2aa36e976c276e52ca0b4987d1087a8eebdcc/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/7c83bb6a3300a51a6cf28cd54489b98e6a0cd9c9258aab3425552bdfdb1924fc/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8f8214980584ef47d1f4631cc63767e269247d56ed916c184951f7ab178d235f/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1e5800d7303e050709fd08a0308d6294fcd422add6107bf6933105356d13900f/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8dfe07a8a894271d4646bc036daca12efb67d14bbdfa429c30b2a5ba7f53e627/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/e86f32b60cdf51c25500623de6db3903ce74e31e034b844cea6caff659989861/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/29d55cd1f9db09cff030bd8ceaf2b53044fcf4cfe1beeb67b696d5b99830d153/merged 2024-04-26T04:50:30.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1dd1edbf9f4162a6142c8f3b35931d6a01cc4237adcebbda0a6075b2640862c4/merged 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8b21d56139978dd97f33d15127c3769039e695fe76a7fb298714d862e7350a29/merged 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/6a5ae1104dff906b4d8d08d1bbc42daea09e3c6b6b87f43662b1d2a145038127/merged 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/ae0cedc41fd89b590dc77158ad469cfc7e9bec24e8e3e667252e592c85fa976f/merged 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:50:30.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:30.135 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-04-26T04:50:30.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi077.stderr:2024-04-26T04:50:30.140+0000 7fe4de4c4300 -1 init, newargv = 0x5615197e0920 newargc=15 2024-04-26T04:50:30.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi077.stderr:ceph-fuse[73838]: starting ceph client 2024-04-26T04:50:30.147 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi077.stderr:ceph-fuse[73838]: starting fuse 2024-04-26T04:50:30.156 INFO:teuthology.orchestra.run.smithi077.stdout:145 2024-04-26T04:50:30.157 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [145] 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> sudo stdin-killer -- python3 -c ' 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> import glob 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> import re 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> import os 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> import subprocess 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> def _find_admin_socket(client_name): 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> files = glob.glob(asok_path) 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> # Given a non-glob path, it better be there 2024-04-26T04:50:30.157 DEBUG:teuthology.orchestra.run.smithi077:> if "*" not in asok_path: 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> assert(len(files) == 1) 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> return files[0] 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> for f in files: 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> contents = proc_f.read() 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> if mountpoint in contents: 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> return f 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> print(_find_admin_socket("client.0")) 2024-04-26T04:50:30.158 DEBUG:teuthology.orchestra.run.smithi077:> ' 2024-04-26T04:50:30.216 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:29 smithi077 ceph-mon[26311]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:29 smithi078 ceph-mon[33746]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:30.272 INFO:teuthology.orchestra.run.smithi077.stdout:/var/run/ceph/ceph-client.0.73838.asok 2024-04-26T04:50:30.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:30.282 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73838.asok 2024-04-26T04:50:30.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:30.283 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73838.asok status 2024-04-26T04:50:30.412 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "metadata": { 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_sha1": "d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 17.2.7-904-gd8c6b0a3 (d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34) quincy (stable)", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "entity_id": "0", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": "smithi077", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "pid": "73838", 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "root": "/" 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_count": 0, 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_pinned_count": 0, 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14522, 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "inst": { 2024-04-26T04:50:30.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": { 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "client", 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "num": 14522 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4289005399 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4289005399 2024-04-26T04:50:30.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "inst_str": "client.14522 192.168.0.1:0/4289005399", 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "addr_str": "192.168.0.1:0/4289005399", 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "inode_count": 1, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_epoch": 13, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch": 46, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch_barrier": 0, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "blocklisted": false, 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs" 2024-04-26T04:50:30.415 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:50:30.421 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T04:50:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:32.145 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:31 smithi077 ceph-mon[26311]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:31 smithi078 ceph-mon[33746]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 -- 172.21.15.77:0/1306266505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488102de0 msgr2=0x7f44881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 --2- 172.21.15.77:0/1306266505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488102de0 0x7f44881031b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4478004940 tx=0x7f4478009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 -- 172.21.15.77:0/1306266505 shutdown_connections 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 --2- 172.21.15.77:0/1306266505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488103780 0x7f44880fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 --2- 172.21.15.77:0/1306266505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488102de0 0x7f44881031b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 -- 172.21.15.77:0/1306266505 >> 172.21.15.77:0/1306266505 conn(0x7f44880f8990 msgr2=0x7f44880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 -- 172.21.15.77:0/1306266505 shutdown_connections 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.361+0000 7f448e614700 1 -- 172.21.15.77:0/1306266505 wait complete. 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 Processor -- start 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 -- start start 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488102de0 0x7f4488113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4488114500 con 0x7f4488103780 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f448e614700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448810db50 con 0x7f4488102de0 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f44877fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f44877fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40844/0 (socket says 172.21.15.77:40844) 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f44877fe700 1 -- 172.21.15.77:0/3002170117 learned_addr learned my addr 172.21.15.77:0/3002170117 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.362+0000 7f4487fff700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488102de0 0x7f4488113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f44877fe700 1 -- 172.21.15.77:0/3002170117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488102de0 msgr2=0x7f4488113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f44877fe700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488102de0 0x7f4488113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f44877fe700 1 -- 172.21.15.77:0/3002170117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478005040 con 0x7f4488103780 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f44877fe700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f447c002a50 tx=0x7f447c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f447c007cc0 con 0x7f4488103780 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f447c00f040 con 0x7f4488103780 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f447c00d9c0 con 0x7f4488103780 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448810de30 con 0x7f4488103780 2024-04-26T04:50:32.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.363+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448810e3f0 con 0x7f4488103780 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.364+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44881000f0 con 0x7f4488103780 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.365+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f447c009730 con 0x7f4488103780 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.365+0000 7f4484ff9700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 0x7f447006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.365+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f447c086e20 con 0x7f4488103780 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.365+0000 7f4487fff700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 0x7f447006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:32.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.365+0000 7f4487fff700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 0x7f447006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4478004f90 tx=0x7f4478009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:32.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.369+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f447c056be0 con 0x7f4488103780 2024-04-26T04:50:32.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.599+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f448810efc0 con 0x7f4488103780 2024-04-26T04:50:32.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.599+0000 7f4484ff9700 1 -- 172.21.15.77:0/3002170117 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f447c056770 con 0x7f4488103780 2024-04-26T04:50:32.601 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 msgr2=0x7f447006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 0x7f447006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4478004f90 tx=0x7f4478009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 msgr2=0x7f4488113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f447c002a50 tx=0x7f447c004600 comp rx=0 tx=0).stop 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 shutdown_connections 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f44700692a0 0x7f447006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4488102de0 0x7f4488113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 --2- 172.21.15.77:0/3002170117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4488103780 0x7f4488113eb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 >> 172.21.15.77:0/3002170117 conn(0x7f44880f8990 msgr2=0x7f4488107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 shutdown_connections 2024-04-26T04:50:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:32.601+0000 7f448e614700 1 -- 172.21.15.77:0/3002170117 wait complete. 2024-04-26T04:50:33.195 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:32 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3002170117' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:50:33.226 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T04:50:33.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T04:50:33.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-04-26T04:50:33.226 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T04:50:33.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:33.227 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T04:50:33.227 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T04:50:33.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T04:50:33.227 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T04:50:33.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:33.227 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42251sec preferred_lft 42251sec 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::2413:3a89:b6c9:ab65/64 scope link noprefixroute 2024-04-26T04:50:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:50:33.247 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T04:50:33.247 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:33.247 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-26T04:50:33.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-04-26T04:50:33.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-04-26T04:50:33.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-04-26T04:50:33.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T04:50:33.248 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-26T04:50:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:32 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3002170117' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:50:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:33.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:33.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:33.436 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T04:50:33.462 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-04-26T04:50:33.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:33.464 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-04-26T04:50:33.531 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-04-26T04:50:33.532 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T04:50:33.532 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T04:50:33.532 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T04:50:33.532 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T04:50:33.532 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:33.532 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-26T04:50:33.532 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T04:50:33.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T04:50:33.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T04:50:33.533 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-26T04:50:33.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:33.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:33.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:33.817 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-26T04:50:33.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:33.833 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-04-26T04:50:33.888 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:33.888 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-26T04:50:33.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:50:33.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T04:50:33.888 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-26T04:50:33.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:33.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:34.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T04:50:34.006 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:34.006 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T04:50:34.007 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-26T04:50:34.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:34.137 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:50:34.137 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-26T04:50:34.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-04-26T04:50:34.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T04:50:34.138 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-26T04:50:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:50:34.192 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:33 smithi078 ceph-mon[33746]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:34.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:34.211 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T04:50:34.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:50:34.211 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:34.225 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T04:50:34.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:50:34.226 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:33 smithi077 ceph-mon[26311]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:34.282 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-04-26T04:50:34.347 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T04:50:34.402 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-04-26T04:50:34.402 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-04-26T04:50:34.403 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:34.404 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/7f99a8aa80f4e7bf24a3fe0b87b6f3608d850a2705d0af595aee5320316456b1/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0c3419d38cf1f6ccfacf64d1fbc4dd2e958328e55285023aa88130e064cd5ecb/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/8988fbf32b3c339fa399d14869118215cc71cb50b7a770fe26d884f72b1070f7/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c8da8e102a5bc24f8a66bcabdd64145205f2ef730dbd41c14321015a4134a436/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0c75ab3fdf6c4c71386fc1d38749de846dc22281efda703d4c271618787521c2/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/f272253122670265136516a0f73dfd3375e86b61a4840050355744e8eff67de0/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/cccbf7ba36c3953055195ccc8c60c9f8cf62f8de25990272b3d6d6ca1ea059cd/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/397119baa1eaa9cc3e25d22638cd589fc0e61eae756f8185817509608256dc83/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/17897f9a1557fd8dcb10c2dcfd3fd336a1c10f2b26aa651d98948aad1cb7cc80/merged 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:50:34.405 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:50:34.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.406 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-04-26T04:50:34.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T04:50:34.457 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T04:50:34.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-04-26T04:50:34.522 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T04:50:34.577 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-04-26T04:50:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-04-26T04:50:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-04-26T04:50:34.580 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/7f99a8aa80f4e7bf24a3fe0b87b6f3608d850a2705d0af595aee5320316456b1/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0c3419d38cf1f6ccfacf64d1fbc4dd2e958328e55285023aa88130e064cd5ecb/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/8988fbf32b3c339fa399d14869118215cc71cb50b7a770fe26d884f72b1070f7/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c8da8e102a5bc24f8a66bcabdd64145205f2ef730dbd41c14321015a4134a436/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0c75ab3fdf6c4c71386fc1d38749de846dc22281efda703d4c271618787521c2/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/f272253122670265136516a0f73dfd3375e86b61a4840050355744e8eff67de0/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/cccbf7ba36c3953055195ccc8c60c9f8cf62f8de25990272b3d6d6ca1ea059cd/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/397119baa1eaa9cc3e25d22638cd589fc0e61eae756f8185817509608256dc83/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/17897f9a1557fd8dcb10c2dcfd3fd336a1c10f2b26aa651d98948aad1cb7cc80/merged 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:50:34.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.582 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-04-26T04:50:34.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi078.stderr:2024-04-26T04:50:34.606+0000 7f050acc6300 -1 init, newargv = 0x563fed85ad20 newargc=15 2024-04-26T04:50:34.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi078.stderr:ceph-fuse[58928]: starting ceph client 2024-04-26T04:50:34.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi078.stderr:ceph-fuse[58928]: starting fuse 2024-04-26T04:50:34.621 INFO:teuthology.orchestra.run.smithi078.stdout:112 2024-04-26T04:50:34.622 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo stdin-killer -- python3 -c ' 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> import re 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> import os 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2024-04-26T04:50:34.622 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-26T04:50:34.623 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> return f 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.1")) 2024-04-26T04:50:34.624 DEBUG:teuthology.orchestra.run.smithi078:> ' 2024-04-26T04:50:34.738 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.1.58928.asok 2024-04-26T04:50:34.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-26T04:50:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:50:34.748 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58928.asok 2024-04-26T04:50:34.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.749 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58928.asok status 2024-04-26T04:50:34.877 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 17.2.7-904-gd8c6b0a3 (d8c6b0a3ba8ca216f4aadb85bc7242e7c3b18d34) quincy (stable)", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "1", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "58928", 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24305, 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2024-04-26T04:50:34.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 24305 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4280036521 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4280036521 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.24305 192.168.0.1:0/4280036521", 2024-04-26T04:50:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/4280036521", 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 13, 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 46, 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2024-04-26T04:50:34.880 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-26T04:50:34.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.886 DEBUG:teuthology.orchestra.run.smithi077:> stat --file-system '--printf=%T 2024-04-26T04:50:34.887 DEBUG:teuthology.orchestra.run.smithi077:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:34.907 INFO:teuthology.orchestra.run.smithi077.stdout:fuseblk 2024-04-26T04:50:34.907 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:34.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.907 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:34.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.979 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2024-04-26T04:50:34.979 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:34.994 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2024-04-26T04:50:34.995 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:34.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:50:34.995 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:35.061 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:50:35.076 INFO:teuthology.task.print:**** done client 2024-04-26T04:50:35.076 INFO:teuthology.run_tasks:Running task parallel... 2024-04-26T04:50:35.120 INFO:teuthology.task.parallel:starting parallel... 2024-04-26T04:50:35.120 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-26T04:50:35.121 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:50:35.121 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:35.121 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-04-26T04:50:35.122 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-26T04:50:35.122 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-26T04:50:35.126 INFO:tasks.workunit:Pulling workunits from ref b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:35.126 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T04:50:35.126 INFO:tasks.workunit:timeout=3h 2024-04-26T04:50:35.126 INFO:tasks.workunit:cleanup=True 2024-04-26T04:50:35.127 DEBUG:teuthology.orchestra.run.smithi077:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:35.145 INFO:teuthology.orchestra.run.smithi077.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:35.145 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 91h/145d Inode: 1 Links: 2 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-04-26 04:50:00.027784154 +0000 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-04-26 04:50:34.976521747 +0000 2024-04-26T04:50:35.146 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2024-04-26T04:50:35.146 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T04:50:35.146 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T04:50:35.214 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:35.230 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:35.230 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-26T04:50:35.230 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-26 04:50:35.209226224 +0000 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-26 04:50:35.209226224 +0000 2024-04-26T04:50:35.231 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2024-04-26T04:50:35.231 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-26T04:50:35.231 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-26T04:50:35.298 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:35.299 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:35.314 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T04:50:35.354 INFO:tasks.workunit.client.1.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-26T04:50:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:36.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:35 smithi077 ceph-mon[26311]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:35 smithi078 ceph-mon[33746]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- 172.21.15.77:0/540103027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fbcc0 msgr2=0x7f605c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 --2- 172.21.15.77:0/540103027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fbcc0 0x7f605c0fc0d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6058004940 tx=0x7f6058009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- 172.21.15.77:0/540103027 shutdown_connections 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 --2- 172.21.15.77:0/540103027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f605c0fcf70 0x7f605c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 --2- 172.21.15.77:0/540103027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fbcc0 0x7f605c0fc0d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- 172.21.15.77:0/540103027 >> 172.21.15.77:0/540103027 conn(0x7f605c0f7440 msgr2=0x7f605c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- 172.21.15.77:0/540103027 shutdown_connections 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- 172.21.15.77:0/540103027 wait complete. 2024-04-26T04:50:36.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 Processor -- start 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- start start 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f605c0fbcc0 0x7f605c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fcf70 0x7f605c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c10e460 con 0x7f605c0fcf70 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.959+0000 7f6063434700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c10e5d0 con 0x7f605c0fbcc0 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fcf70 0x7f605c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fcf70 0x7f605c10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40866/0 (socket says 172.21.15.77:40866) 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 -- 172.21.15.77:0/2102907068 learned_addr learned my addr 172.21.15.77:0/2102907068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60611d0700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f605c0fbcc0 0x7f605c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 -- 172.21.15.77:0/2102907068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f605c0fbcc0 msgr2=0x7f605c10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f605c0fbcc0 0x7f605c10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 -- 172.21.15.77:0/2102907068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058005040 con 0x7f605c0fcf70 2024-04-26T04:50:36.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f60609cf700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f605c0fcf70 0x7f605c10de60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f604c002a50 tx=0x7f604c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604c007cc0 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f6063434700 1 -- 172.21.15.77:0/2102907068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c113030 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f604c00f040 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.960+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604c00da30 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.961+0000 7f6063434700 1 -- 172.21.15.77:0/2102907068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c1135c0 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.961+0000 7f6063434700 1 -- 172.21.15.77:0/2102907068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c101b00 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.962+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f604c00db90 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.962+0000 7f6051ffb700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f60480692a0 0x7f604806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.962+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f604c086f50 con 0x7f605c0fcf70 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.962+0000 7f60611d0700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f60480692a0 0x7f604806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:36.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.963+0000 7f60611d0700 1 --2- 172.21.15.77:0/2102907068 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f60480692a0 0x7f604806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f605800b970 tx=0x7f6058009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:36.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:36.966+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f604c056c90 con 0x7f605c0fcf70 2024-04-26T04:50:37.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:37.156+0000 7f6063434700 1 -- 172.21.15.77:0/2102907068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f605c04c710 con 0x7f605c0fcf70 2024-04-26T04:50:37.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:37.157+0000 7f6051ffb700 1 -- 172.21.15.77:0/2102907068 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f604c056820 con 0x7f605c0fcf70 2024-04-26T04:50:37.160 INFO:teuthology.orchestra.run.smithi077.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-04-26T04:50:37.936 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:50:37.936 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:37.937 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-26T04:50:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:37 smithi077 ceph-mon[26311]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:38.259 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:37 smithi078 ceph-mon[33746]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:40.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- 172.21.15.77:0/3129524952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 msgr2=0x7f29a00fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/3129524952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a00fc050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f299c004940 tx=0x7f299c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- 172.21.15.77:0/3129524952 shutdown_connections 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/3129524952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 0x7f29a00fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/3129524952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a00fc050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- 172.21.15.77:0/3129524952 >> 172.21.15.77:0/3129524952 conn(0x7f29a00f73a0 msgr2=0x7f29a00f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- 172.21.15.77:0/3129524952 shutdown_connections 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- 172.21.15.77:0/3129524952 wait complete. 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 Processor -- start 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- start start 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 0x7f29a010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a010e5d0 con 0x7f29a00fbc40 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a7b5b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a010ed70 con 0x7f29a00fcef0 2024-04-26T04:50:40.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a58f7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a58f7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45398/0 (socket says 172.21.15.77:45398) 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.007+0000 7f29a58f7700 1 -- 172.21.15.77:0/491885152 learned_addr learned my addr 172.21.15.77:0/491885152 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a50f6700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 0x7f29a010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a58f7700 1 -- 172.21.15.77:0/491885152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 msgr2=0x7f29a010df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a58f7700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 0x7f29a010df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a58f7700 1 -- 172.21.15.77:0/491885152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c005040 con 0x7f29a00fbc40 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a58f7700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b3b0 tx=0x7f299c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299c00e040 con 0x7f29a00fbc40 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f299c00bcc0 con 0x7f29a00fbc40 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a010eff0 con 0x7f29a00fbc40 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f299c01c630 con 0x7f29a00fbc40 2024-04-26T04:50:40.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.008+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a010f5b0 con 0x7f29a00fbc40 2024-04-26T04:50:40.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.009+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0101a80 con 0x7f29a00fbc40 2024-04-26T04:50:40.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.010+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f299c013980 con 0x7f29a00fbc40 2024-04-26T04:50:40.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.010+0000 7f29967fc700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 0x7f298c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:40.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.010+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f299c087fd0 con 0x7f29a00fbc40 2024-04-26T04:50:40.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.010+0000 7f29a50f6700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 0x7f298c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:40.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.010+0000 7f29a50f6700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 0x7f298c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:40.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.015+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f299c057d10 con 0x7f29a00fbc40 2024-04-26T04:50:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.202+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f29a00fb040 con 0x7f29a00fbc40 2024-04-26T04:50:40.204 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:39 smithi077 ceph-mon[26311]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.202+0000 7f29967fc700 1 -- 172.21.15.77:0/491885152 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f299c0578a0 con 0x7f29a00fbc40 2024-04-26T04:50:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 msgr2=0x7f298c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 0x7f298c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2024-04-26T04:50:40.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 msgr2=0x7f29a010da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b3b0 tx=0x7f299c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 shutdown_connections 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f298c0692a0 0x7f298c06b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a00fbc40 0x7f29a010da00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 --2- 172.21.15.77:0/491885152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a00fcef0 0x7f29a010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 >> 172.21.15.77:0/491885152 conn(0x7f29a00f73a0 msgr2=0x7f29a0066660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 shutdown_connections 2024-04-26T04:50:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:40.204+0000 7f29a7b5b700 1 -- 172.21.15.77:0/491885152 wait complete. 2024-04-26T04:50:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:39 smithi078 ceph-mon[33746]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:40.852 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-26T04:50:41.107 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:41 smithi077 ceph-mon[26311]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-26T04:50:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:41 smithi078 ceph-mon[33746]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-26T04:50:42.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1094652404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fce80 msgr2=0x7f2ef40fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1094652404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fce80 0x7f2ef40fd290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8004940 tx=0x7f2ee8009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1094652404 shutdown_connections 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1094652404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fdff0 0x7f2ef40fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1094652404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fce80 0x7f2ef40fd290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1094652404 >> 172.21.15.77:0/1094652404 conn(0x7f2ef40f8540 msgr2=0x7f2ef40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1094652404 shutdown_connections 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.889+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1094652404 wait complete. 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 Processor -- start 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 -- start start 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fce80 0x7f2ef410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:42.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef410e580 con 0x7f2ef40fdff0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef94b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef410edd0 con 0x7f2ef40fce80 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45410/0 (socket says 172.21.15.77:45410) 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 -- 172.21.15.77:0/1943756830 learned_addr learned my addr 172.21.15.77:0/1943756830 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef2ffd700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fce80 0x7f2ef410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 -- 172.21.15.77:0/1943756830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fce80 msgr2=0x7f2ef410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fce80 0x7f2ef410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.890+0000 7f2ef27fc700 1 -- 172.21.15.77:0/1943756830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8005040 con 0x7f2ef40fdff0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2ef27fc700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c8c0 tx=0x7f2ee40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee400a760 con 0x7f2ef40fdff0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ee400ad90 con 0x7f2ef40fdff0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef410f0b0 con 0x7f2ef40fdff0 2024-04-26T04:50:42.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee40113a0 con 0x7f2ef40fdff0 2024-04-26T04:50:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.891+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef410f5f0 con 0x7f2ef40fdff0 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.892+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef405f5c0 con 0x7f2ef40fdff0 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.892+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2ee4012400 con 0x7f2ef40fdff0 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.893+0000 7f2edb7fe700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 0x7f2edc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.893+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2ee40871b0 con 0x7f2ef40fdff0 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.893+0000 7f2ef2ffd700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 0x7f2edc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.893+0000 7f2ef2ffd700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 0x7f2edc06b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800b3b0 tx=0x7f2ee8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:42.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:42.897+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ee4056f70 con 0x7f2ef40fdff0 2024-04-26T04:50:43.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.086+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2ef404bed0 con 0x7f2ef40fdff0 2024-04-26T04:50:43.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.087+0000 7f2edb7fe700 1 -- 172.21.15.77:0/1943756830 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f2ee4056b00 con 0x7f2ef40fdff0 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 msgr2=0x7f2edc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 0x7f2edc06b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800b3b0 tx=0x7f2ee8009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 msgr2=0x7f2ef410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c8c0 tx=0x7f2ee40044c0 comp rx=0 tx=0).stop 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 shutdown_connections 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2edc0692b0 0x7f2edc06b760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40fce80 0x7f2ef410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 --2- 172.21.15.77:0/1943756830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ef40fdff0 0x7f2ef410df80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 >> 172.21.15.77:0/1943756830 conn(0x7f2ef40f8540 msgr2=0x7f2ef40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 shutdown_connections 2024-04-26T04:50:43.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:43.089+0000 7f2ef94b5700 1 -- 172.21.15.77:0/1943756830 wait complete. 2024-04-26T04:50:43.817 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-26T04:50:44.072 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:44.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:43 smithi077 ceph-mon[26311]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:50:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:43 smithi078 ceph-mon[33746]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:50:45.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 -- 172.21.15.77:0/4095537454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 msgr2=0x7ffbc40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 --2- 172.21.15.77:0/4095537454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc40fc0f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4004940 tx=0x7ffbb4009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 -- 172.21.15.77:0/4095537454 shutdown_connections 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 --2- 172.21.15.77:0/4095537454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 --2- 172.21.15.77:0/4095537454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc40fc0f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 -- 172.21.15.77:0/4095537454 >> 172.21.15.77:0/4095537454 conn(0x7ffbc40f7440 msgr2=0x7ffbc40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 -- 172.21.15.77:0/4095537454 shutdown_connections 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.677+0000 7ffbca875700 1 -- 172.21.15.77:0/4095537454 wait complete. 2024-04-26T04:50:45.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.678+0000 7ffbca875700 1 Processor -- start 2024-04-26T04:50:45.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.678+0000 7ffbca875700 1 -- start start 2024-04-26T04:50:45.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc4112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4066be0 con 0x7ffbc40fbce0 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4066d20 con 0x7ffbc40fcf90 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45426/0 (socket says 172.21.15.77:45426) 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 -- 172.21.15.77:0/434347509 learned_addr learned my addr 172.21.15.77:0/434347509 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc37fe700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc4112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 -- 172.21.15.77:0/434347509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 msgr2=0x7ffbc4112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc4112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 -- 172.21.15.77:0/434347509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4005040 con 0x7ffbc40fbce0 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc37fe700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc4112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:45.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc3fff700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400b3b0 tx=0x7ffbb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbb400e040 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbb400bcc0 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbb401c630 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4113230 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.679+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc41137c0 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.680+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc405f5c0 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.681+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffbb4013980 con 0x7ffbc40fbce0 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.681+0000 7ffbc0ff9700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 0x7ffbac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.681+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ffbb4087f10 con 0x7ffbc40fbce0 2024-04-26T04:50:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.681+0000 7ffbc37fe700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 0x7ffbac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.682+0000 7ffbc37fe700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 0x7ffbac06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007fd0 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:45.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.686+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffbb4054240 con 0x7ffbc40fbce0 2024-04-26T04:50:45.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.870+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7ffbc404bed0 con 0x7ffbc40fbce0 2024-04-26T04:50:45.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.871+0000 7ffbc0ff9700 1 -- 172.21.15.77:0/434347509 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7ffbb4057860 con 0x7ffbc40fbce0 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 msgr2=0x7ffbac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 0x7ffbac06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007fd0 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 msgr2=0x7ffbc4110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400b3b0 tx=0x7ffbb4009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 shutdown_connections 2024-04-26T04:50:45.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffbac0692a0 0x7ffbac06b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbc40fbce0 0x7ffbc4110740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 --2- 172.21.15.77:0/434347509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbc40fcf90 0x7ffbc4112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 >> 172.21.15.77:0/434347509 conn(0x7ffbc40f7440 msgr2=0x7ffbc40f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 shutdown_connections 2024-04-26T04:50:45.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:45.873+0000 7ffbca875700 1 -- 172.21.15.77:0/434347509 wait complete. 2024-04-26T04:50:46.184 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:45 smithi078 ceph-mon[33746]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-26T04:50:46.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:45 smithi077 ceph-mon[26311]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-26T04:50:46.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-26T04:50:46.806 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:48.215 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:47 smithi077 ceph-mon[26311]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-26T04:50:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:47 smithi078 ceph-mon[33746]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 -- 172.21.15.77:0/1228509119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 msgr2=0x7f85480fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 --2- 172.21.15.77:0/1228509119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f85480fc0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8544004940 tx=0x7f8544009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 -- 172.21.15.77:0/1228509119 shutdown_connections 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 --2- 172.21.15.77:0/1228509119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 0x7f85480fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 --2- 172.21.15.77:0/1228509119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f85480fc0d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 -- 172.21.15.77:0/1228509119 >> 172.21.15.77:0/1228509119 conn(0x7f85480f7420 msgr2=0x7f85480f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 -- 172.21.15.77:0/1228509119 shutdown_connections 2024-04-26T04:50:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.461+0000 7f8550381700 1 -- 172.21.15.77:0/1228509119 wait complete. 2024-04-26T04:50:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 Processor -- start 2024-04-26T04:50:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 -- start start 2024-04-26T04:50:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 0x7f854810cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854810d3d0 con 0x7f85480fbcc0 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f8550381700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854810dc20 con 0x7f85480fcf70 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43822/0 (socket says 172.21.15.77:43822) 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 -- 172.21.15.77:0/2941779595 learned_addr learned my addr 172.21.15.77:0/2941779595 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854d91c700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 0x7f854810cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 -- 172.21.15.77:0/2941779595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 msgr2=0x7f854810cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 0x7f854810cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 -- 172.21.15.77:0/2941779595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544005040 con 0x7f85480fbcc0 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.462+0000 7f854e11d700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8544013870 tx=0x7f8544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.463+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8544021070 con 0x7f85480fbcc0 2024-04-26T04:50:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.463+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854810dea0 con 0x7f85480fbcc0 2024-04-26T04:50:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.463+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8544009430 con 0x7f85480fbcc0 2024-04-26T04:50:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.463+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854400e070 con 0x7f85480fbcc0 2024-04-26T04:50:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.463+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854810e3e0 con 0x7f85480fbcc0 2024-04-26T04:50:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.464+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854805f5c0 con 0x7f85480fbcc0 2024-04-26T04:50:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.464+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f854401c750 con 0x7f85480fbcc0 2024-04-26T04:50:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.465+0000 7f853effd700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 0x7f853406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.465+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8544088460 con 0x7f85480fbcc0 2024-04-26T04:50:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.465+0000 7f854d91c700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 0x7f853406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.465+0000 7f854d91c700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 0x7f853406b6e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8538009b80 tx=0x7f853800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.473+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8544058220 con 0x7f85480fbcc0 2024-04-26T04:50:48.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.677+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}) v1 -- 0x7f854810e790 con 0x7f8534069230 2024-04-26T04:50:48.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.682+0000 7f853effd700 1 -- 172.21.15.77:0/2941779595 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f854810e790 con 0x7f8534069230 2024-04-26T04:50:48.684 INFO:teuthology.orchestra.run.smithi077.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:48.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 msgr2=0x7f853406b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 0x7f853406b6e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8538009b80 tx=0x7f853800b040 comp rx=0 tx=0).stop 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 msgr2=0x7f854810c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8544013870 tx=0x7f8544009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 shutdown_connections 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8534069230 0x7f853406b6e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85480fbcc0 0x7f854810c890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 --2- 172.21.15.77:0/2941779595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85480fcf70 0x7f854810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 >> 172.21.15.77:0/2941779595 conn(0x7f85480f7420 msgr2=0x7f85480f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 shutdown_connections 2024-04-26T04:50:48.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:48.684+0000 7f8550381700 1 -- 172.21.15.77:0/2941779595 wait complete. 2024-04-26T04:50:49.469 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:50:49.469 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T04:50:49.470 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-26T04:50:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T04:50:49.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:50:49.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:49.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:49.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:49.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:49.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:49.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:49.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:49 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:50:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:50.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:50.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:50.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:50.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:50.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:49 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:50:50.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:50 smithi077 ceph-mon[26311]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:50 smithi078 ceph-mon[33746]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 -- 172.21.15.77:0/3369082094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 msgr2=0x7f40980fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 --2- 172.21.15.77:0/3369082094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f40980fd3d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f408c004940 tx=0x7f408c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 -- 172.21.15.77:0/3369082094 shutdown_connections 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 --2- 172.21.15.77:0/3369082094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f40980fd3d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 --2- 172.21.15.77:0/3369082094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 0x7f40980fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 -- 172.21.15.77:0/3369082094 >> 172.21.15.77:0/3369082094 conn(0x7f40980f7430 msgr2=0x7f40980f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 -- 172.21.15.77:0/3369082094 shutdown_connections 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.700+0000 7f409c978700 1 -- 172.21.15.77:0/3369082094 wait complete. 2024-04-26T04:50:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 Processor -- start 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 -- start start 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 0x7f409810c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409810d4b0 con 0x7f40980fcf60 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409c978700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409810dca0 con 0x7f40980fbcb0 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f4095d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f4095d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43828/0 (socket says 172.21.15.77:43828) 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f4095d9b700 1 -- 172.21.15.77:0/3922869970 learned_addr learned my addr 172.21.15.77:0/3922869970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.701+0000 7f409659c700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 0x7f409810c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4095d9b700 1 -- 172.21.15.77:0/3922869970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 msgr2=0x7f409810c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4095d9b700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 0x7f409810c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4095d9b700 1 -- 172.21.15.77:0/3922869970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c005040 con 0x7f40980fcf60 2024-04-26T04:50:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4095d9b700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b3b0 tx=0x7f408c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408c00e040 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f408c00bbf0 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409810df20 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408c01c630 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.702+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409810e4e0 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.703+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409805f5c0 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.703+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f408c00bd60 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.704+0000 7f4086ffd700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 0x7f408006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.704+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f408c087da0 con 0x7f40980fcf60 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.704+0000 7f409659c700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 0x7f408006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.704+0000 7f409659c700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 0x7f408006b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4088009bf0 tx=0x7f40880091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.708+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f408c057c00 con 0x7f40980fcf60 2024-04-26T04:50:51.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.921+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f409810e7c0 con 0x7f40800692a0 2024-04-26T04:50:51.922 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:51 smithi077 ceph-mon[26311]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:50:51.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.922+0000 7f4086ffd700 1 -- 172.21.15.77:0/3922869970 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f409810e7c0 con 0x7f40800692a0 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 msgr2=0x7f408006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 0x7f408006b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4088009bf0 tx=0x7f40880091a0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 msgr2=0x7f409810ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b3b0 tx=0x7f408c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 shutdown_connections 2024-04-26T04:50:51.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f40800692a0 0x7f408006b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40980fbcb0 0x7f409810c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 --2- 172.21.15.77:0/3922869970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980fcf60 0x7f409810ce20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:51.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 >> 172.21.15.77:0/3922869970 conn(0x7f40980f7430 msgr2=0x7f40980f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:51.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.924+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 shutdown_connections 2024-04-26T04:50:51.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:51.925+0000 7f409c978700 1 -- 172.21.15.77:0/3922869970 wait complete. 2024-04-26T04:50:51.935 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:50:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:51 smithi078 ceph-mon[33746]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 -- 172.21.15.77:0/1611100868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 msgr2=0x7fbbec0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/1611100868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec0fd3d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc004940 tx=0x7fbbdc009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 -- 172.21.15.77:0/1611100868 shutdown_connections 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/1611100868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec0fd3d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/1611100868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 0x7fbbec0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 -- 172.21.15.77:0/1611100868 >> 172.21.15.77:0/1611100868 conn(0x7fbbec0f7430 msgr2=0x7fbbec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 -- 172.21.15.77:0/1611100868 shutdown_connections 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.030+0000 7fbbf36ac700 1 -- 172.21.15.77:0/1611100868 wait complete. 2024-04-26T04:50:52.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 Processor -- start 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 -- start start 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 0x7fbbec110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec066330 con 0x7fbbec0fcf60 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf36ac700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec066470 con 0x7fbbec0fbcb0 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf0c47700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf0c47700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43850/0 (socket says 172.21.15.77:43850) 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf0c47700 1 -- 172.21.15.77:0/2549109766 learned_addr learned my addr 172.21.15.77:0/2549109766 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.031+0000 7fbbf1448700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 0x7fbbec110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf0c47700 1 -- 172.21.15.77:0/2549109766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 msgr2=0x7fbbec110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf0c47700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 0x7fbbec110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf0c47700 1 -- 172.21.15.77:0/2549109766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc005040 con 0x7fbbec0fcf60 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf0c47700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b3b0 tx=0x7fbbdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc00e040 con 0x7fbbec0fcf60 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec1131b0 con 0x7fbbec0fcf60 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbdc00bbf0 con 0x7fbbec0fcf60 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc01c630 con 0x7fbbec0fcf60 2024-04-26T04:50:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.032+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec113770 con 0x7fbbec0fcf60 2024-04-26T04:50:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.033+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec05f5c0 con 0x7fbbec0fcf60 2024-04-26T04:50:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.033+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fbbdc026440 con 0x7fbbec0fcf60 2024-04-26T04:50:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.034+0000 7fbbe1ffb700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 0x7fbbd806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.034+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fbbdc087e90 con 0x7fbbec0fcf60 2024-04-26T04:50:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.034+0000 7fbbf1448700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 0x7fbbd806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.034+0000 7fbbf1448700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 0x7fbbd806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbec065fb0 tx=0x7fbbe800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.038+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbbdc054230 con 0x7fbbec0fcf60 2024-04-26T04:50:52.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.260+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbec1012c0 con 0x7fbbd80692a0 2024-04-26T04:50:52.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.261+0000 7fbbe1ffb700 1 -- 172.21.15.77:0/2549109766 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fbbec1012c0 con 0x7fbbd80692a0 2024-04-26T04:50:52.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 msgr2=0x7fbbd806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 0x7fbbd806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbec065fb0 tx=0x7fbbe800b040 comp rx=0 tx=0).stop 2024-04-26T04:50:52.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 msgr2=0x7fbbec112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b3b0 tx=0x7fbbdc009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 shutdown_connections 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fbbd80692a0 0x7fbbd806b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbec0fbcb0 0x7fbbec110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 --2- 172.21.15.77:0/2549109766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbec0fcf60 0x7fbbec112c70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 >> 172.21.15.77:0/2549109766 conn(0x7fbbec0f7430 msgr2=0x7fbbec0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 shutdown_connections 2024-04-26T04:50:52.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.263+0000 7fbbf36ac700 1 -- 172.21.15.77:0/2549109766 wait complete. 2024-04-26T04:50:52.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 -- 172.21.15.77:0/1828718038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fcf10 msgr2=0x7f2b900fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 --2- 172.21.15.77:0/1828718038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fcf10 0x7f2b900fd320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b80004940 tx=0x7f2b80009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 -- 172.21.15.77:0/1828718038 shutdown_connections 2024-04-26T04:50:52.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 --2- 172.21.15.77:0/1828718038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fe080 0x7f2b900fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 --2- 172.21.15.77:0/1828718038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fcf10 0x7f2b900fd320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 -- 172.21.15.77:0/1828718038 >> 172.21.15.77:0/1828718038 conn(0x7f2b900f8560 msgr2=0x7f2b900fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.368+0000 7f2b96781700 1 -- 172.21.15.77:0/1828718038 shutdown_connections 2024-04-26T04:50:52.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.369+0000 7f2b96781700 1 -- 172.21.15.77:0/1828718038 wait complete. 2024-04-26T04:50:52.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.369+0000 7f2b96781700 1 Processor -- start 2024-04-26T04:50:52.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.369+0000 7f2b96781700 1 -- start start 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b96781700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fcf10 0x7f2b90114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b96781700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b96781700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90114b60 con 0x7f2b900fe080 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b96781700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b901153b0 con 0x7f2b900fcf10 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43866/0 (socket says 172.21.15.77:43866) 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 -- 172.21.15.77:0/85991800 learned_addr learned my addr 172.21.15.77:0/85991800 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8ffff700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fcf10 0x7f2b90114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 -- 172.21.15.77:0/85991800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fcf10 msgr2=0x7f2b90114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fcf10 0x7f2b90114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 -- 172.21.15.77:0/85991800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80005040 con 0x7f2b900fe080 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.370+0000 7f2b8f7fe700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c830 tx=0x7f2b84004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.371+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b840106f0 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.371+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b90115690 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.371+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b84010d20 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.371+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b90115bd0 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.371+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b840113a0 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.372+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c0052b0 con 0x7f2b900fe080 2024-04-26T04:50:52.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.373+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2b84010850 con 0x7f2b900fe080 2024-04-26T04:50:52.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.373+0000 7f2b8cff9700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 0x7f2b7806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.373+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2b84086fa0 con 0x7f2b900fe080 2024-04-26T04:50:52.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.373+0000 7f2b8ffff700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 0x7f2b7806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.374+0000 7f2b8ffff700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 0x7f2b7806b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000b3b0 tx=0x7f2b80009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.378+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2b84056e00 con 0x7f2b900fe080 2024-04-26T04:50:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.583+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b7c000bc0 con 0x7f2b780692b0 2024-04-26T04:50:52.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.594+0000 7f2b8cff9700 1 -- 172.21.15.77:0/85991800 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f2b7c000bc0 con 0x7f2b780692b0 2024-04-26T04:50:52.594 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:50:52.594 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (4m) 29s ago 6m 25.7M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:50:52.594 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (6m) 29s ago 6m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (5m) 35s ago 5m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (4m) 29s ago 6m 85.1M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (53s) 29s ago 53s 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (48s) 29s ago 48s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (46s) 35s ago 45s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (51s) 35s ago 50s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (7m) 29s ago 7m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (5m) 35s ago 5m 432M - 17.2.7-902-gc61472c8 0de407c1b3af 7b0ffd125c63 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (7m) 29s ago 7m 44.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (5m) 35s ago 5m 35.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (6m) 29s ago 6m 14.3M - 1.5.0 0da6a335fe13 e44a82d54ec7 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (5m) 35s ago 5m 16.3M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (4m) 29s ago 4m 44.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:50:52.595 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (4m) 29s ago 4m 44.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:50:52.596 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (3m) 29s ago 3m 43.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:50:52.596 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (3m) 35s ago 3m 45.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:50:52.596 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (2m) 35s ago 2m 42.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:50:52.596 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (2m) 35s ago 2m 43.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:50:52.596 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (4m) 29s ago 5m 56.1M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 msgr2=0x7f2b7806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 0x7f2b7806b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000b3b0 tx=0x7f2b80009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 msgr2=0x7f2b90114560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c830 tx=0x7f2b84004470 comp rx=0 tx=0).stop 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 shutdown_connections 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f2b780692b0 0x7f2b7806b760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b900fcf10 0x7f2b90114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 --2- 172.21.15.77:0/85991800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b900fe080 0x7f2b90114560 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 >> 172.21.15.77:0/85991800 conn(0x7f2b900f8560 msgr2=0x7f2b900f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 shutdown_connections 2024-04-26T04:50:52.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.596+0000 7f2b96781700 1 -- 172.21.15.77:0/85991800 wait complete. 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 -- 172.21.15.77:0/599324122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0070830 msgr2=0x7f80f0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 --2- 172.21.15.77:0/599324122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0070830 0x7f80f0070ca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f80e0004940 tx=0x7f80e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 -- 172.21.15.77:0/599324122 shutdown_connections 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 --2- 172.21.15.77:0/599324122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0070830 0x7f80f0070ca0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 --2- 172.21.15.77:0/599324122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0071de0 0x7f80f00702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.698+0000 7f80f7e52700 1 -- 172.21.15.77:0/599324122 >> 172.21.15.77:0/599324122 conn(0x7f80f00f8590 msgr2=0x7f80f00fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.699+0000 7f80f7e52700 1 -- 172.21.15.77:0/599324122 shutdown_connections 2024-04-26T04:50:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.699+0000 7f80f7e52700 1 -- 172.21.15.77:0/599324122 wait complete. 2024-04-26T04:50:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.699+0000 7f80f7e52700 1 Processor -- start 2024-04-26T04:50:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.699+0000 7f80f7e52700 1 -- start start 2024-04-26T04:50:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f7e52700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 0x7f80f010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f7e52700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f7e52700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0110770 con 0x7f80f0071de0 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f7e52700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0110fc0 con 0x7f80f0070830 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43892/0 (socket says 172.21.15.77:43892) 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 -- 172.21.15.77:0/998024774 learned_addr learned my addr 172.21.15.77:0/998024774 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f5bee700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 0x7f80f010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 -- 172.21.15.77:0/998024774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 msgr2=0x7f80f010fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 0x7f80f010fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 -- 172.21.15.77:0/998024774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0005040 con 0x7f80f0071de0 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f5bee700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 0x7f80f010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f53ed700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3b0 tx=0x7f80e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e000e040 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f0111240 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80e000bcc0 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.700+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e001c630 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.701+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f0111780 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.701+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f005f5c0 con 0x7f80f0071de0 2024-04-26T04:50:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.702+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f80e0013980 con 0x7f80f0071de0 2024-04-26T04:50:52.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.703+0000 7f80e67fc700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 0x7f80dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:52.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.703+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f80e0087f10 con 0x7f80f0071de0 2024-04-26T04:50:52.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.703+0000 7f80f5bee700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 0x7f80dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:52.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.703+0000 7f80f5bee700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 0x7f80dc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fd0 tx=0x7f80ec009f00 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:52.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.707+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f80e0057cd0 con 0x7f80f0071de0 2024-04-26T04:50:52.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.990+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80f0107ac0 con 0x7f80f0071de0 2024-04-26T04:50:52.991 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:52 smithi077 ceph-mon[26311]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.991+0000 7f80e67fc700 1 -- 172.21.15.77:0/998024774 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f80e0057860 con 0x7f80f0071de0 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:50:52.992 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:50:52.993 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:50:52.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 msgr2=0x7f80dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 0x7f80dc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fd0 tx=0x7f80ec009f00 comp rx=0 tx=0).stop 2024-04-26T04:50:52.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 msgr2=0x7f80f0110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:52.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3b0 tx=0x7f80e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 shutdown_connections 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f80dc0692a0 0x7f80dc06b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.993+0000 7f80f7e52700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80f0070830 0x7f80f010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.994+0000 7f80f7e52700 1 --2- 172.21.15.77:0/998024774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80f0071de0 0x7f80f0110170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.994+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 >> 172.21.15.77:0/998024774 conn(0x7f80f00f8590 msgr2=0x7f80f00fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.994+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 shutdown_connections 2024-04-26T04:50:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:52.994+0000 7f80f7e52700 1 -- 172.21.15.77:0/998024774 wait complete. 2024-04-26T04:50:53.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:52 smithi078 ceph-mon[33746]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:53.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- 172.21.15.77:0/2583410480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 msgr2=0x7f8d601006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 --2- 172.21.15.77:0/2583410480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d601006f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004940 tx=0x7f8d4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- 172.21.15.77:0/2583410480 shutdown_connections 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 --2- 172.21.15.77:0/2583410480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d601006f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 --2- 172.21.15.77:0/2583410480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d600ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- 172.21.15.77:0/2583410480 >> 172.21.15.77:0/2583410480 conn(0x7f8d600faab0 msgr2=0x7f8d600fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- 172.21.15.77:0/2583410480 shutdown_connections 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- 172.21.15.77:0/2583410480 wait complete. 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 Processor -- start 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.098+0000 7f8d66df7700 1 -- start start 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d66df7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d6010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d66df7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d66df7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d601107f0 con 0x7f8d60100280 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d66df7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60110fe0 con 0x7f8d600fefd0 2024-04-26T04:50:53.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43922/0 (socket says 172.21.15.77:43922) 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 -- 172.21.15.77:0/1221474767 learned_addr learned my addr 172.21.15.77:0/1221474767 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d64b93700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d6010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 -- 172.21.15.77:0/1221474767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 msgr2=0x7f8d6010fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d6010fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 -- 172.21.15.77:0/1221474767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c005040 con 0x7f8d60100280 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d64b93700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d6010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.099+0000 7f8d5ffff700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b3b0 tx=0x7f8d4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.100+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c00e040 con 0x7f8d60100280 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.100+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d60111260 con 0x7f8d60100280 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.100+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4c00bcc0 con 0x7f8d60100280 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.100+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c01c630 con 0x7f8d60100280 2024-04-26T04:50:53.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.100+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d601117d0 con 0x7f8d60100280 2024-04-26T04:50:53.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.101+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6005f5c0 con 0x7f8d60100280 2024-04-26T04:50:53.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.101+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8d4c013980 con 0x7f8d60100280 2024-04-26T04:50:53.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.102+0000 7f8d5d7fa700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 0x7f8d5006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.102+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8d4c087fc0 con 0x7f8d60100280 2024-04-26T04:50:53.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.102+0000 7f8d64b93700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 0x7f8d5006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.102+0000 7f8d64b93700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 0x7f8d5006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009b10 tx=0x7f8d54008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.106+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8d4c057d00 con 0x7f8d60100280 2024-04-26T04:50:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.350+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d60111a40 con 0x7f8d60100280 2024-04-26T04:50:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.350+0000 7f8d5d7fa700 1 -- 172.21.15.77:0/1221474767 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f8d4c057890 con 0x7f8d60100280 2024-04-26T04:50:53.351 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:50:53.351 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:50:53.351 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:53.351 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:50:53.352 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:50:53.353 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:53.354 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 msgr2=0x7f8d5006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 0x7f8d5006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009b10 tx=0x7f8d54008040 comp rx=0 tx=0).stop 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 msgr2=0x7f8d60110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b3b0 tx=0x7f8d4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 shutdown_connections 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8d500692a0 0x7f8d5006b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d600fefd0 0x7f8d6010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 --2- 172.21.15.77:0/1221474767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60100280 0x7f8d60110160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 >> 172.21.15.77:0/1221474767 conn(0x7f8d600faab0 msgr2=0x7f8d600fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:53.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 shutdown_connections 2024-04-26T04:50:53.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.353+0000 7f8d66df7700 1 -- 172.21.15.77:0/1221474767 wait complete. 2024-04-26T04:50:53.356 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.458+0000 7f5586b75700 1 -- 172.21.15.77:0/112790964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 msgr2=0x7f55800fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.458+0000 7f5586b75700 1 --2- 172.21.15.77:0/112790964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55800fd400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f556c004940 tx=0x7f556c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- 172.21.15.77:0/112790964 shutdown_connections 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 --2- 172.21.15.77:0/112790964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55800fd400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 --2- 172.21.15.77:0/112790964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 0x7f55800fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- 172.21.15.77:0/112790964 >> 172.21.15.77:0/112790964 conn(0x7f55800f7480 msgr2=0x7f55800f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- 172.21.15.77:0/112790964 shutdown_connections 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- 172.21.15.77:0/112790964 wait complete. 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 Processor -- start 2024-04-26T04:50:53.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- start start 2024-04-26T04:50:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 0x7f558010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580110770 con 0x7f55800fcf90 2024-04-26T04:50:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.459+0000 7f5586b75700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801150c0 con 0x7f55800fbce0 2024-04-26T04:50:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43944/0 (socket says 172.21.15.77:43944) 2024-04-26T04:50:53.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 -- 172.21.15.77:0/4066175459 learned_addr learned my addr 172.21.15.77:0/4066175459 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f5584911700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 0x7f558010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 -- 172.21.15.77:0/4066175459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 msgr2=0x7f558010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 0x7f558010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 -- 172.21.15.77:0/4066175459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c005040 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557ffff700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f556c009150 tx=0x7f556c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556c00e040 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f556c00bbf0 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556c01c630 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55801152c0 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.460+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5580115830 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.461+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5580101920 con 0x7f55800fcf90 2024-04-26T04:50:53.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.462+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f556c00bd60 con 0x7f55800fcf90 2024-04-26T04:50:53.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.462+0000 7f557d7fa700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 0x7f557006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.462+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f556c088420 con 0x7f55800fcf90 2024-04-26T04:50:53.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.462+0000 7f5584911700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 0x7f557006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.463+0000 7f5584911700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 0x7f557006b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5574009bf0 tx=0x7f55740091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.467+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f556c058330 con 0x7f55800fcf90 2024-04-26T04:50:53.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.685+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f558005e3a0 con 0x7f55700692a0 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.686+0000 7f557d7fa700 1 -- 172.21.15.77:0/4066175459 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f558005e3a0 con 0x7f55700692a0 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:50:53.687 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:50:53.688 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "", 2024-04-26T04:50:53.688 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a image", 2024-04-26T04:50:53.688 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:50:53.688 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:50:53.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 msgr2=0x7f557006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 0x7f557006b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5574009bf0 tx=0x7f55740091a0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 msgr2=0x7f55801100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f556c009150 tx=0x7f556c009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:53.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 shutdown_connections 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f55700692a0 0x7f557006b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55800fbce0 0x7f558010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 --2- 172.21.15.77:0/4066175459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55800fcf90 0x7f55801100e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 >> 172.21.15.77:0/4066175459 conn(0x7f55800f7480 msgr2=0x7f5580066800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 shutdown_connections 2024-04-26T04:50:53.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.689+0000 7f5586b75700 1 -- 172.21.15.77:0/4066175459 wait complete. 2024-04-26T04:50:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 -- 172.21.15.77:0/2904544400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 msgr2=0x7fe8f80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/2904544400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f80fc0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80049c0 tx=0x7fe8e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 -- 172.21.15.77:0/2904544400 shutdown_connections 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/2904544400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f80fcf60 0x7fe8f80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/2904544400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f80fc0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 -- 172.21.15.77:0/2904544400 >> 172.21.15.77:0/2904544400 conn(0x7fe8f80f7430 msgr2=0x7fe8f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 -- 172.21.15.77:0/2904544400 shutdown_connections 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.795+0000 7fe8fe44c700 1 -- 172.21.15.77:0/2904544400 wait complete. 2024-04-26T04:50:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 Processor -- start 2024-04-26T04:50:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 -- start start 2024-04-26T04:50:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f80fcf60 0x7fe8f810bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f810c240 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8fe44c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f810ca90 con 0x7fe8f80fcf60 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8f7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.796+0000 7fe8f7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43966/0 (socket says 172.21.15.77:43966) 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f7fff700 1 -- 172.21.15.77:0/4133827647 learned_addr learned my addr 172.21.15.77:0/4133827647 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f7fff700 1 -- 172.21.15.77:0/4133827647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f80fcf60 msgr2=0x7fe8f810bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f7fff700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f80fcf60 0x7fe8f810bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f7fff700 1 -- 172.21.15.77:0/4133827647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e8005040 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f7fff700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ba10 tx=0x7fe8e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e800bab0 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8e800bc10 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f810cd10 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e80246b0 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.797+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f810d250 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.798+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f805f5c0 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.799+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe8e8021070 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.799+0000 7fe8f4ff9700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 0x7fe8e006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.799+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe8e808b0c0 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.799+0000 7fe8f77fe700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 0x7fe8e006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.800+0000 7fe8f77fe700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 0x7fe8e006b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec004d00 tx=0x7fe8ec009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:53.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:53.804+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe8e805b0e0 con 0x7fe8f80fbcb0 2024-04-26T04:50:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:50:53.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:53.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/998024774' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:53.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1221474767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:53.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/998024774' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1221474767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:54.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.088+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8f804bed0 con 0x7fe8f80fbcb0 2024-04-26T04:50:54.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.089+0000 7fe8f4ff9700 1 -- 172.21.15.77:0/4133827647 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe8e805ac70 con 0x7fe8f80fbcb0 2024-04-26T04:50:54.089 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:50:54.089 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:50:54.090 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 msgr2=0x7fe8e006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 0x7fe8e006b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec004d00 tx=0x7fe8ec009ce0 comp rx=0 tx=0).stop 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 msgr2=0x7fe8f810b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ba10 tx=0x7fe8e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 shutdown_connections 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe8e00692b0 0x7fe8e006b760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f80fbcb0 0x7fe8f810b700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 --2- 172.21.15.77:0/4133827647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f80fcf60 0x7fe8f810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 >> 172.21.15.77:0/4133827647 conn(0x7fe8f80f7430 msgr2=0x7fe8f80f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 shutdown_connections 2024-04-26T04:50:54.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:50:54.091+0000 7fe8fe44c700 1 -- 172.21.15.77:0/4133827647 wait complete. 2024-04-26T04:50:54.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:54 smithi077 ceph-mon[26311]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:54.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:54 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/4133827647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:54 smithi078 ceph-mon[33746]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:54 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/4133827647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:55 smithi077 ceph-mon[26311]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:50:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:55 smithi078 ceph-mon[33746]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:50:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:57 smithi077 ceph-mon[26311]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:58.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:57 smithi078 ceph-mon[33746]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:50:59.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:50:59 smithi077 ceph-mon[26311]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:50:59 smithi078 ceph-mon[33746]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:01 smithi077 ceph-mon[26311]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:01 smithi078 ceph-mon[33746]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:04.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:03 smithi077 ceph-mon[26311]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:03 smithi078 ceph-mon[33746]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:06.079 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:05 smithi078 ceph-mon[33746]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:06.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:05 smithi077 ceph-mon[26311]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:08.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:07 smithi077 ceph-mon[26311]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:07 smithi078 ceph-mon[33746]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:10.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:09 smithi077 ceph-mon[26311]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:09 smithi078 ceph-mon[33746]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:11 smithi077 ceph-mon[26311]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:11 smithi078 ceph-mon[33746]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:13 smithi077 ceph-mon[26311]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:13 smithi078 ceph-mon[33746]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:16.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:15 smithi077 ceph-mon[26311]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:15 smithi078 ceph-mon[33746]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:17 smithi077 ceph-mon[26311]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:17 smithi078 ceph-mon[33746]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:20.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:19 smithi077 ceph-mon[26311]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:19 smithi078 ceph-mon[33746]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:22.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:21 smithi077 ceph-mon[26311]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:21 smithi078 ceph-mon[33746]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:24.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:23 smithi077 ceph-mon[26311]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:24.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 -- 172.21.15.77:0/4103617388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c0fffa0 msgr2=0x7ffa0c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 --2- 172.21.15.77:0/4103617388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c102380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004940 tx=0x7ff9f8009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 -- 172.21.15.77:0/4103617388 shutdown_connections 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 --2- 172.21.15.77:0/4103617388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c102380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 --2- 172.21.15.77:0/4103617388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c066200 0x7ffa0c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 -- 172.21.15.77:0/4103617388 >> 172.21.15.77:0/4103617388 conn(0x7ffa0c0f7430 msgr2=0x7ffa0c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 -- 172.21.15.77:0/4103617388 shutdown_connections 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.240+0000 7ffa1314f700 1 -- 172.21.15.77:0/4103617388 wait complete. 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.241+0000 7ffa1314f700 1 Processor -- start 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.241+0000 7ffa1314f700 1 -- start start 2024-04-26T04:51:24.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.241+0000 7ffa1314f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa1314f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa1314f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c10e540 con 0x7ffa0c066200 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa1314f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c10ed40 con 0x7ffa0c0fffa0 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56154/0 (socket says 172.21.15.77:56154) 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 -- 172.21.15.77:0/2820123073 learned_addr learned my addr 172.21.15.77:0/2820123073 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:37974/0 (socket says 172.21.15.77:37974) 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 -- 172.21.15.77:0/2820123073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 msgr2=0x7ffa0c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 -- 172.21.15.77:0/2820123073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8005040 con 0x7ffa0c066200 2024-04-26T04:51:24.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.242+0000 7ffa10eeb700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000eb70 tx=0x7ffa0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.243+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa00007cc0 con 0x7ffa0c066200 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.243+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa0000f040 con 0x7ffa0c066200 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.243+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0000cb20 con 0x7ffa0c066200 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.243+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c10efc0 con 0x7ffa0c066200 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.243+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c10f500 con 0x7ffa0c066200 2024-04-26T04:51:24.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.244+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c05f5c0 con 0x7ffa0c066200 2024-04-26T04:51:24.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.245+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa0000cc80 con 0x7ffa0c066200 2024-04-26T04:51:24.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.246+0000 7ffa097fa700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 0x7ff9fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.246+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ffa00086b40 con 0x7ffa0c066200 2024-04-26T04:51:24.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.246+0000 7ffa0bfff700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 0x7ff9fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.247+0000 7ffa0bfff700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 0x7ff9fc06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004890 tx=0x7ff9f8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:24.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.251+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa0004e250 con 0x7ffa0c066200 2024-04-26T04:51:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:23 smithi078 ceph-mon[33746]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.583+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa0c10f7e0 con 0x7ff9fc0692a0 2024-04-26T04:51:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.584+0000 7ffa097fa700 1 -- 172.21.15.77:0/2820123073 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ffa0c10f7e0 con 0x7ff9fc0692a0 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 msgr2=0x7ff9fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 0x7ff9fc06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004890 tx=0x7ff9f8009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 msgr2=0x7ffa0c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000eb70 tx=0x7ffa0000b040 comp rx=0 tx=0).stop 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 shutdown_connections 2024-04-26T04:51:24.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff9fc0692a0 0x7ff9fc06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa0c066200 0x7ffa0c10da00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 --2- 172.21.15.77:0/2820123073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa0c0fffa0 0x7ffa0c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.587+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 >> 172.21.15.77:0/2820123073 conn(0x7ffa0c0f7430 msgr2=0x7ffa0c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:24.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.588+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 shutdown_connections 2024-04-26T04:51:24.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.588+0000 7ffa1314f700 1 -- 172.21.15.77:0/2820123073 wait complete. 2024-04-26T04:51:24.600 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3500223946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fbcb0 msgr2=0x7ffa580fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3500223946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fbcb0 0x7ffa580fc0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa48004940 tx=0x7ffa48009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3500223946 shutdown_connections 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3500223946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fcf60 0x7ffa580fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3500223946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fbcb0 0x7ffa580fc0c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3500223946 >> 172.21.15.77:0/3500223946 conn(0x7ffa580f7430 msgr2=0x7ffa580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3500223946 shutdown_connections 2024-04-26T04:51:24.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.741+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3500223946 wait complete. 2024-04-26T04:51:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 Processor -- start 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 -- start start 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fbcb0 0x7ffa58113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58114a00 con 0x7ffa580fcf60 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5cd65700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58114b70 con 0x7ffa580fbcb0 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa5659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fbcb0 0x7ffa58113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa55d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa55d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56168/0 (socket says 172.21.15.77:56168) 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.742+0000 7ffa55d9b700 1 -- 172.21.15.77:0/3016863886 learned_addr learned my addr 172.21.15.77:0/3016863886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa55d9b700 1 -- 172.21.15.77:0/3016863886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fbcb0 msgr2=0x7ffa58113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa55d9b700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fbcb0 0x7ffa58113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa55d9b700 1 -- 172.21.15.77:0/3016863886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa48005040 con 0x7ffa580fcf60 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa55d9b700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00eb70 tx=0x7ffa4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4c007cc0 con 0x7ffa580fcf60 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa58119570 con 0x7ffa580fcf60 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa58119ab0 con 0x7ffa580fcf60 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa4c00f040 con 0x7ffa580fcf60 2024-04-26T04:51:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.743+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4c00ca20 con 0x7ffa580fcf60 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.744+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa380052e0 con 0x7ffa580fcf60 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.745+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa4c00cb80 con 0x7ffa580fcf60 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.745+0000 7ffa46ffd700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 0x7ffa4006b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.745+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ffa4c086b70 con 0x7ffa580fcf60 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.746+0000 7ffa5659c700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 0x7ffa4006b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.746+0000 7ffa5659c700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 0x7ffa4006b790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa48013870 tx=0x7ffa48009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:24.750+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa4c056930 con 0x7ffa580fcf60 2024-04-26T04:51:25.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.102+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa38000bf0 con 0x7ffa400692e0 2024-04-26T04:51:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.103+0000 7ffa46ffd700 1 -- 172.21.15.77:0/3016863886 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ffa38000bf0 con 0x7ffa400692e0 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 msgr2=0x7ffa4006b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 0x7ffa4006b790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa48013870 tx=0x7ffa48009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 msgr2=0x7ffa58114400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00eb70 tx=0x7ffa4c00b040 comp rx=0 tx=0).stop 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 shutdown_connections 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ffa400692e0 0x7ffa4006b790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa580fbcb0 0x7ffa58113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 --2- 172.21.15.77:0/3016863886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa580fcf60 0x7ffa58114400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 >> 172.21.15.77:0/3016863886 conn(0x7ffa580f7430 msgr2=0x7ffa58066660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 shutdown_connections 2024-04-26T04:51:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.106+0000 7ffa5cd65700 1 -- 172.21.15.77:0/3016863886 wait complete. 2024-04-26T04:51:25.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 -- 172.21.15.77:0/3682790110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 msgr2=0x7f76480faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 --2- 172.21.15.77:0/3682790110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f76480faf30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7630004940 tx=0x7f7630009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:25.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 -- 172.21.15.77:0/3682790110 shutdown_connections 2024-04-26T04:51:25.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 --2- 172.21.15.77:0/3682790110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f76480faf30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 --2- 172.21.15.77:0/3682790110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 0x7f76480fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 -- 172.21.15.77:0/3682790110 >> 172.21.15.77:0/3682790110 conn(0x7f76480f6b80 msgr2=0x7f76480f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 -- 172.21.15.77:0/3682790110 shutdown_connections 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 -- 172.21.15.77:0/3682790110 wait complete. 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.241+0000 7f764cb3c700 1 Processor -- start 2024-04-26T04:51:25.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764cb3c700 1 -- start start 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764cb3c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 0x7f764810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764cb3c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764cb3c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764810e4c0 con 0x7f76480fedf0 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764cb3c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764810ed10 con 0x7f76480fe4a0 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56200/0 (socket says 172.21.15.77:56200) 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 -- 172.21.15.77:0/2955830625 learned_addr learned my addr 172.21.15.77:0/2955830625 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f764659c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 0x7f764810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 -- 172.21.15.77:0/2955830625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 msgr2=0x7f764810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 0x7f764810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.242+0000 7f763ffff700 1 -- 172.21.15.77:0/2955830625 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630005040 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f763ffff700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7630013870 tx=0x7f7630009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7630021070 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764810ef90 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764810f4d0 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763001c400 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.243+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763000e070 con 0x7f76480fedf0 2024-04-26T04:51:25.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.244+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76280052e0 con 0x7f76480fedf0 2024-04-26T04:51:25.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.247+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7630013af0 con 0x7f76480fedf0 2024-04-26T04:51:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.247+0000 7f763effd700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 0x7f763406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.247+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f76300884e0 con 0x7f76480fedf0 2024-04-26T04:51:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.247+0000 7f764659c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 0x7f763406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.248+0000 7f764659c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 0x7f763406b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7638004600 tx=0x7f763800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:25.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.254+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7630058200 con 0x7f76480fedf0 2024-04-26T04:51:25.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.592+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7628000bf0 con 0x7f76340692f0 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (5m) 62s ago 6m 25.7M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (7m) 62s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (6m) 68s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (5m) 62s ago 6m 85.1M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (86s) 62s ago 86s 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (81s) 62s ago 81s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:51:25.610 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (79s) 68s ago 78s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (84s) 68s ago 83s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (8m) 62s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (5m) 68s ago 5m 432M - 17.2.7-902-gc61472c8 0de407c1b3af 7b0ffd125c63 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (8m) 62s ago 8m 44.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (5m) 68s ago 5m 35.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (6m) 62s ago 6m 14.3M - 1.5.0 0da6a335fe13 e44a82d54ec7 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (5m) 68s ago 5m 16.3M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (5m) 62s ago 5m 44.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (4m) 62s ago 4m 44.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (4m) 62s ago 4m 43.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (3m) 68s ago 3m 45.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (3m) 68s ago 3m 42.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (2m) 68s ago 2m 43.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:51:25.611 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (5m) 62s ago 6m 56.1M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:51:25.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.607+0000 7f763effd700 1 -- 172.21.15.77:0/2955830625 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f7628000bf0 con 0x7f76340692f0 2024-04-26T04:51:25.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 msgr2=0x7f763406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 0x7f763406b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7638004600 tx=0x7f763800b040 comp rx=0 tx=0).stop 2024-04-26T04:51:25.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 msgr2=0x7f764810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7630013870 tx=0x7f7630009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 shutdown_connections 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f76340692f0 0x7f763406b7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fe4a0 0x7f764810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 --2- 172.21.15.77:0/2955830625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fedf0 0x7f764810dec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.611+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 >> 172.21.15.77:0/2955830625 conn(0x7f76480f6b80 msgr2=0x7f76480f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.612+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 shutdown_connections 2024-04-26T04:51:25.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.612+0000 7f764cb3c700 1 -- 172.21.15.77:0/2955830625 wait complete. 2024-04-26T04:51:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 -- 172.21.15.77:0/2967549103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 msgr2=0x7f30380fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 --2- 172.21.15.77:0/2967549103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f30380fc0c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3028004880 tx=0x7f3028009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 -- 172.21.15.77:0/2967549103 shutdown_connections 2024-04-26T04:51:25.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 --2- 172.21.15.77:0/2967549103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f30380fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 --2- 172.21.15.77:0/2967549103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f30380fc0c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 -- 172.21.15.77:0/2967549103 >> 172.21.15.77:0/2967549103 conn(0x7f30380f7430 msgr2=0x7f30380f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 -- 172.21.15.77:0/2967549103 shutdown_connections 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.761+0000 7f303ccf0700 1 -- 172.21.15.77:0/2967549103 wait complete. 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.762+0000 7f303ccf0700 1 Processor -- start 2024-04-26T04:51:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.762+0000 7f303ccf0700 1 -- start start 2024-04-26T04:51:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303ccf0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303ccf0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f3038110a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303ccf0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038066390 con 0x7f30380fbcb0 2024-04-26T04:51:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303ccf0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380664d0 con 0x7f30380fcf60 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56224/0 (socket says 172.21.15.77:56224) 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f303659c700 1 -- 172.21.15.77:0/1745173180 learned_addr learned my addr 172.21.15.77:0/1745173180 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.763+0000 7f3035d9b700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f3038110a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f303659c700 1 -- 172.21.15.77:0/1745173180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 msgr2=0x7f3038110a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f303659c700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f3038110a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f303659c700 1 -- 172.21.15.77:0/1745173180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028005040 con 0x7f30380fbcb0 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f3035d9b700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f3038110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:51:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f303659c700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3028008fb0 tx=0x7f3028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.764+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302800e030 con 0x7f30380fbcb0 2024-04-26T04:51:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.765+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3028009430 con 0x7f30380fbcb0 2024-04-26T04:51:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.765+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3038110fa0 con 0x7f30380fbcb0 2024-04-26T04:51:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.765+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30280123c0 con 0x7f30380fbcb0 2024-04-26T04:51:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.765+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3038111560 con 0x7f30380fbcb0 2024-04-26T04:51:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.766+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303805f5c0 con 0x7f30380fbcb0 2024-04-26T04:51:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.767+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3028013b70 con 0x7f30380fbcb0 2024-04-26T04:51:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.767+0000 7f3026ffd700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 0x7f302006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.768+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3028088b20 con 0x7f30380fbcb0 2024-04-26T04:51:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.768+0000 7f3035d9b700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 0x7f302006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.769+0000 7f3035d9b700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 0x7f302006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f302c0043d0 tx=0x7f302c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:25.776+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3028054df0 con 0x7f30380fbcb0 2024-04-26T04:51:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:25 smithi077 ceph-mon[26311]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:25 smithi077 ceph-mon[26311]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.262+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30381047a0 con 0x7f30380fbcb0 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.263+0000 7f3026ffd700 1 -- 172.21.15.77:0/1745173180 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f3028058410 con 0x7f30380fbcb0 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:51:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:51:26.266 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:51:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:25 smithi078 ceph-mon[33746]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:25 smithi078 ceph-mon[33746]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:26.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.267+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 msgr2=0x7f302006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.267+0000 7f303ccf0700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 0x7f302006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f302c0043d0 tx=0x7f302c00b040 comp rx=0 tx=0).stop 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.267+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 msgr2=0x7f303810e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.267+0000 7f303ccf0700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3028008fb0 tx=0x7f3028009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 shutdown_connections 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f30200692a0 0x7f302006b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30380fbcb0 0x7f303810e4b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 --2- 172.21.15.77:0/1745173180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30380fcf60 0x7f3038110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 >> 172.21.15.77:0/1745173180 conn(0x7f30380f7430 msgr2=0x7f30380f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:26.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 shutdown_connections 2024-04-26T04:51:26.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.268+0000 7f303ccf0700 1 -- 172.21.15.77:0/1745173180 wait complete. 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 -- 172.21.15.77:0/887761420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 msgr2=0x7f8e400fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 --2- 172.21.15.77:0/887761420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e400fe7f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8e30004940 tx=0x7f8e30009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 -- 172.21.15.77:0/887761420 shutdown_connections 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 --2- 172.21.15.77:0/887761420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e400fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 --2- 172.21.15.77:0/887761420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e400fe7f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 -- 172.21.15.77:0/887761420 >> 172.21.15.77:0/887761420 conn(0x7f8e400f6b80 msgr2=0x7f8e400f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 -- 172.21.15.77:0/887761420 shutdown_connections 2024-04-26T04:51:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.424+0000 7f8e44907700 1 -- 172.21.15.77:0/887761420 wait complete. 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 Processor -- start 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 -- start start 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e4010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4010e4b0 con 0x7f8e400fe3e0 2024-04-26T04:51:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e44907700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4010ed00 con 0x7f8e400fed30 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e3e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e3e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56246/0 (socket says 172.21.15.77:56246) 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e3e59c700 1 -- 172.21.15.77:0/38442218 learned_addr learned my addr 172.21.15.77:0/38442218 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.425+0000 7f8e3dd9b700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e4010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e3e59c700 1 -- 172.21.15.77:0/38442218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 msgr2=0x7f8e4010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e3e59c700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e4010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e3e59c700 1 -- 172.21.15.77:0/38442218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30005040 con 0x7f8e400fe3e0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e3dd9b700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e4010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e3e59c700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b0d0 tx=0x7f8e30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3000e030 con 0x7f8e400fe3e0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4010ef80 con 0x7f8e400fe3e0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e3001c400 con 0x7f8e400fe3e0 2024-04-26T04:51:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3000b630 con 0x7f8e400fe3e0 2024-04-26T04:51:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.426+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4010f4c0 con 0x7f8e400fe3e0 2024-04-26T04:51:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.427+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4005f5c0 con 0x7f8e400fe3e0 2024-04-26T04:51:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.430+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8e3001c570 con 0x7f8e400fe3e0 2024-04-26T04:51:26.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.430+0000 7f8e2effd700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 0x7f8e2806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.430+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8e30088ff0 con 0x7f8e400fe3e0 2024-04-26T04:51:26.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.431+0000 7f8e3dd9b700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 0x7f8e2806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.432+0000 7f8e3dd9b700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 0x7f8e2806b7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009b80 tx=0x7f8e3400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:26.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.434+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8e30058d90 con 0x7f8e400fe3e0 2024-04-26T04:51:26.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.824+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e40105890 con 0x7f8e400fe3e0 2024-04-26T04:51:26.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.824+0000 7f8e2effd700 1 -- 172.21.15.77:0/38442218 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f8e3001c820 con 0x7f8e400fe3e0 2024-04-26T04:51:26.826 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:51:26.826 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:51:26.826 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:51:26.827 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:51:26.828 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:26.829 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 msgr2=0x7f8e2806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 0x7f8e2806b7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009b80 tx=0x7f8e3400b040 comp rx=0 tx=0).stop 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 msgr2=0x7f8e4010d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b0d0 tx=0x7f8e30009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 shutdown_connections 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f8e280692f0 0x7f8e2806b7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e400fe3e0 0x7f8e4010d970 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 --2- 172.21.15.77:0/38442218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e400fed30 0x7f8e4010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 >> 172.21.15.77:0/38442218 conn(0x7f8e400f6b80 msgr2=0x7f8e400f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.827+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 shutdown_connections 2024-04-26T04:51:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.828+0000 7f8e44907700 1 -- 172.21.15.77:0/38442218 wait complete. 2024-04-26T04:51:26.831 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:51:26.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/918357964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 msgr2=0x7fe9e40fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/918357964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e40fc6c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00049c0 tx=0x7fe9e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:26.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/918357964 shutdown_connections 2024-04-26T04:51:26.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/918357964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e40fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/918357964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e40fc6c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/918357964 >> 172.21.15.77:0/918357964 conn(0x7fe9e40f6b80 msgr2=0x7fe9e40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:26.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/918357964 shutdown_connections 2024-04-26T04:51:26.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.978+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/918357964 wait complete. 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.979+0000 7fe9eb5a5700 1 Processor -- start 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.979+0000 7fe9eb5a5700 1 -- start start 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9eb5a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e410b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9eb5a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9eb5a5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e410c380 con 0x7fe9e40fc2b0 2024-04-26T04:51:26.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9eb5a5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e410cbd0 con 0x7fe9e40fcc00 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e9341700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e410b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:38068/0 (socket says 172.21.15.77:38068) 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 -- 172.21.15.77:0/3747279065 learned_addr learned my addr 172.21.15.77:0/3747279065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 -- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 msgr2=0x7fe9e410b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e410b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e8b40700 1 -- 172.21.15.77:0/3747279065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e0005040 con 0x7fe9e40fcc00 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.980+0000 7fe9e9341700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e410b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:51:26.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9e8b40700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4010140 tx=0x7fe9d400f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9d4010d30 con 0x7fe9e40fcc00 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e410ce50 con 0x7fe9e40fcc00 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9d4013040 con 0x7fe9e40fcc00 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9d40177e0 con 0x7fe9e40fcc00 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.981+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e410d390 con 0x7fe9e40fcc00 2024-04-26T04:51:26.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.982+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e405f5c0 con 0x7fe9e40fcc00 2024-04-26T04:51:26.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.983+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe9d4018480 con 0x7fe9e40fcc00 2024-04-26T04:51:26.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.983+0000 7fe9d9ffb700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 0x7fe9d006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:26.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.983+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe9d408b2a0 con 0x7fe9e40fcc00 2024-04-26T04:51:26.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.984+0000 7fe9e9341700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 0x7fe9d006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:26.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.984+0000 7fe9e9341700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 0x7fe9d006b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015040 tx=0x7fe9e0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:26.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:26.992+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe9d4052250 con 0x7fe9e40fcc00 2024-04-26T04:51:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:26 smithi077 ceph-mon[26311]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:26 smithi077 ceph-mon[26311]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:27.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:26 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1745173180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:27.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:26 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/38442218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:26 smithi078 ceph-mon[33746]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:26 smithi078 ceph-mon[33746]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:26 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1745173180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:26 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/38442218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:27.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.300+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9e410d7c0 con 0x7fe9d00692b0 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.301+0000 7fe9d9ffb700 1 -- 172.21.15.77:0/3747279065 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe9e410d7c0 con 0x7fe9d00692b0 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "", 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a image", 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:51:27.303 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:51:27.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 msgr2=0x7fe9d006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 0x7fe9d006b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015040 tx=0x7fe9e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 msgr2=0x7fe9e410bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4010140 tx=0x7fe9d400f040 comp rx=0 tx=0).stop 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 shutdown_connections 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7fe9d00692b0 0x7fe9d006b760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e40fc2b0 0x7fe9e410b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 --2- 172.21.15.77:0/3747279065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9e40fcc00 0x7fe9e410bd80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.304+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 >> 172.21.15.77:0/3747279065 conn(0x7fe9e40f6b80 msgr2=0x7fe9e40f8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.305+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 shutdown_connections 2024-04-26T04:51:27.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.305+0000 7fe9eb5a5700 1 -- 172.21.15.77:0/3747279065 wait complete. 2024-04-26T04:51:27.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 -- 172.21.15.77:0/761669880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fbcb0 msgr2=0x7f28040fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 --2- 172.21.15.77:0/761669880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fbcb0 0x7f28040fc0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004940 tx=0x7f27f8009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:27.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 -- 172.21.15.77:0/761669880 shutdown_connections 2024-04-26T04:51:27.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 --2- 172.21.15.77:0/761669880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fcf60 0x7f28040fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 --2- 172.21.15.77:0/761669880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fbcb0 0x7f28040fc0c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 -- 172.21.15.77:0/761669880 >> 172.21.15.77:0/761669880 conn(0x7f28040f7430 msgr2=0x7f28040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:27.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 -- 172.21.15.77:0/761669880 shutdown_connections 2024-04-26T04:51:27.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.440+0000 7f280a7d1700 1 -- 172.21.15.77:0/761669880 wait complete. 2024-04-26T04:51:27.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 Processor -- start 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 -- start start 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 0x7f2804114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804114b50 con 0x7f28040fcf60 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.441+0000 7f280a7d1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28041153a0 con 0x7f28040fbcb0 2024-04-26T04:51:27.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2803fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 0x7f2804114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2803fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 0x7f2804114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:38088/0 (socket says 172.21.15.77:38088) 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56286/0 (socket says 172.21.15.77:56286) 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2803fff700 1 -- 172.21.15.77:0/704475515 learned_addr learned my addr 172.21.15.77:0/704475515 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 -- 172.21.15.77:0/704475515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 msgr2=0x7f2804114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 0x7f2804114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 -- 172.21.15.77:0/704475515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8005040 con 0x7f28040fcf60 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f28037fe700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f27f4002820 tx=0x7f27f4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f4007cc0 con 0x7f28040fcf60 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27f400f040 con 0x7f28040fcf60 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f4013920 con 0x7f28040fcf60 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2804115680 con 0x7f28040fcf60 2024-04-26T04:51:27.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.442+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804115bc0 con 0x7f28040fcf60 2024-04-26T04:51:27.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.444+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280405f5c0 con 0x7f28040fcf60 2024-04-26T04:51:27.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.444+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f27f401c400 con 0x7f28040fcf60 2024-04-26T04:51:27.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.445+0000 7f2800ff9700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 0x7f27ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:27.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.445+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f27f4086e00 con 0x7f28040fcf60 2024-04-26T04:51:27.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.445+0000 7f2803fff700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 0x7f27ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:27.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.445+0000 7f2803fff700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 0x7f27ec06b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004f90 tx=0x7f27f8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:27.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.452+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f27f4056bc0 con 0x7f28040fcf60 2024-04-26T04:51:27.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.905+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f280404bed0 con 0x7f28040fcf60 2024-04-26T04:51:27.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.905+0000 7f2800ff9700 1 -- 172.21.15.77:0/704475515 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f27f4056750 con 0x7f28040fcf60 2024-04-26T04:51:27.907 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:51:27.907 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:51:27.907 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:51:27.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 msgr2=0x7f27ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 0x7f27ec06b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27f8004f90 tx=0x7f27f8009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 msgr2=0x7f2804114550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f27f4002820 tx=0x7f27f4004570 comp rx=0 tx=0).stop 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 shutdown_connections 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f27ec0692a0 0x7f27ec06b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28040fbcb0 0x7f2804114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 --2- 172.21.15.77:0/704475515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28040fcf60 0x7f2804114550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:27.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.908+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 >> 172.21.15.77:0/704475515 conn(0x7f28040f7430 msgr2=0x7f28040f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:27.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.909+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 shutdown_connections 2024-04-26T04:51:27.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:27.909+0000 7f280a7d1700 1 -- 172.21.15.77:0/704475515 wait complete. 2024-04-26T04:51:28.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:27 smithi077 ceph-mon[26311]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:27 smithi078 ceph-mon[33746]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:28 smithi077 ceph-mon[26311]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:29.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:28 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/704475515' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:28 smithi078 ceph-mon[33746]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:28 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/704475515' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:30.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:29 smithi077 ceph-mon[26311]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:29 smithi078 ceph-mon[33746]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:31 smithi077 ceph-mon[26311]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:31 smithi078 ceph-mon[33746]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:33 smithi077 ceph-mon[26311]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:33 smithi078 ceph-mon[33746]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:35.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (unknown) 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: Upgrade: Need to upgrade myself (mgr.smithi077.hkvsor) 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:35.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:35 smithi077 ceph-mon[26311]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 on smithi078 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (unknown) 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: Upgrade: Need to upgrade myself (mgr.smithi077.hkvsor) 2024-04-26T04:51:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:35.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:35 smithi078 ceph-mon[33746]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 on smithi078 2024-04-26T04:51:37.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:37 smithi077 ceph-mon[26311]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:37 smithi078 ceph-mon[33746]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:38.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:38 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:39.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:38 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:39.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:39 smithi077 ceph-mon[26311]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:39 smithi078 ceph-mon[33746]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:41.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:41 smithi077 ceph-mon[26311]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:41 smithi078 ceph-mon[33746]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:43.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:43 smithi077 ceph-mon[26311]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:43 smithi078 ceph-mon[33746]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:45.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:45 smithi077 ceph-mon[26311]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:46.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:45 smithi078 ceph-mon[33746]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:47.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:47 smithi077 ceph-mon[26311]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:48.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:47 smithi078 ceph-mon[33746]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:49.886 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:49 smithi077 ceph-mon[26311]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:49 smithi078 ceph-mon[33746]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:51.176 INFO:tasks.workunit.client.1.smithi078.stderr:Updating files: 94% (11759/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'. 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.292 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-04-26T04:51:51.293 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at b22e2ebdeb2 Merge pull request #57078 from adk3798/squid-cephadm-no-downgrade 2024-04-26T04:51:51.296 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T04:51:51.321 INFO:tasks.workunit.client.0.smithi077.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T04:51:51.323 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T04:51:51.323 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:Note: switching to 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'. 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr: git switch -c 2024-04-26T04:51:51.494 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr:Or undo this operation with: 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr: git switch - 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr: 2024-04-26T04:51:51.495 INFO:tasks.workunit.client.1.smithi078.stderr:HEAD is now at b22e2ebdeb2 Merge pull request #57078 from adk3798/squid-cephadm-no-downgrade 2024-04-26T04:51:51.497 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-26T04:51:51.563 INFO:tasks.workunit.client.1.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T04:51:51.566 INFO:tasks.workunit.client.1.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T04:51:51.566 INFO:tasks.workunit.client.1.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T04:51:51.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:51 smithi077 ceph-mon[26311]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:51.983 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T04:51:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:51 smithi078 ceph-mon[33746]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:52.035 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T04:51:52.074 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T04:51:52.076 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T04:51:52.076 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T04:51:52.102 INFO:tasks.workunit.client.1.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T04:51:52.114 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T04:51:52.117 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T04:51:52.117 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T04:51:52.156 INFO:tasks.workunit.client.1.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T04:51:52.173 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-26T04:51:52.175 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-26T04:51:52.175 DEBUG:teuthology.orchestra.run.smithi077:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b22e2ebdeb24376882b7bda2a7329c8cccc2276a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-26T04:51:52.196 INFO:tasks.workunit.client.1.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T04:51:52.197 INFO:tasks.workunit.client.1.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T04:51:52.197 INFO:tasks.workunit.client.1.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T04:51:52.237 INFO:tasks.workunit.client.0.smithi077.stderr:+ mkdir -p fsstress 2024-04-26T04:51:52.239 INFO:tasks.workunit.client.0.smithi077.stderr:+ pushd fsstress 2024-04-26T04:51:52.240 INFO:tasks.workunit.client.0.smithi077.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:51:52.240 INFO:tasks.workunit.client.0.smithi077.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-26T04:51:52.240 INFO:tasks.workunit.client.1.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T04:51:52.243 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-26T04:51:52.243 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-26T04:51:52.300 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-26T04:51:52.301 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-26T04:51:52.301 DEBUG:teuthology.orchestra.run.smithi078:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b22e2ebdeb24376882b7bda2a7329c8cccc2276a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-26T04:51:52.365 INFO:tasks.workunit.client.1.smithi078.stderr:+ mkdir -p fsstress 2024-04-26T04:51:52.366 INFO:tasks.workunit.client.1.smithi078.stderr:+ pushd fsstress 2024-04-26T04:51:52.368 INFO:tasks.workunit.client.1.smithi078.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:51:52.368 INFO:tasks.workunit.client.1.smithi078.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-26T04:51:52.938 INFO:tasks.workunit.client.1.smithi078.stderr:+ tar xzf ltp-full.tgz 2024-04-26T04:51:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:53 smithi077 ceph-mon[26311]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:53 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:53 smithi078 ceph-mon[33746]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:53 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:54.306 INFO:tasks.workunit.client.0.smithi077.stderr:+ tar xzf ltp-full.tgz 2024-04-26T04:51:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:55 smithi077 ceph-mon[26311]: pgmap v237: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 563 KiB/s wr, 4 op/s 2024-04-26T04:51:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:55 smithi078 ceph-mon[33746]: pgmap v237: 65 pgs: 65 active+clean; 7.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 563 KiB/s wr, 4 op/s 2024-04-26T04:51:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:57 smithi077 ceph-mon[26311]: pgmap v238: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 40 op/s 2024-04-26T04:51:58.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:57 smithi078 ceph-mon[33746]: pgmap v238: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 40 op/s 2024-04-26T04:51:58.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 -- 172.21.15.77:0/2479398753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fedf0 msgr2=0x7f38280faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 --2- 172.21.15.77:0/2479398753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fedf0 0x7f38280faf30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f381c004940 tx=0x7f381c009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 -- 172.21.15.77:0/2479398753 shutdown_connections 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 --2- 172.21.15.77:0/2479398753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fedf0 0x7f38280faf30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 --2- 172.21.15.77:0/2479398753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fe4a0 0x7f38280fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 -- 172.21.15.77:0/2479398753 >> 172.21.15.77:0/2479398753 conn(0x7f38280f6b80 msgr2=0x7f38280f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 -- 172.21.15.77:0/2479398753 shutdown_connections 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.061+0000 7f382ca54700 1 -- 172.21.15.77:0/2479398753 wait complete. 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 Processor -- start 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 -- start start 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 0x7f38280fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38280fc710 con 0x7f38280fe4a0 2024-04-26T04:51:58.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f382ca54700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38280fc880 con 0x7f38280fedf0 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.062+0000 7f3825d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41342/0 (socket says 172.21.15.77:41342) 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 -- 172.21.15.77:0/213283701 learned_addr learned my addr 172.21.15.77:0/213283701 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f382659c700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 0x7f38280fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 -- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 msgr2=0x7f38280fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 0x7f38280fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 -- 172.21.15.77:0/213283701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c005040 con 0x7f38280fedf0 2024-04-26T04:51:58.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f382659c700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 0x7f38280fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3825d9b700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b0d0 tx=0x7f381c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381c00e030 con 0x7f38280fedf0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f381c01c400 con 0x7f38280fedf0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381c00b630 con 0x7f38280fedf0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.063+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38280fcb00 con 0x7f38280fedf0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.064+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38280fcfb0 con 0x7f38280fedf0 2024-04-26T04:51:58.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.065+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382805f5c0 con 0x7f38280fedf0 2024-04-26T04:51:58.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.066+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f381c01c570 con 0x7f38280fedf0 2024-04-26T04:51:58.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.066+0000 7f3816ffd700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 0x7f381006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.066+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f381c088bf0 con 0x7f38280fedf0 2024-04-26T04:51:58.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.066+0000 7f382659c700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 0x7f381006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.067+0000 7f382659c700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 0x7f381006b7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38280fda20 tx=0x7f381800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:58.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.071+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f381c054e80 con 0x7f38280fedf0 2024-04-26T04:51:58.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.386+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38281024f0 con 0x7f38100692f0 2024-04-26T04:51:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.388+0000 7f3816ffd700 1 -- 172.21.15.77:0/213283701 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f38281024f0 con 0x7f38100692f0 2024-04-26T04:51:58.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 msgr2=0x7f381006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 0x7f381006b7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38280fda20 tx=0x7f381800b040 comp rx=0 tx=0).stop 2024-04-26T04:51:58.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 msgr2=0x7f38280fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b0d0 tx=0x7f381c009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 shutdown_connections 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692f0 0x7f381006b7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38280fe4a0 0x7f38280fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 --2- 172.21.15.77:0/213283701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38280fedf0 0x7f38280fc160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.390+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 >> 172.21.15.77:0/213283701 conn(0x7f38280f6b80 msgr2=0x7f38280f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.391+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 shutdown_connections 2024-04-26T04:51:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.391+0000 7f382ca54700 1 -- 172.21.15.77:0/213283701 wait complete. 2024-04-26T04:51:58.404 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:51:58.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 -- 172.21.15.77:0/1648692137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fce60 msgr2=0x7ff3800fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 --2- 172.21.15.77:0/1648692137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fce60 0x7ff3800fd270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff370004940 tx=0x7ff370009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 -- 172.21.15.77:0/1648692137 shutdown_connections 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 --2- 172.21.15.77:0/1648692137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fdfd0 0x7ff3800fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 --2- 172.21.15.77:0/1648692137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fce60 0x7ff3800fd270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 -- 172.21.15.77:0/1648692137 >> 172.21.15.77:0/1648692137 conn(0x7ff3800f8500 msgr2=0x7ff3800fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 -- 172.21.15.77:0/1648692137 shutdown_connections 2024-04-26T04:51:58.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 -- 172.21.15.77:0/1648692137 wait complete. 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.525+0000 7ff3851b2700 1 Processor -- start 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff3851b2700 1 -- start start 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff3851b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 0x7ff38010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff3851b2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff3851b2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38010e520 con 0x7ff3800fdfd0 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff3851b2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38010ed70 con 0x7ff3800fce60 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 0x7ff38010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 0x7ff38010d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41356/0 (socket says 172.21.15.77:41356) 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51010/0 (socket says 172.21.15.77:51010) 2024-04-26T04:51:58.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37ed9d700 1 -- 172.21.15.77:0/2290601233 learned_addr learned my addr 172.21.15.77:0/2290601233 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37e59c700 1 -- 172.21.15.77:0/2290601233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 msgr2=0x7ff38010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37e59c700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 0x7ff38010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.526+0000 7ff37e59c700 1 -- 172.21.15.77:0/2290601233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff370005040 con 0x7ff3800fdfd0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff37e59c700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff374009fd0 tx=0x7ff37400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff374004e70 con 0x7ff3800fdfd0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff37400f040 con 0x7ff3800fdfd0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37400c980 con 0x7ff3800fdfd0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38010eff0 con 0x7ff3800fdfd0 2024-04-26T04:51:58.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.527+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38010f530 con 0x7ff3800fdfd0 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.528+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38005f5c0 con 0x7ff3800fdfd0 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.528+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff374007cc0 con 0x7ff3800fdfd0 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.529+0000 7ff3677fe700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 0x7ff36806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.529+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff374086c90 con 0x7ff3800fdfd0 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.529+0000 7ff37ed9d700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 0x7ff36806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:58.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.529+0000 7ff37ed9d700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 0x7ff36806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff370009b30 tx=0x7ff370009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:58.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.535+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff374056a50 con 0x7ff3800fdfd0 2024-04-26T04:51:58.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.862+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff38010f7a0 con 0x7ff3680692a0 2024-04-26T04:51:58.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.863+0000 7ff3677fe700 1 -- 172.21.15.77:0/2290601233 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7ff38010f7a0 con 0x7ff3680692a0 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 msgr2=0x7ff36806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 0x7ff36806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff370009b30 tx=0x7ff370009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 msgr2=0x7ff38010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff374009fd0 tx=0x7ff37400b040 comp rx=0 tx=0).stop 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 shutdown_connections 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff3680692a0 0x7ff36806b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3800fce60 0x7ff38010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 --2- 172.21.15.77:0/2290601233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3800fdfd0 0x7ff38010df20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 >> 172.21.15.77:0/2290601233 conn(0x7ff3800f8500 msgr2=0x7ff3800f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 shutdown_connections 2024-04-26T04:51:58.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:58.866+0000 7ff3851b2700 1 -- 172.21.15.77:0/2290601233 wait complete. 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 -- 172.21.15.77:0/3030220964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 msgr2=0x7f09200ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3030220964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f09200ffa40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0910004940 tx=0x7f0910009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 -- 172.21.15.77:0/3030220964 shutdown_connections 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3030220964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f0920102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3030220964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f09200ffa40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 -- 172.21.15.77:0/3030220964 >> 172.21.15.77:0/3030220964 conn(0x7f09200f7430 msgr2=0x7f09200f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 -- 172.21.15.77:0/3030220964 shutdown_connections 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.012+0000 7f0924a2f700 1 -- 172.21.15.77:0/3030220964 wait complete. 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 Processor -- start 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 -- start start 2024-04-26T04:51:59.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f092010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092010e540 con 0x7f0920066220 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f0924a2f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092010ed40 con 0x7f09200fff80 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f091e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f091e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51040/0 (socket says 172.21.15.77:51040) 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f091e59c700 1 -- 172.21.15.77:0/3871790390 learned_addr learned my addr 172.21.15.77:0/3871790390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:59.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.013+0000 7f091dd9b700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f092010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f091dd9b700 1 -- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 msgr2=0x7f092010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f091dd9b700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f091dd9b700 1 -- 172.21.15.77:0/3871790390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0910005040 con 0x7f09200fff80 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f091e59c700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f091dd9b700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f092010df40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f091400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0914004e90 con 0x7f09200fff80 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092010efc0 con 0x7f09200fff80 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f091400f040 con 0x7f09200fff80 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091400ca50 con 0x7f09200fff80 2024-04-26T04:51:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.014+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092010f500 con 0x7f09200fff80 2024-04-26T04:51:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.015+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092005f5c0 con 0x7f09200fff80 2024-04-26T04:51:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.016+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f091400cc00 con 0x7f09200fff80 2024-04-26T04:51:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.016+0000 7f090effd700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 0x7f090806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.016+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0914086be0 con 0x7f09200fff80 2024-04-26T04:51:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.016+0000 7f091e59c700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 0x7f090806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.017+0000 7f091e59c700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 0x7f090806b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09100136a0 tx=0x7f0910009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.022+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f091404e250 con 0x7f09200fff80 2024-04-26T04:51:59.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.314+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f092010f7e0 con 0x7f09080692b0 2024-04-26T04:51:59.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.331+0000 7f090effd700 1 -- 172.21.15.77:0/3871790390 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f092010f7e0 con 0x7f09080692b0 2024-04-26T04:51:59.332 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (6m) 95s ago 7m 25.7M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (7m) 95s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (6m) 102s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (5m) 95s ago 7m 85.1M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (2m) 95s ago 2m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (115s) 95s ago 114s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (112s) 102s ago 112s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (118s) 102s ago 117s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (8m) 95s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (6m) 102s ago 6m 432M - 17.2.7-902-gc61472c8 0de407c1b3af 7b0ffd125c63 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (8m) 95s ago 8m 44.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (6m) 102s ago 6m 35.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (7m) 95s ago 7m 14.3M - 1.5.0 0da6a335fe13 e44a82d54ec7 2024-04-26T04:51:59.333 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (6m) 102s ago 6m 16.3M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (5m) 95s ago 5m 44.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (5m) 95s ago 5m 44.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (4m) 95s ago 4m 43.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (4m) 102s ago 4m 45.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (3m) 102s ago 3m 42.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (3m) 102s ago 3m 43.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:51:59.334 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (5m) 95s ago 6m 56.1M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 msgr2=0x7f090806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 0x7f090806b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09100136a0 tx=0x7f0910009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 msgr2=0x7f092010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f092010df40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0914009fd0 tx=0x7f091400b040 comp rx=0 tx=0).stop 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 shutdown_connections 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f09080692b0 0x7f090806b760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0920066220 0x7f092010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 --2- 172.21.15.77:0/3871790390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09200fff80 0x7f092010df40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 >> 172.21.15.77:0/3871790390 conn(0x7f09200f7430 msgr2=0x7f09200f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 shutdown_connections 2024-04-26T04:51:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.334+0000 7f0924a2f700 1 -- 172.21.15.77:0/3871790390 wait complete. 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.478+0000 7ff6025c7700 1 -- 172.21.15.77:0/676083871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fce60 msgr2=0x7ff5fc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.478+0000 7ff6025c7700 1 --2- 172.21.15.77:0/676083871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc0fd270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec004940 tx=0x7ff5ec009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 -- 172.21.15.77:0/676083871 shutdown_connections 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 --2- 172.21.15.77:0/676083871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 --2- 172.21.15.77:0/676083871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc0fd270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 -- 172.21.15.77:0/676083871 >> 172.21.15.77:0/676083871 conn(0x7ff5fc0f8540 msgr2=0x7ff5fc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 -- 172.21.15.77:0/676083871 shutdown_connections 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 -- 172.21.15.77:0/676083871 wait complete. 2024-04-26T04:51:59.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.479+0000 7ff6025c7700 1 Processor -- start 2024-04-26T04:51:59.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff6025c7700 1 -- start start 2024-04-26T04:51:59.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff6025c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff6025c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff6025c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc070860 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff6025c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0709d0 con 0x7ff5fc0fce60 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff5fb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff5fbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff5fb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51054/0 (socket says 172.21.15.77:51054) 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.480+0000 7ff5fb7fe700 1 -- 172.21.15.77:0/4055701176 learned_addr learned my addr 172.21.15.77:0/4055701176 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5fb7fe700 1 -- 172.21.15.77:0/4055701176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fce60 msgr2=0x7ff5fc071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5fb7fe700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5fb7fe700 1 -- 172.21.15.77:0/4055701176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec005040 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5fb7fe700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0002a80 tx=0x7ff5f00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f0007cc0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc070cb0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f000f040 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f000dac0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.481+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc1159c0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.482+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc05f5c0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.483+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff5f000dcc0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.484+0000 7ff5f8ff9700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 0x7ff5e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:59.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.484+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff5f0086e20 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.484+0000 7ff5fbfff700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 0x7ff5e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:59.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.484+0000 7ff5fbfff700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 0x7ff5e406b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec004890 tx=0x7ff5ec009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:59.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.490+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff5f00530b0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:59 smithi077 ceph-mon[26311]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:59.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:59 smithi077 ceph-mon[26311]: pgmap v239: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 47 op/s 2024-04-26T04:51:59.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:51:59 smithi077 ceph-mon[26311]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:59.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.890+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5fc115ce0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.891+0000 7ff5f8ff9700 1 -- 172.21.15.77:0/4055701176 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7ff5f00566d0 con 0x7ff5fc0fdfd0 2024-04-26T04:51:59.891 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:51:59.892 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-26T04:51:59.893 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:51:59.893 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 msgr2=0x7ff5e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 0x7ff5e406b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec004890 tx=0x7ff5ec009e30 comp rx=0 tx=0).stop 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 msgr2=0x7ff5fc0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0002a80 tx=0x7ff5f00046b0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 shutdown_connections 2024-04-26T04:51:59.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7ff5e40692a0 0x7ff5e406b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5fc0fce60 0x7ff5fc071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 --2- 172.21.15.77:0/4055701176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5fc0fdfd0 0x7ff5fc0702b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:59.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 >> 172.21.15.77:0/4055701176 conn(0x7ff5fc0f8540 msgr2=0x7ff5fc0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:59.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.893+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 shutdown_connections 2024-04-26T04:51:59.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:51:59.894+0000 7ff6025c7700 1 -- 172.21.15.77:0/4055701176 wait complete. 2024-04-26T04:52:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:59 smithi078 ceph-mon[33746]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:59 smithi078 ceph-mon[33746]: pgmap v239: 65 pgs: 65 active+clean; 25 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2.1 MiB/s wr, 47 op/s 2024-04-26T04:52:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:51:59 smithi078 ceph-mon[33746]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.027+0000 7f50a4e38700 1 -- 172.21.15.77:0/3179598701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fe090 msgr2=0x7f50a00fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.027+0000 7f50a4e38700 1 --2- 172.21.15.77:0/3179598701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fe090 0x7f50a00fe500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5094004940 tx=0x7f5094009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 -- 172.21.15.77:0/3179598701 shutdown_connections 2024-04-26T04:52:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 --2- 172.21.15.77:0/3179598701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fe090 0x7f50a00fe500 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 --2- 172.21.15.77:0/3179598701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fce90 0x7f50a00fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 -- 172.21.15.77:0/3179598701 >> 172.21.15.77:0/3179598701 conn(0x7f50a00f85c0 msgr2=0x7f50a00fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 -- 172.21.15.77:0/3179598701 shutdown_connections 2024-04-26T04:52:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 -- 172.21.15.77:0/3179598701 wait complete. 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 Processor -- start 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.028+0000 7f50a4e38700 1 -- start start 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f50a4e38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f50a4e38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fe090 0x7f50a00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f50a4e38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0070860 con 0x7f50a00fce90 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f50a4e38700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00709d0 con 0x7f50a00fe090 2024-04-26T04:52:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51060/0 (socket says 172.21.15.77:51060) 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 -- 172.21.15.77:0/1783117879 learned_addr learned my addr 172.21.15.77:0/1783117879 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509dd9b700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fe090 0x7f50a00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 -- 172.21.15.77:0/1783117879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fe090 msgr2=0x7f50a00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fe090 0x7f50a00702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 -- 172.21.15.77:0/1783117879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094005040 con 0x7f50a00fce90 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.029+0000 7f509e59c700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f509000eb70 tx=0x7f509000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.030+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5090007cc0 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.030+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f509000f040 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.030+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0070c50 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.030+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509000cb20 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.030+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a01159e0 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.031+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a005f5c0 con 0x7f50a00fce90 2024-04-26T04:52:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.032+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f509000cc80 con 0x7f50a00fce90 2024-04-26T04:52:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.032+0000 7f508effd700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 0x7f508806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.032+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5090086b40 con 0x7f50a00fce90 2024-04-26T04:52:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.032+0000 7f509dd9b700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 0x7f508806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.033+0000 7f509dd9b700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 0x7f508806b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5094009b30 tx=0x7f5094009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.038+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f509004e250 con 0x7f50a00fce90 2024-04-26T04:52:00.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.354+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50a004bed0 con 0x7f50a00fce90 2024-04-26T04:52:00.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.354+0000 7f508effd700 1 -- 172.21.15.77:0/1783117879 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f5090019720 con 0x7f50a00fce90 2024-04-26T04:52:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:52:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:52:00.355 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:52:00.356 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:52:00.357 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:00.358 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:00.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 msgr2=0x7f508806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 0x7f508806b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5094009b30 tx=0x7f5094009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:00.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 msgr2=0x7f50a0071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f509000eb70 tx=0x7f509000b040 comp rx=0 tx=0).stop 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 shutdown_connections 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f50880692a0 0x7f508806b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50a00fce90 0x7f50a0071de0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 --2- 172.21.15.77:0/1783117879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50a00fe090 0x7f50a00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.357+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 >> 172.21.15.77:0/1783117879 conn(0x7f50a00f85c0 msgr2=0x7f50a00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.358+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 shutdown_connections 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.358+0000 7f50a4e38700 1 -- 172.21.15.77:0/1783117879 wait complete. 2024-04-26T04:52:00.360 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:52:00.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 -- 172.21.15.77:0/2160100420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38200fcbe0 msgr2=0x7f38200fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 --2- 172.21.15.77:0/2160100420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38200fcbe0 0x7f38200fd050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f380c004940 tx=0x7f380c009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 -- 172.21.15.77:0/2160100420 shutdown_connections 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 --2- 172.21.15.77:0/2160100420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38200fcbe0 0x7f38200fd050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 --2- 172.21.15.77:0/2160100420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f38200fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 -- 172.21.15.77:0/2160100420 >> 172.21.15.77:0/2160100420 conn(0x7f38200f6b80 msgr2=0x7f38200f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 -- 172.21.15.77:0/2160100420 shutdown_connections 2024-04-26T04:52:00.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.478+0000 7f3826e54700 1 -- 172.21.15.77:0/2160100420 wait complete. 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.479+0000 7f3826e54700 1 Processor -- start 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.479+0000 7f3826e54700 1 -- start start 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3826e54700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3826e54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 0x7f3820115510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3826e54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820114b80 con 0x7f38201146c0 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3826e54700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820114cc0 con 0x7f38200fc290 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41412/0 (socket says 172.21.15.77:41412) 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 -- 172.21.15.77:0/631563481 learned_addr learned my addr 172.21.15.77:0/631563481 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f381ffff700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 0x7f3820115510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 -- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 msgr2=0x7f3820115510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 0x7f3820115510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 -- 172.21.15.77:0/631563481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c005040 con 0x7f38200fc290 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f381ffff700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 0x7f3820115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.480+0000 7f3824bf0700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f381400eb70 tx=0x7f381400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.481+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3814007cc0 con 0x7f38200fc290 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.481+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f381400f040 con 0x7f38200fc290 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.481+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381400cb20 con 0x7f38200fc290 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.481+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3820115b10 con 0x7f38200fc290 2024-04-26T04:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.481+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38201160a0 con 0x7f38200fc290 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.482+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38201019e0 con 0x7f38200fc290 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.483+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f381400cc80 con 0x7f38200fc290 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.483+0000 7f381d7fa700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 0x7f381006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.483+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3814086b80 con 0x7f38200fc290 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.483+0000 7f381ffff700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 0x7f381006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.484+0000 7f381ffff700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 0x7f381006b760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f380c013870 tx=0x7f380c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.488+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f381404e250 con 0x7f38200fc290 2024-04-26T04:52:00.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:00 smithi077 ceph-mon[26311]: from='client.24375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:00.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:00 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/4055701176' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:00 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1783117879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:00.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.809+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 --> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38201047f0 con 0x7f38100692b0 2024-04-26T04:52:00.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.810+0000 7f381d7fa700 1 -- 172.21.15.77:0/631563481 <== mgr.14208 v2:172.21.15.77:6800/1184941417 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f38201047f0 con 0x7f38100692b0 2024-04-26T04:52:00.810 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "0/21 daemons upgraded", 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 image on host smithi078", 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:52:00.811 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 msgr2=0x7f381006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 0x7f381006b760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f380c013870 tx=0x7f380c009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 msgr2=0x7f3820114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f381400eb70 tx=0x7f381400b040 comp rx=0 tx=0).stop 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 shutdown_connections 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f38100692b0 0x7f381006b760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38200fc290 0x7f3820114180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 --2- 172.21.15.77:0/631563481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38201146c0 0x7f3820115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 >> 172.21.15.77:0/631563481 conn(0x7f38200f6b80 msgr2=0x7f38200fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 shutdown_connections 2024-04-26T04:52:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.813+0000 7f3826e54700 1 -- 172.21.15.77:0/631563481 wait complete. 2024-04-26T04:52:00.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 -- 172.21.15.77:0/2243978399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 msgr2=0x7f1f000ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 --2- 172.21.15.77:0/2243978399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f000ffa60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1efc004940 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 -- 172.21.15.77:0/2243978399 shutdown_connections 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 --2- 172.21.15.77:0/2243978399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 0x7f1f00102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 --2- 172.21.15.77:0/2243978399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f000ffa60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 -- 172.21.15.77:0/2243978399 >> 172.21.15.77:0/2243978399 conn(0x7f1f000f7430 msgr2=0x7f1f000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 -- 172.21.15.77:0/2243978399 shutdown_connections 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.948+0000 7f1f072b8700 1 -- 172.21.15.77:0/2243978399 wait complete. 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 Processor -- start 2024-04-26T04:52:00.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 -- start start 2024-04-26T04:52:00.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 0x7f1f001142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00114920 con 0x7f1f000fffa0 2024-04-26T04:52:00.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.949+0000 7f1f072b8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f00119270 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f04853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 0x7f1f001142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41430/0 (socket says 172.21.15.77:41430) 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 -- 172.21.15.77:0/1806055219 learned_addr learned my addr 172.21.15.77:0/1806055219 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 -- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 msgr2=0x7f1f001142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 0x7f1f001142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 -- 172.21.15.77:0/1806055219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc005040 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1f05054700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1efc004890 tx=0x7f1efc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.950+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1efc021070 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.951+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f001193b0 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.951+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1efc013d00 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.951+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1efc00e070 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.951+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f00119890 con 0x7f1f00066e10 2024-04-26T04:52:00.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.952+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f001060d0 con 0x7f1f00066e10 2024-04-26T04:52:00.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.952+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1efc01c9d0 con 0x7f1f00066e10 2024-04-26T04:52:00.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.953+0000 7f1ef5ffb700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 0x7f1eec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:00.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.953+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1efc0883d0 con 0x7f1f00066e10 2024-04-26T04:52:00.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.953+0000 7f1f04853700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 0x7f1eec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:00.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.953+0000 7f1f04853700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 0x7f1eec06b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009b80 tx=0x7f1ef000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:00.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:00.957+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1efc054790 con 0x7f1f00066e10 2024-04-26T04:52:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:00 smithi078 ceph-mon[33746]: from='client.24375 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:00 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/4055701176' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:00 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1783117879' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:01.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.335+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1f0004c710 con 0x7f1f00066e10 2024-04-26T04:52:01.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.335+0000 7f1ef5ffb700 1 -- 172.21.15.77:0/1806055219 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1efc057db0 con 0x7f1f00066e10 2024-04-26T04:52:01.336 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:52:01.336 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:52:01.336 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:52:01.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 msgr2=0x7f1eec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:01.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 0x7f1eec06b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009b80 tx=0x7f1ef000b040 comp rx=0 tx=0).stop 2024-04-26T04:52:01.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 msgr2=0x7f1f00113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:01.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1efc004890 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 shutdown_connections 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:6800/1184941417,v1:172.21.15.77:6801/1184941417] conn(0x7f1eec0692a0 0x7f1eec06b750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f00066e10 0x7f1f00113d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 --2- 172.21.15.77:0/1806055219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f000fffa0 0x7f1f001142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.337+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 >> 172.21.15.77:0/1806055219 conn(0x7f1f000f7430 msgr2=0x7f1f00100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.338+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 shutdown_connections 2024-04-26T04:52:01.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:01.338+0000 7f1f072b8700 1 -- 172.21.15.77:0/1806055219 wait complete. 2024-04-26T04:52:01.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:01 smithi077 ceph-mon[26311]: pgmap v240: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.1 MiB/s wr, 123 op/s 2024-04-26T04:52:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:01 smithi077 ceph-mon[26311]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:01 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1806055219' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:01 smithi078 ceph-mon[33746]: pgmap v240: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.1 MiB/s wr, 123 op/s 2024-04-26T04:52:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:01 smithi078 ceph-mon[33746]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:01 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1806055219' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:03.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:03 smithi078 ceph-mon[33746]: pgmap v241: 65 pgs: 65 active+clean; 38 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 130 op/s 2024-04-26T04:52:03.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:03 smithi077 ceph-mon[26311]: pgmap v241: 65 pgs: 65 active+clean; 38 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 130 op/s 2024-04-26T04:52:05.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: Upgrade: Updating mgr.smithi078.navvuw 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: Deploying daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:52:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:05 smithi077 ceph-mon[26311]: pgmap v242: 65 pgs: 65 active+clean; 38 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 137 op/s 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: Upgrade: Updating mgr.smithi078.navvuw 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: Deploying daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:52:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:05 smithi078 ceph-mon[33746]: pgmap v242: 65 pgs: 65 active+clean; 38 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 137 op/s 2024-04-26T04:52:07.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:07 smithi077 ceph-mon[26311]: pgmap v243: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 168 op/s 2024-04-26T04:52:08.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:07 smithi078 ceph-mon[33746]: pgmap v243: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 168 op/s 2024-04-26T04:52:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:08 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:09.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:08 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:09.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:09 smithi077 ceph-mon[26311]: pgmap v244: 65 pgs: 65 active+clean; 64 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 145 op/s 2024-04-26T04:52:10.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:09 smithi078 ceph-mon[33746]: pgmap v244: 65 pgs: 65 active+clean; 64 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 145 op/s 2024-04-26T04:52:11.870 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:11 smithi078 ceph-mon[33746]: pgmap v245: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.7 MiB/s wr, 242 op/s 2024-04-26T04:52:11.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:11 smithi077 ceph-mon[26311]: pgmap v245: 65 pgs: 65 active+clean; 92 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.7 MiB/s wr, 242 op/s 2024-04-26T04:52:13.727 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:13 smithi078 ceph-mon[33746]: pgmap v246: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 176 op/s 2024-04-26T04:52:13.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:13 smithi077 ceph-mon[26311]: pgmap v246: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 176 op/s 2024-04-26T04:52:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:15 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:15 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:16 smithi078 ceph-mon[33746]: pgmap v247: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 178 op/s 2024-04-26T04:52:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:16 smithi077 ceph-mon[26311]: pgmap v247: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 178 op/s 2024-04-26T04:52:18.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:17 smithi078 ceph-mon[33746]: pgmap v248: 65 pgs: 65 active+clean; 109 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 6.0 MiB/s wr, 224 op/s 2024-04-26T04:52:18.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.234 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:17 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:17 smithi077 ceph-mon[26311]: pgmap v248: 65 pgs: 65 active+clean; 109 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 6.0 MiB/s wr, 224 op/s 2024-04-26T04:52:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:17 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:20.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:19 smithi077 ceph-mon[26311]: pgmap v249: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 198 op/s 2024-04-26T04:52:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:19 smithi078 ceph-mon[33746]: pgmap v249: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 198 op/s 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: pgmap v250: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.1 MiB/s wr, 240 op/s 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:52:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:21 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: pgmap v250: 65 pgs: 65 active+clean; 124 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.1 MiB/s wr, 240 op/s 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:52:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:52:22.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:21 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:52:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:22 smithi078 ceph-mon[33746]: mgrmap e20: smithi077.hkvsor(active, since 7m), standbys: smithi078.navvuw 2024-04-26T04:52:23.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:22 smithi077 ceph-mon[26311]: mgrmap e20: smithi077.hkvsor(active, since 7m), standbys: smithi078.navvuw 2024-04-26T04:52:24.250 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:23 smithi078 ceph-mon[33746]: pgmap v251: 65 pgs: 65 active+clean; 125 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 151 op/s 2024-04-26T04:52:24.250 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:23 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:24.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:23 smithi077 ceph-mon[26311]: pgmap v251: 65 pgs: 65 active+clean; 125 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 151 op/s 2024-04-26T04:52:24.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:23 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:26.251 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:25 smithi078 ceph-mon[33746]: pgmap v252: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 160 op/s 2024-04-26T04:52:26.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:25 smithi077 ceph-mon[26311]: pgmap v252: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 160 op/s 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr fail", "who": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:52:27.632 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: Activating manager daemon smithi078.navvuw 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: osdmap e47: 6 total, 6 up, 6 in 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "mgr fail", "who": "smithi077.hkvsor"}]': finished 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: mgrmap e21: smithi078.navvuw(active, starting, since 0.00912015s) 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:52:27.633 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: Manager daemon smithi078.navvuw is now available 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/trash_purge_schedule"}]: dispatch 2024-04-26T04:52:27.634 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:27 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/trash_purge_schedule"}]: dispatch 2024-04-26T04:52:27.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr fail", "who": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: Activating manager daemon smithi078.navvuw 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: osdmap e47: 6 total, 6 up, 6 in 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14208 172.21.15.77:0/2309770382' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "mgr fail", "who": "smithi077.hkvsor"}]': finished 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: mgrmap e21: smithi078.navvuw(active, starting, since 0.00912015s) 2024-04-26T04:52:27.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:52:27.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: Manager daemon smithi078.navvuw is now available 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/trash_purge_schedule"}]: dispatch 2024-04-26T04:52:27.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:27 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi078.navvuw/trash_purge_schedule"}]: dispatch 2024-04-26T04:52:28.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: Deploying cephadm binary to smithi078 2024-04-26T04:52:28.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: Deploying cephadm binary to smithi077 2024-04-26T04:52:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: mgrmap e22: smithi078.navvuw(active, since 1.00809s) 2024-04-26T04:52:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: pgmap v3: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:28.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:28 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: Deploying cephadm binary to smithi078 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: Deploying cephadm binary to smithi077 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: mgrmap e22: smithi078.navvuw(active, since 1.00809s) 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: pgmap v3: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:28.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:28 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:29.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:29 smithi077 ceph-mon[26311]: pgmap v4: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:29.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:29 smithi077 ceph-mon[26311]: mgrmap e23: smithi078.navvuw(active, since 2s) 2024-04-26T04:52:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:29 smithi078 ceph-mon[33746]: pgmap v4: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:29 smithi078 ceph-mon[33746]: mgrmap e23: smithi078.navvuw(active, since 2s) 2024-04-26T04:52:30.614 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:30 smithi077 ceph-mon[26311]: [26/Apr/2024:04:52:29] ENGINE Bus STARTING 2024-04-26T04:52:30.614 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:30 smithi077 ceph-mon[26311]: [26/Apr/2024:04:52:29] ENGINE Serving on https://172.21.15.78:7150 2024-04-26T04:52:30.614 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:30 smithi077 ceph-mon[26311]: [26/Apr/2024:04:52:29] ENGINE Serving on http://172.21.15.78:8765 2024-04-26T04:52:30.614 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:30 smithi077 ceph-mon[26311]: [26/Apr/2024:04:52:29] ENGINE Bus STARTED 2024-04-26T04:52:30.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:30 smithi078 ceph-mon[33746]: [26/Apr/2024:04:52:29] ENGINE Bus STARTING 2024-04-26T04:52:30.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:30 smithi078 ceph-mon[33746]: [26/Apr/2024:04:52:29] ENGINE Serving on https://172.21.15.78:7150 2024-04-26T04:52:30.771 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:30 smithi078 ceph-mon[33746]: [26/Apr/2024:04:52:29] ENGINE Serving on http://172.21.15.78:8765 2024-04-26T04:52:30.771 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:30 smithi078 ceph-mon[33746]: [26/Apr/2024:04:52:29] ENGINE Bus STARTED 2024-04-26T04:52:31.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 -- 172.21.15.77:0/3758427835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fbcc0 msgr2=0x7faf380fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 --2- 172.21.15.77:0/3758427835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fbcc0 0x7faf380fc0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf28004940 tx=0x7faf28009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:31.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 -- 172.21.15.77:0/3758427835 shutdown_connections 2024-04-26T04:52:31.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 --2- 172.21.15.77:0/3758427835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fcf70 0x7faf380fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 --2- 172.21.15.77:0/3758427835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fbcc0 0x7faf380fc0d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 -- 172.21.15.77:0/3758427835 >> 172.21.15.77:0/3758427835 conn(0x7faf380f7440 msgr2=0x7faf380f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 -- 172.21.15.77:0/3758427835 shutdown_connections 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.489+0000 7faf3cacb700 1 -- 172.21.15.77:0/3758427835 wait complete. 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 Processor -- start 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 -- start start 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fbcc0 0x7faf3810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3810e510 con 0x7faf380fbcc0 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3cacb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38112eb0 con 0x7faf380fcf70 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fbcc0 0x7faf3810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf35d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf35d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41208/0 (socket says 172.21.15.77:41208) 2024-04-26T04:52:31.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.490+0000 7faf35d9b700 1 -- 172.21.15.77:0/1448352325 learned_addr learned my addr 172.21.15.77:0/1448352325 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.491+0000 7faf35d9b700 1 -- 172.21.15.77:0/1448352325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fbcc0 msgr2=0x7faf3810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.491+0000 7faf35d9b700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fbcc0 0x7faf3810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.491+0000 7faf35d9b700 1 -- 172.21.15.77:0/1448352325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28005040 con 0x7faf380fcf70 2024-04-26T04:52:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.491+0000 7faf35d9b700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00b790 tx=0x7faf2c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.491+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2c00f6f0 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.492+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf2c00fd20 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.492+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2c0103a0 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.492+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38113110 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.492+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf38113680 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.493+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7faf2c010500 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.493+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf38101b00 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.493+0000 7faf26ffd700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 0x7faf2003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.493+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faf2c04c800 con 0x7faf380fcf70 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.493+0000 7faf3659c700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 0x7faf2003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.494+0000 7faf3659c700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 0x7faf2003b3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faf2800b3b0 tx=0x7faf28009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:31.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.499+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7faf2c01a400 con 0x7faf380fcf70 2024-04-26T04:52:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:31 smithi077 ceph-mon[26311]: pgmap v5: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:31 smithi077 ceph-mon[26311]: mgrmap e24: smithi078.navvuw(active, since 4s) 2024-04-26T04:52:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:31 smithi078 ceph-mon[33746]: pgmap v5: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:31 smithi078 ceph-mon[33746]: mgrmap e24: smithi078.navvuw(active, since 4s) 2024-04-26T04:52:31.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.804+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf3802b460 con 0x7faf20038f10 2024-04-26T04:52:31.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.811+0000 7faf26ffd700 1 -- 172.21.15.77:0/1448352325 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7faf3802b460 con 0x7faf20038f10 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 msgr2=0x7faf2003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 0x7faf2003b3c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faf2800b3b0 tx=0x7faf28009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 msgr2=0x7faf3810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00b790 tx=0x7faf2c009250 comp rx=0 tx=0).stop 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 shutdown_connections 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faf20038f10 0x7faf2003b3c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf380fbcc0 0x7faf3810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 --2- 172.21.15.77:0/1448352325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf380fcf70 0x7faf3810de80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 >> 172.21.15.77:0/1448352325 conn(0x7faf380f7440 msgr2=0x7faf380f8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 shutdown_connections 2024-04-26T04:52:31.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.814+0000 7faf3cacb700 1 -- 172.21.15.77:0/1448352325 wait complete. 2024-04-26T04:52:31.828 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:52:31.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 -- 172.21.15.77:0/1986511153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fcf90 msgr2=0x7f4df40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1986511153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fcf90 0x7f4df40fd400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004940 tx=0x7f4de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 -- 172.21.15.77:0/1986511153 shutdown_connections 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1986511153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fcf90 0x7f4df40fd400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1986511153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df40fbce0 0x7f4df40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 -- 172.21.15.77:0/1986511153 >> 172.21.15.77:0/1986511153 conn(0x7f4df40f7480 msgr2=0x7f4df40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:31.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 -- 172.21.15.77:0/1986511153 shutdown_connections 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.941+0000 7f4df9e50700 1 -- 172.21.15.77:0/1986511153 wait complete. 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 Processor -- start 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 -- start start 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 0x7f4df4111ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df410d330 con 0x7f4df40fbce0 2024-04-26T04:52:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df9e50700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df410d470 con 0x7f4df410ce70 2024-04-26T04:52:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.942+0000 7f4df37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 0x7f4df4111ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59976/0 (socket says 172.21.15.77:59976) 2024-04-26T04:52:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 -- 172.21.15.77:0/1599017705 learned_addr learned my addr 172.21.15.77:0/1599017705 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 -- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 msgr2=0x7f4df4111ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 0x7f4df4111ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 -- 172.21.15.77:0/1599017705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8005040 con 0x7f4df40fbce0 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df2ffd700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 0x7f4df4111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.943+0000 7f4df37fe700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4de400eb40 tx=0x7f4de400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.944+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de4007cc0 con 0x7f4df40fbce0 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.944+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4de400f040 con 0x7f4df40fbce0 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.944+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de400ca10 con 0x7f4df40fbce0 2024-04-26T04:52:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.944+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4066270 con 0x7f4df40fbce0 2024-04-26T04:52:31.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.944+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df40667e0 con 0x7f4df40fbce0 2024-04-26T04:52:31.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.945+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f4de400cb70 con 0x7f4df40fbce0 2024-04-26T04:52:31.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.945+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df404c710 con 0x7f4df40fbce0 2024-04-26T04:52:31.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.946+0000 7f4ddbfff700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 0x7f4ddc03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:31.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.946+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4de404c810 con 0x7f4df40fbce0 2024-04-26T04:52:31.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.946+0000 7f4df2ffd700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 0x7f4ddc03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:31.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.947+0000 7f4df2ffd700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 0x7f4ddc03b450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:31.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:31.955+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f4de4007e20 con 0x7f4df40fbce0 2024-04-26T04:52:32.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.287+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4df4067000 con 0x7f4ddc038fa0 2024-04-26T04:52:32.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.298+0000 7f4ddbfff700 1 -- 172.21.15.77:0/1599017705 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f4df4067000 con 0x7f4ddc038fa0 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 msgr2=0x7f4ddc03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 0x7f4ddc03b450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 msgr2=0x7f4df410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4de400eb40 tx=0x7f4de400b040 comp rx=0 tx=0).stop 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 shutdown_connections 2024-04-26T04:52:32.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f4ddc038fa0 0x7f4ddc03b450 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4df40fbce0 0x7f4df410c930 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 --2- 172.21.15.77:0/1599017705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df410ce70 0x7f4df4111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 >> 172.21.15.77:0/1599017705 conn(0x7f4df40f7480 msgr2=0x7f4df40f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:32.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 shutdown_connections 2024-04-26T04:52:32.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.301+0000 7f4df9e50700 1 -- 172.21.15.77:0/1599017705 wait complete. 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.430+0000 7f55b896b700 1 -- 172.21.15.77:0/1394973235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fe430 msgr2=0x7f55b40fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.430+0000 7f55b896b700 1 --2- 172.21.15.77:0/1394973235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fe430 0x7f55b40fe840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f55a4004940 tx=0x7f55a4009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 -- 172.21.15.77:0/1394973235 shutdown_connections 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 --2- 172.21.15.77:0/1394973235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fed80 0x7f55b40faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 --2- 172.21.15.77:0/1394973235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fe430 0x7f55b40fe840 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 -- 172.21.15.77:0/1394973235 >> 172.21.15.77:0/1394973235 conn(0x7f55b40f6bd0 msgr2=0x7f55b40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 -- 172.21.15.77:0/1394973235 shutdown_connections 2024-04-26T04:52:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.431+0000 7f55b896b700 1 -- 172.21.15.77:0/1394973235 wait complete. 2024-04-26T04:52:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.432+0000 7f55b896b700 1 Processor -- start 2024-04-26T04:52:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.432+0000 7f55b896b700 1 -- start start 2024-04-26T04:52:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.433+0000 7f55b896b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fe430 0x7f55b410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.433+0000 7f55b896b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.433+0000 7f55b896b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b410e420 con 0x7f55b40fed80 2024-04-26T04:52:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.433+0000 7f55b896b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b410e590 con 0x7f55b40fe430 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59998/0 (socket says 172.21.15.77:59998) 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 -- 172.21.15.77:0/803082143 learned_addr learned my addr 172.21.15.77:0/803082143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fe430 0x7f55b410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 -- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fe430 msgr2=0x7f55b410d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fe430 0x7f55b410d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 -- 172.21.15.77:0/803082143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4005040 con 0x7f55b40fed80 2024-04-26T04:52:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.434+0000 7f55b1d9b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55a800eb40 tx=0x7f55a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.435+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a8007cc0 con 0x7f55b40fed80 2024-04-26T04:52:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.435+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4112f90 con 0x7f55b40fed80 2024-04-26T04:52:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.435+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55a800f040 con 0x7f55b40fed80 2024-04-26T04:52:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.435+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b41134a0 con 0x7f55b40fed80 2024-04-26T04:52:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.435+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a800ca10 con 0x7f55b40fed80 2024-04-26T04:52:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.436+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55940052e0 con 0x7f55b40fed80 2024-04-26T04:52:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.436+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f55a800cb70 con 0x7f55b40fed80 2024-04-26T04:52:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.437+0000 7f55a2ffd700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 0x7f559c03b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.437+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f55a804c780 con 0x7f55b40fed80 2024-04-26T04:52:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.437+0000 7f55b259c700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 0x7f559c03b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.437+0000 7f55b259c700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 0x7f559c03b3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009b30 tx=0x7f55a4009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.443+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f55a8012db0 con 0x7f55b40fed80 2024-04-26T04:52:32.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:32 smithi077 ceph-mon[26311]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:32.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:32 smithi078 ceph-mon[33746]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:32.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.789+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5594000bf0 con 0x7f559c038f20 2024-04-26T04:52:32.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.802+0000 7f55a2ffd700 1 -- 172.21.15.77:0/803082143 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f5594000bf0 con 0x7f559c038f20 2024-04-26T04:52:32.803 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:52:32.803 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (6m) 2m ago 7m 25.7M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (8m) 2m ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (7m) 15s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (6m) 2m ago 7m 85.1M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (2m) 2m ago 2m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (2m) 2m ago 2m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (2m) 15s ago 2m 14.1M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (2m) 15s ago 2m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (19s) 15s ago 7m 47.2M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 70ff04585016 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (9m) 2m ago 9m 44.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (6m) 15s ago 6m 36.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (8m) 2m ago 8m 14.3M - 1.5.0 0da6a335fe13 e44a82d54ec7 2024-04-26T04:52:32.804 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (7m) 15s ago 7m 16.1M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (6m) 2m ago 6m 44.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (5m) 2m ago 5m 44.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (5m) 2m ago 5m 43.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (4m) 15s ago 4m 73.4M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (4m) 15s ago 4m 69.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (4m) 15s ago 4m 70.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:52:32.805 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (6m) 2m ago 7m 56.1M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 msgr2=0x7f559c03b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 0x7f559c03b3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009b30 tx=0x7f55a4009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 msgr2=0x7f55b410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55a800eb40 tx=0x7f55a800b040 comp rx=0 tx=0).stop 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 shutdown_connections 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f559c038f20 0x7f559c03b3d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55b40fe430 0x7f55b410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 --2- 172.21.15.77:0/803082143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55b40fed80 0x7f55b410de20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 >> 172.21.15.77:0/803082143 conn(0x7f55b40f6bd0 msgr2=0x7f55b40f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:32.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 shutdown_connections 2024-04-26T04:52:32.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.804+0000 7f55b896b700 1 -- 172.21.15.77:0/803082143 wait complete. 2024-04-26T04:52:32.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 -- 172.21.15.77:0/1656364547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 msgr2=0x7fb4380fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/1656364547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb4380fc0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb428004940 tx=0x7fb428009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 -- 172.21.15.77:0/1656364547 shutdown_connections 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/1656364547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb4380fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/1656364547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb4380fc0c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 -- 172.21.15.77:0/1656364547 >> 172.21.15.77:0/1656364547 conn(0x7fb4380f7430 msgr2=0x7fb4380f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 -- 172.21.15.77:0/1656364547 shutdown_connections 2024-04-26T04:52:32.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 -- 172.21.15.77:0/1656364547 wait complete. 2024-04-26T04:52:32.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.954+0000 7fb43c8ff700 1 Processor -- start 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43c8ff700 1 -- start start 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43c8ff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb43810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43c8ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43c8ff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43810d3e0 con 0x7fb4380fbcb0 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43c8ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43810dc30 con 0x7fb4380fcf60 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb435d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb435d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41234/0 (socket says 172.21.15.77:41234) 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb435d9b700 1 -- 172.21.15.77:0/3596462436 learned_addr learned my addr 172.21.15.77:0/3596462436 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43659c700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb43810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43659c700 1 -- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 msgr2=0x7fb43810cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43659c700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:32.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb43659c700 1 -- 172.21.15.77:0/3596462436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb428005040 con 0x7fb4380fbcb0 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.955+0000 7fb435d9b700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.956+0000 7fb43659c700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb43810c8a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b0d0 tx=0x7fb428009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.956+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb42800e040 con 0x7fb4380fbcb0 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.956+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb428013bc0 con 0x7fb4380fbcb0 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.956+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43810deb0 con 0x7fb4380fbcb0 2024-04-26T04:52:32.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.957+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43810e330 con 0x7fb4380fbcb0 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.957+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4280123c0 con 0x7fb4380fbcb0 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.957+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7fb428012600 con 0x7fb4380fbcb0 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.958+0000 7fb426ffd700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 0x7fb42003b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.958+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb428013d30 con 0x7fb4380fbcb0 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.958+0000 7fb435d9b700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 0x7fb42003b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:32.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.958+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43805f5c0 con 0x7fb4380fbcb0 2024-04-26T04:52:32.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.959+0000 7fb435d9b700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 0x7fb42003b1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0041d0 tx=0x7fb42c00b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:32.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:32.968+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb428022070 con 0x7fb4380fbcb0 2024-04-26T04:52:33.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.550+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb4381047a0 con 0x7fb4380fbcb0 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.551+0000 7fb426ffd700 1 -- 172.21.15.77:0/3596462436 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb42801a8e0 con 0x7fb4380fbcb0 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:52:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:52:33.553 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 msgr2=0x7fb42003b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 0x7fb42003b1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0041d0 tx=0x7fb42c00b040 comp rx=0 tx=0).stop 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 msgr2=0x7fb43810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb43810c8a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b0d0 tx=0x7fb428009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 shutdown_connections 2024-04-26T04:52:33.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb420038d00 0x7fb42003b1b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4380fbcb0 0x7fb43810c8a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 --2- 172.21.15.77:0/3596462436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4380fcf60 0x7fb43810cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.554+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 >> 172.21.15.77:0/3596462436 conn(0x7fb4380f7430 msgr2=0x7fb4380f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:33.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.555+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 shutdown_connections 2024-04-26T04:52:33.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.555+0000 7fb43c8ff700 1 -- 172.21.15.77:0/3596462436 wait complete. 2024-04-26T04:52:33.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 -- 172.21.15.77:0/836697833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 msgr2=0x7f35480fd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 --2- 172.21.15.77:0/836697833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f35480fd6b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3544004940 tx=0x7f3544009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 -- 172.21.15.77:0/836697833 shutdown_connections 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 --2- 172.21.15.77:0/836697833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f35480fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 --2- 172.21.15.77:0/836697833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f35480fd6b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.711+0000 7f35500fc700 1 -- 172.21.15.77:0/836697833 >> 172.21.15.77:0/836697833 conn(0x7f35480f6b80 msgr2=0x7f35480f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.712+0000 7f35500fc700 1 -- 172.21.15.77:0/836697833 shutdown_connections 2024-04-26T04:52:33.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.712+0000 7f35500fc700 1 -- 172.21.15.77:0/836697833 wait complete. 2024-04-26T04:52:33.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 Processor -- start 2024-04-26T04:52:33.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 -- start start 2024-04-26T04:52:33.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:33.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f354810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354810e5b0 con 0x7f35480fd2a0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f35500fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354810ee00 con 0x7f35480fdbf0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f354de98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f354d697700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f354810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f354de98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60030/0 (socket says 172.21.15.77:60030) 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f354d697700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f354810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:41256/0 (socket says 172.21.15.77:41256) 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.713+0000 7f354de98700 1 -- 172.21.15.77:0/2373778169 learned_addr learned my addr 172.21.15.77:0/2373778169 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f354de98700 1 -- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 msgr2=0x7f354810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f354de98700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f354810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f354de98700 1 -- 172.21.15.77:0/2373778169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544005040 con 0x7f35480fd2a0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f354de98700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3544009150 tx=0x7f3544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354400e040 con 0x7f35480fd2a0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f354400bbf0 con 0x7f35480fd2a0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354401c630 con 0x7f35480fd2a0 2024-04-26T04:52:33.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.714+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354810f080 con 0x7f35480fd2a0 2024-04-26T04:52:33.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.715+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354810f5c0 con 0x7f35480fd2a0 2024-04-26T04:52:33.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.716+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f354401c790 con 0x7f35480fd2a0 2024-04-26T04:52:33.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.716+0000 7f353e7fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 0x7f353403b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:33.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.716+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354805f5c0 con 0x7f35480fd2a0 2024-04-26T04:52:33.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.716+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3544023c30 con 0x7f35480fd2a0 2024-04-26T04:52:33.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.716+0000 7f354d697700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 0x7f353403b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:33.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.717+0000 7f354d697700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 0x7f353403b3d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f35480655b0 tx=0x7f3538013040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:33.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:33.724+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3544013e60 con 0x7f35480fd2a0 2024-04-26T04:52:33.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:33 smithi078 ceph-mon[33746]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:33.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:33 smithi078 ceph-mon[33746]: pgmap v6: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:33.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:33 smithi078 ceph-mon[33746]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:33.840 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:33 smithi077 ceph-mon[26311]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:33.841 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:33 smithi077 ceph-mon[26311]: pgmap v6: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:52:33.841 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:33 smithi077 ceph-mon[26311]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.104+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35481058b0 con 0x7f35480fd2a0 2024-04-26T04:52:34.106 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:52:34.106 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:52:34.106 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:52:34.107 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:52:34.108 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:34.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.105+0000 7f353e7fc700 1 -- 172.21.15.77:0/2373778169 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f354403f070 con 0x7f35480fd2a0 2024-04-26T04:52:34.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.109+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 msgr2=0x7f353403b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.109+0000 7f35500fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 0x7f353403b3d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f35480655b0 tx=0x7f3538013040 comp rx=0 tx=0).stop 2024-04-26T04:52:34.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.109+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 msgr2=0x7f354810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.109+0000 7f35500fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3544009150 tx=0x7f3544009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:34.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 shutdown_connections 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f3534038f20 0x7f353403b3d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35480fd2a0 0x7f354810da70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 --2- 172.21.15.77:0/2373778169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35480fdbf0 0x7f354810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 >> 172.21.15.77:0/2373778169 conn(0x7f35480f6b80 msgr2=0x7f35480f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 shutdown_connections 2024-04-26T04:52:34.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.110+0000 7f35500fc700 1 -- 172.21.15.77:0/2373778169 wait complete. 2024-04-26T04:52:34.112 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:52:34.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/2356467111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 msgr2=0x7fe0a8102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/2356467111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a8102330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe09c004940 tx=0x7fe09c009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:34.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/2356467111 shutdown_connections 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/2356467111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a8102330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/2356467111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a80ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/2356467111 >> 172.21.15.77:0/2356467111 conn(0x7fe0a80f7440 msgr2=0x7fe0a80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/2356467111 shutdown_connections 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.239+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/2356467111 wait complete. 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 Processor -- start 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 -- start start 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a810b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a810c030 con 0x7fe0a80fff10 2024-04-26T04:52:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0ad7cc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a810c1a0 con 0x7fe0a8066f10 2024-04-26T04:52:34.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60046/0 (socket says 172.21.15.77:60046) 2024-04-26T04:52:34.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 -- 172.21.15.77:0/3119076439 learned_addr learned my addr 172.21.15.77:0/3119076439 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:34.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a6ffd700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a810b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 -- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 msgr2=0x7fe0a810b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a810b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a67fc700 1 -- 172.21.15.77:0/3119076439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c005040 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.240+0000 7fe0a6ffd700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe0a67fc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009040 tx=0x7fe09c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09c021070 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a8110ba0 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe09c01c8d0 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a81110e0 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.241+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09c00e070 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.242+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7fe09c00b9e0 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.242+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0940052b0 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.242+0000 7fe08f7fe700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 0x7fe09003b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.242+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe09c04f390 con 0x7fe0a80fff10 2024-04-26T04:52:34.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.243+0000 7fe0a6ffd700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 0x7fe09003b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.243+0000 7fe0a6ffd700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 0x7fe09003b3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8066c50 tx=0x7fe09800b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:34.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.248+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe09c026cb0 con 0x7fe0a80fff10 2024-04-26T04:52:34.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.546+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe094000bc0 con 0x7fe090038f20 2024-04-26T04:52:34.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.547+0000 7fe08f7fe700 1 -- 172.21.15.77:0/3119076439 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe094000bc0 con 0x7fe090038f20 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "1/21 daemons upgraded", 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "", 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:52:34.548 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:52:34.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 msgr2=0x7fe09003b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 0x7fe09003b3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8066c50 tx=0x7fe09800b040 comp rx=0 tx=0).stop 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 msgr2=0x7fe0a810ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009040 tx=0x7fe09c009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 shutdown_connections 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe090038f20 0x7fe09003b3d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a8066f10 0x7fe0a810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 --2- 172.21.15.77:0/3119076439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0a80fff10 0x7fe0a810ba30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 >> 172.21.15.77:0/3119076439 conn(0x7fe0a80f7440 msgr2=0x7fe0a80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 shutdown_connections 2024-04-26T04:52:34.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.550+0000 7fe0ad7cc700 1 -- 172.21.15.77:0/3119076439 wait complete. 2024-04-26T04:52:34.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 -- 172.21.15.77:0/2465138851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc0fffa0 msgr2=0x7fe3bc102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/2465138851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc102380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac004940 tx=0x7fe3ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 -- 172.21.15.77:0/2465138851 shutdown_connections 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/2465138851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc102380 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/2465138851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc066200 0x7fe3bc0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 -- 172.21.15.77:0/2465138851 >> 172.21.15.77:0/2465138851 conn(0x7fe3bc0f7430 msgr2=0x7fe3bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 -- 172.21.15.77:0/2465138851 shutdown_connections 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 -- 172.21.15.77:0/2465138851 wait complete. 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.671+0000 7fe3c43ed700 1 Processor -- start 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c43ed700 1 -- start start 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c43ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c43ed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c43ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc10e520 con 0x7fe3bc066200 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c43ed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc10ed20 con 0x7fe3bc0fffa0 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60064/0 (socket says 172.21.15.77:60064) 2024-04-26T04:52:34.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 -- 172.21.15.77:0/1070591563 learned_addr learned my addr 172.21.15.77:0/1070591563 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:52:34.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c1988700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 -- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 msgr2=0x7fe3bc10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 -- 172.21.15.77:0/1070591563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac005040 con 0x7fe3bc066200 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c1988700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.672+0000 7fe3c2189700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800eb70 tx=0x7fe3b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.673+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3b8007cc0 con 0x7fe3bc066200 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.673+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3b800f040 con 0x7fe3bc066200 2024-04-26T04:52:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.673+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3b800cb20 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.673+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc10efa0 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.673+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc10f420 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.674+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc05f5c0 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.674+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) v1 ==== 46295+0+0 (secure 0 0 0) 0x7fe3b800cc80 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.675+0000 7fe3b2ffd700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 0x7fe3a803b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.675+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe3b804c620 con 0x7fe3bc066200 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.675+0000 7fe3c1988700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 0x7fe3a803b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.675+0000 7fe3c1988700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 0x7fe3a803b460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac004890 tx=0x7fe3ac009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:34.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:34.680+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe3b80097c0 con 0x7fe3bc066200 2024-04-26T04:52:34.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:34 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3596462436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:34.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:34 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2373778169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:34.933 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3596462436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:34.933 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:34 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2373778169' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:35.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.179+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3bc04bed0 con 0x7fe3bc066200 2024-04-26T04:52:35.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.180+0000 7fe3b2ffd700 1 -- 172.21.15.77:0/1070591563 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe3b8012bd0 con 0x7fe3bc066200 2024-04-26T04:52:35.181 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:52:35.181 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:52:35.181 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:52:35.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 msgr2=0x7fe3a803b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:35.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 0x7fe3a803b460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac004890 tx=0x7fe3ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:52:35.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 msgr2=0x7fe3bc10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:35.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800eb70 tx=0x7fe3b800b040 comp rx=0 tx=0).stop 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 shutdown_connections 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe3a8038fb0 0x7fe3a803b460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3bc066200 0x7fe3bc10d9e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 --2- 172.21.15.77:0/1070591563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3bc0fffa0 0x7fe3bc10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.183+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 >> 172.21.15.77:0/1070591563 conn(0x7fe3bc0f7430 msgr2=0x7fe3bc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.184+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 shutdown_connections 2024-04-26T04:52:35.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:52:35.184+0000 7fe3c43ed700 1 -- 172.21.15.77:0/1070591563 wait complete. 2024-04-26T04:52:35.734 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: pgmap v7: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-04-26T04:52:35.734 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/1070591563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: Standby manager daemon smithi077.hkvsor started 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/crt"}]: dispatch 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/key"}]: dispatch 2024-04-26T04:52:35.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:35 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: pgmap v7: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/1070591563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: Standby manager daemon smithi077.hkvsor started 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/crt"}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/key"}]: dispatch 2024-04-26T04:52:35.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:35 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/21190184' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:52:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:36 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:52:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:36 smithi078 ceph-mon[33746]: mgrmap e25: smithi078.navvuw(active, since 9s), standbys: smithi077.hkvsor 2024-04-26T04:52:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:36 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:36 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:36.777 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:36 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:52:36.777 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:36 smithi077 ceph-mon[26311]: mgrmap e25: smithi078.navvuw(active, since 9s), standbys: smithi077.hkvsor 2024-04-26T04:52:36.777 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:36 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:36.777 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:36 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:38.125 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:37 smithi078 ceph-mon[33746]: pgmap v8: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-04-26T04:52:38.125 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:37 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:38.125 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:37 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:37 smithi077 ceph-mon[26311]: pgmap v8: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-04-26T04:52:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:37 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:37 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:39.457 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:39.457 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:39.457 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:39 smithi078 ceph-mon[33746]: pgmap v9: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-04-26T04:52:39.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:39.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:39.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:39 smithi077 ceph-mon[26311]: pgmap v9: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-04-26T04:52:40.630 INFO:tasks.workunit.client.1.smithi078.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-26T04:52:40.634 INFO:tasks.workunit.client.1.smithi078.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:52:40.635 INFO:tasks.workunit.client.1.smithi078.stderr:+ make 2024-04-26T04:52:40.698 INFO:tasks.workunit.client.0.smithi077.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-26T04:52:40.704 INFO:tasks.workunit.client.0.smithi077.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:52:40.704 INFO:tasks.workunit.client.0.smithi077.stderr:+ make 2024-04-26T04:52:40.916 INFO:tasks.workunit.client.1.smithi078.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-26T04:52:40.940 INFO:tasks.workunit.client.0.smithi077.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-26T04:52:40.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi077 to 1557M 2024-04-26T04:52:40.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:40 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi077 to 1557M 2024-04-26T04:52:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:40 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:41.274 INFO:tasks.workunit.client.0.smithi077.stderr:++ readlink -f fsstress 2024-04-26T04:52:41.276 INFO:tasks.workunit.client.0.smithi077.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-26T04:52:41.276 INFO:tasks.workunit.client.0.smithi077.stderr:+ popd 2024-04-26T04:52:41.277 INFO:tasks.workunit.client.0.smithi077.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:52:41.277 INFO:tasks.workunit.client.0.smithi077.stderr:+ popd 2024-04-26T04:52:41.277 INFO:tasks.workunit.client.0.smithi077.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:52:41.278 INFO:tasks.workunit.client.0.smithi077.stderr:++ mktemp -d -p . 2024-04-26T04:52:41.280 INFO:tasks.workunit.client.0.smithi077.stderr:+ T=./tmp.FvYFaNvwQz 2024-04-26T04:52:41.280 INFO:tasks.workunit.client.0.smithi077.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.FvYFaNvwQz -l 1 -n 1000 -p 10 -v 2024-04-26T04:52:41.283 INFO:tasks.workunit.client.0.smithi077.stdout:seed = 1714382817 2024-04-26T04:52:41.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/0: fdatasync - no filename 2024-04-26T04:52:41.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/1: rename - no filename 2024-04-26T04:52:41.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/2: dread - no filename 2024-04-26T04:52:41.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/3: truncate - no filename 2024-04-26T04:52:41.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/4: dread - no filename 2024-04-26T04:52:41.289 INFO:tasks.workunit.client.0.smithi077.stdout:1/5: creat f0 x:0 0 0 2024-04-26T04:52:41.290 INFO:tasks.workunit.client.0.smithi077.stdout:3/0: dread - no filename 2024-04-26T04:52:41.290 INFO:tasks.workunit.client.0.smithi077.stdout:3/1: unlink - no file 2024-04-26T04:52:41.290 INFO:tasks.workunit.client.0.smithi077.stdout:1/6: symlink l1 0 2024-04-26T04:52:41.291 INFO:tasks.workunit.client.0.smithi077.stdout:1/7: truncate f0 757724 0 2024-04-26T04:52:41.291 INFO:tasks.workunit.client.0.smithi077.stdout:2/0: mknod c0 0 2024-04-26T04:52:41.293 INFO:tasks.workunit.client.0.smithi077.stdout:1/8: dread f0 [0,4194304] 0 2024-04-26T04:52:41.293 INFO:tasks.workunit.client.0.smithi077.stdout:4/0: read - no filename 2024-04-26T04:52:41.294 INFO:tasks.workunit.client.0.smithi077.stdout:3/2: mkdir d0 0 2024-04-26T04:52:41.296 INFO:tasks.workunit.client.0.smithi077.stdout:1/9: write f0 [1685683,65900] 0 2024-04-26T04:52:41.297 INFO:tasks.workunit.client.0.smithi077.stdout:1/10: write f0 [1815289,28548] 0 2024-04-26T04:52:41.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/1: symlink l0 0 2024-04-26T04:52:41.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/2: read - no filename 2024-04-26T04:52:41.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/3: dread - no filename 2024-04-26T04:52:41.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/4: fsync - no filename 2024-04-26T04:52:41.298 INFO:tasks.workunit.client.0.smithi077.stdout:4/5: chown l0 240972915 1 2024-04-26T04:52:41.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/6: chown l0 5994 1 2024-04-26T04:52:41.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/7: read - no filename 2024-04-26T04:52:41.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/8: dread - no filename 2024-04-26T04:52:41.300 INFO:tasks.workunit.client.0.smithi077.stdout:6/0: write - no filename 2024-04-26T04:52:41.300 INFO:tasks.workunit.client.0.smithi077.stdout:6/1: rmdir - no directory 2024-04-26T04:52:41.300 INFO:tasks.workunit.client.0.smithi077.stdout:6/2: rename - no filename 2024-04-26T04:52:41.300 INFO:tasks.workunit.client.0.smithi077.stdout:3/3: creat d0/f1 x:0 0 0 2024-04-26T04:52:41.302 INFO:tasks.workunit.client.0.smithi077.stdout:7/0: stat - no entries 2024-04-26T04:52:41.302 INFO:tasks.workunit.client.0.smithi077.stdout:5/0: symlink l0 0 2024-04-26T04:52:41.302 INFO:tasks.workunit.client.0.smithi077.stdout:5/1: write - no filename 2024-04-26T04:52:41.303 INFO:tasks.workunit.client.0.smithi077.stdout:5/2: dread - no filename 2024-04-26T04:52:41.303 INFO:tasks.workunit.client.0.smithi077.stdout:5/3: chown l0 11 1 2024-04-26T04:52:41.303 INFO:tasks.workunit.client.0.smithi077.stdout:5/4: fsync - no filename 2024-04-26T04:52:41.304 INFO:tasks.workunit.client.0.smithi077.stdout:4/9: creat f1 x:0 0 0 2024-04-26T04:52:41.308 INFO:tasks.workunit.client.0.smithi077.stdout:5/5: mkdir d1 0 2024-04-26T04:52:41.309 INFO:tasks.workunit.client.0.smithi077.stdout:6/3: creat f0 x:0 0 0 2024-04-26T04:52:41.310 INFO:tasks.workunit.client.0.smithi077.stdout:4/10: rename f1 to f2 0 2024-04-26T04:52:41.311 INFO:tasks.workunit.client.0.smithi077.stdout:3/4: link d0/f1 d0/f2 0 2024-04-26T04:52:41.312 INFO:tasks.workunit.client.0.smithi077.stdout:7/1: creat f0 x:0 0 0 2024-04-26T04:52:41.312 INFO:tasks.workunit.client.0.smithi077.stdout:7/2: stat f0 0 2024-04-26T04:52:41.316 INFO:tasks.workunit.client.0.smithi077.stdout:5/6: unlink l0 0 2024-04-26T04:52:41.316 INFO:tasks.workunit.client.0.smithi077.stdout:5/7: dwrite - no filename 2024-04-26T04:52:41.316 INFO:tasks.workunit.client.0.smithi077.stdout:5/8: chown d1 3 1 2024-04-26T04:52:41.317 INFO:tasks.workunit.client.0.smithi077.stdout:6/4: mknod c1 0 2024-04-26T04:52:41.319 INFO:tasks.workunit.client.0.smithi077.stdout:4/11: symlink l3 0 2024-04-26T04:52:41.319 INFO:tasks.workunit.client.0.smithi077.stdout:4/12: chown l3 776922329 1 2024-04-26T04:52:41.319 INFO:tasks.workunit.client.0.smithi077.stdout:7/3: rename f0 to f1 0 2024-04-26T04:52:41.320 INFO:tasks.workunit.client.0.smithi077.stdout:8/0: creat f0 x:0 0 0 2024-04-26T04:52:41.321 INFO:tasks.workunit.client.0.smithi077.stdout:8/1: read - f0 zero size 2024-04-26T04:52:41.322 INFO:tasks.workunit.client.0.smithi077.stdout:6/5: mknod c2 0 2024-04-26T04:52:41.323 INFO:tasks.workunit.client.0.smithi077.stdout:6/6: truncate f0 779840 0 2024-04-26T04:52:41.323 INFO:tasks.workunit.client.0.smithi077.stdout:7/4: creat f2 x:0 0 0 2024-04-26T04:52:41.324 INFO:tasks.workunit.client.0.smithi077.stdout:9/0: mknod c0 0 2024-04-26T04:52:41.324 INFO:tasks.workunit.client.0.smithi077.stdout:9/1: dread - no filename 2024-04-26T04:52:41.326 INFO:tasks.workunit.client.0.smithi077.stdout:6/7: creat f3 x:0 0 0 2024-04-26T04:52:41.329 INFO:tasks.workunit.client.0.smithi077.stdout:6/8: write f0 [1588570,43504] 0 2024-04-26T04:52:41.331 INFO:tasks.workunit.client.0.smithi077.stdout:9/2: link c0 c1 0 2024-04-26T04:52:41.333 INFO:tasks.workunit.client.0.smithi077.stdout:8/2: dwrite f0 [0,4194304] 0 2024-04-26T04:52:41.333 INFO:tasks.workunit.client.0.smithi077.stdout:9/3: creat f2 x:0 0 0 2024-04-26T04:52:41.334 INFO:tasks.workunit.client.0.smithi077.stdout:6/9: creat f4 x:0 0 0 2024-04-26T04:52:41.337 INFO:tasks.workunit.client.0.smithi077.stdout:6/10: symlink l5 0 2024-04-26T04:52:41.338 INFO:tasks.workunit.client.0.smithi077.stdout:7/5: dwrite f2 [0,4194304] 0 2024-04-26T04:52:41.339 INFO:tasks.workunit.client.0.smithi077.stdout:7/6: dread - f1 zero size 2024-04-26T04:52:41.340 INFO:tasks.workunit.client.0.smithi077.stdout:9/4: link f2 f3 0 2024-04-26T04:52:41.342 INFO:tasks.workunit.client.0.smithi077.stdout:6/11: symlink l6 0 2024-04-26T04:52:41.344 INFO:tasks.workunit.client.0.smithi077.stdout:7/7: mknod c3 0 2024-04-26T04:52:41.346 INFO:tasks.workunit.client.0.smithi077.stdout:6/12: symlink l7 0 2024-04-26T04:52:41.348 INFO:tasks.workunit.client.0.smithi077.stdout:6/13: chown c2 125304 1 2024-04-26T04:52:41.349 INFO:tasks.workunit.client.0.smithi077.stdout:7/8: write f1 [574523,123497] 0 2024-04-26T04:52:41.350 INFO:tasks.workunit.client.0.smithi077.stdout:6/14: mkdir d8 0 2024-04-26T04:52:41.352 INFO:tasks.workunit.client.0.smithi077.stdout:6/15: dread - f4 zero size 2024-04-26T04:52:41.352 INFO:tasks.workunit.client.0.smithi077.stdout:7/9: creat f4 x:0 0 0 2024-04-26T04:52:41.355 INFO:tasks.workunit.client.0.smithi077.stdout:6/16: read f0 [1444820,130716] 0 2024-04-26T04:52:41.356 INFO:tasks.workunit.client.0.smithi077.stdout:9/5: dwrite f3 [0,4194304] 0 2024-04-26T04:52:41.356 INFO:tasks.workunit.client.0.smithi077.stdout:7/10: rename f4 to f5 0 2024-04-26T04:52:41.361 INFO:tasks.workunit.client.0.smithi077.stdout:6/17: write f0 [1250106,109001] 0 2024-04-26T04:52:41.366 INFO:tasks.workunit.client.0.smithi077.stdout:9/6: link c1 c4 0 2024-04-26T04:52:41.366 INFO:tasks.workunit.client.0.smithi077.stdout:9/7: rmdir - no directory 2024-04-26T04:52:41.367 INFO:tasks.workunit.client.0.smithi077.stdout:6/18: mknod d8/c9 0 2024-04-26T04:52:41.370 INFO:tasks.workunit.client.0.smithi077.stdout:9/8: dread f2 [0,4194304] 0 2024-04-26T04:52:41.370 INFO:tasks.workunit.client.0.smithi077.stdout:6/19: rename d8/c9 to d8/ca 0 2024-04-26T04:52:41.370 INFO:tasks.workunit.client.0.smithi077.stdout:6/20: readlink l6 0 2024-04-26T04:52:41.372 INFO:tasks.workunit.client.0.smithi077.stdout:9/9: write f2 [511323,125776] 0 2024-04-26T04:52:41.372 INFO:tasks.workunit.client.0.smithi077.stdout:6/21: rmdir d8 39 2024-04-26T04:52:41.373 INFO:tasks.workunit.client.0.smithi077.stdout:6/22: fdatasync f0 0 2024-04-26T04:52:41.377 INFO:tasks.workunit.client.0.smithi077.stdout:9/10: dread f2 [0,4194304] 0 2024-04-26T04:52:41.378 INFO:tasks.workunit.client.0.smithi077.stdout:9/11: write f2 [1017886,89709] 0 2024-04-26T04:52:41.382 INFO:tasks.workunit.client.0.smithi077.stdout:9/12: rename c0 to c5 0 2024-04-26T04:52:41.387 INFO:tasks.workunit.client.0.smithi077.stdout:9/13: dwrite f3 [0,4194304] 0 2024-04-26T04:52:41.389 INFO:tasks.workunit.client.0.smithi077.stdout:9/14: mknod c6 0 2024-04-26T04:52:41.393 INFO:tasks.workunit.client.0.smithi077.stdout:9/15: creat f7 x:0 0 0 2024-04-26T04:52:41.470 INFO:tasks.workunit.client.0.smithi077.stdout:1/11: fsync f0 0 2024-04-26T04:52:41.475 INFO:tasks.workunit.client.0.smithi077.stdout:1/12: rename f0 to f2 0 2024-04-26T04:52:41.475 INFO:tasks.workunit.client.0.smithi077.stdout:2/1: getdents . 0 2024-04-26T04:52:41.475 INFO:tasks.workunit.client.0.smithi077.stdout:2/2: truncate - no filename 2024-04-26T04:52:41.478 INFO:tasks.workunit.client.0.smithi077.stdout:1/13: write f2 [2883208,25553] 0 2024-04-26T04:52:41.483 INFO:tasks.workunit.client.0.smithi077.stdout:1/14: write f2 [159997,101584] 0 2024-04-26T04:52:41.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/15: write f2 [3735754,116694] 0 2024-04-26T04:52:41.485 INFO:tasks.workunit.client.0.smithi077.stdout:2/3: getdents . 0 2024-04-26T04:52:41.491 INFO:tasks.workunit.client.0.smithi077.stdout:1/16: dwrite f2 [0,4194304] 0 2024-04-26T04:52:41.492 INFO:tasks.workunit.client.0.smithi077.stdout:1/17: chown l1 0 1 2024-04-26T04:52:41.492 INFO:tasks.workunit.client.0.smithi077.stdout:1/18: rmdir - no directory 2024-04-26T04:52:41.492 INFO:tasks.workunit.client.0.smithi077.stdout:1/19: chown f2 6777473 1 2024-04-26T04:52:41.493 INFO:tasks.workunit.client.0.smithi077.stdout:2/4: creat f1 x:0 0 0 2024-04-26T04:52:41.493 INFO:tasks.workunit.client.0.smithi077.stdout:2/5: chown f1 2203272 1 2024-04-26T04:52:41.501 INFO:tasks.workunit.client.0.smithi077.stdout:2/6: unlink f1 0 2024-04-26T04:52:41.506 INFO:tasks.workunit.client.0.smithi077.stdout:2/7: unlink c0 0 2024-04-26T04:52:41.509 INFO:tasks.workunit.client.0.smithi077.stdout:2/8: creat f2 x:0 0 0 2024-04-26T04:52:41.517 INFO:tasks.workunit.client.0.smithi077.stdout:2/9: dwrite f2 [0,4194304] 0 2024-04-26T04:52:41.518 INFO:tasks.workunit.client.0.smithi077.stdout:8/3: truncate f0 258458 0 2024-04-26T04:52:41.522 INFO:tasks.workunit.client.0.smithi077.stdout:7/11: dwrite f2 [4194304,4194304] 0 2024-04-26T04:52:41.525 INFO:tasks.workunit.client.0.smithi077.stdout:8/4: creat f1 x:0 0 0 2024-04-26T04:52:41.527 INFO:tasks.workunit.client.0.smithi077.stdout:2/10: dread f2 [0,4194304] 0 2024-04-26T04:52:41.531 INFO:tasks.workunit.client.0.smithi077.stdout:7/12: rename f2 to f6 0 2024-04-26T04:52:41.534 INFO:tasks.workunit.client.0.smithi077.stdout:8/5: creat f2 x:0 0 0 2024-04-26T04:52:41.536 INFO:tasks.workunit.client.0.smithi077.stdout:6/23: creat d8/fb x:0 0 0 2024-04-26T04:52:41.536 INFO:tasks.workunit.client.0.smithi077.stdout:6/24: dread - f3 zero size 2024-04-26T04:52:41.542 INFO:tasks.workunit.client.0.smithi077.stdout:7/13: unlink f5 0 2024-04-26T04:52:41.545 INFO:tasks.workunit.client.0.smithi077.stdout:8/6: mknod c3 0 2024-04-26T04:52:41.547 INFO:tasks.workunit.client.0.smithi077.stdout:6/25: rename f0 to d8/fc 0 2024-04-26T04:52:41.549 INFO:tasks.workunit.client.0.smithi077.stdout:6/26: dread - f4 zero size 2024-04-26T04:52:41.551 INFO:tasks.workunit.client.0.smithi077.stdout:9/16: chown c5 31285232 1 2024-04-26T04:52:41.551 INFO:tasks.workunit.client.0.smithi077.stdout:7/14: dwrite f1 [0,4194304] 0 2024-04-26T04:52:41.553 INFO:tasks.workunit.client.0.smithi077.stdout:7/15: chown f1 522505 1 2024-04-26T04:52:41.556 INFO:tasks.workunit.client.0.smithi077.stdout:8/7: dwrite f2 [0,4194304] 0 2024-04-26T04:52:41.557 INFO:tasks.workunit.client.0.smithi077.stdout:6/27: write d8/fc [1844731,117562] 0 2024-04-26T04:52:41.558 INFO:tasks.workunit.client.0.smithi077.stdout:6/28: read d8/fc [1855336,63040] 0 2024-04-26T04:52:41.558 INFO:tasks.workunit.client.0.smithi077.stdout:6/29: readlink l6 0 2024-04-26T04:52:41.561 INFO:tasks.workunit.client.0.smithi077.stdout:9/17: creat f8 x:0 0 0 2024-04-26T04:52:41.563 INFO:tasks.workunit.client.0.smithi077.stdout:7/16: unlink c3 0 2024-04-26T04:52:41.564 INFO:tasks.workunit.client.0.smithi077.stdout:8/8: mkdir d4 0 2024-04-26T04:52:41.688 INFO:tasks.workunit.client.0.smithi077.stdout:6/30: mknod d8/cd 0 2024-04-26T04:52:41.689 INFO:tasks.workunit.client.0.smithi077.stdout:9/18: mkdir d9 0 2024-04-26T04:52:41.689 INFO:tasks.workunit.client.0.smithi077.stdout:9/19: fsync f2 0 2024-04-26T04:52:41.690 INFO:tasks.workunit.client.0.smithi077.stdout:9/20: write f3 [3475372,101038] 0 2024-04-26T04:52:41.691 INFO:tasks.workunit.client.0.smithi077.stdout:9/21: write f2 [3322003,100099] 0 2024-04-26T04:52:41.696 INFO:tasks.workunit.client.0.smithi077.stdout:6/31: creat d8/fe x:0 0 0 2024-04-26T04:52:41.697 INFO:tasks.workunit.client.0.smithi077.stdout:6/32: write d8/fe [726968,93737] 0 2024-04-26T04:52:41.702 INFO:tasks.workunit.client.0.smithi077.stdout:8/9: creat d4/f5 x:0 0 0 2024-04-26T04:52:41.704 INFO:tasks.workunit.client.0.smithi077.stdout:6/33: creat d8/ff x:0 0 0 2024-04-26T04:52:41.706 INFO:tasks.workunit.client.0.smithi077.stdout:9/22: symlink d9/la 0 2024-04-26T04:52:41.713 INFO:tasks.workunit.client.0.smithi077.stdout:9/23: creat d9/fb x:0 0 0 2024-04-26T04:52:41.716 INFO:tasks.workunit.client.0.smithi077.stdout:9/24: creat d9/fc x:0 0 0 2024-04-26T04:52:41.716 INFO:tasks.workunit.client.0.smithi077.stdout:9/25: chown f2 180 1 2024-04-26T04:52:41.768 INFO:tasks.workunit.client.0.smithi077.stdout:1/20: dwrite f2 [4194304,4194304] 0 2024-04-26T04:52:41.774 INFO:tasks.workunit.client.0.smithi077.stdout:1/21: creat f3 x:0 0 0 2024-04-26T04:52:41.778 INFO:tasks.workunit.client.0.smithi077.stdout:1/22: rename l1 to l4 0 2024-04-26T04:52:41.781 INFO:tasks.workunit.client.0.smithi077.stdout:1/23: fsync f2 0 2024-04-26T04:52:41.781 INFO:tasks.workunit.client.0.smithi077.stdout:1/24: write f3 [909080,87468] 0 2024-04-26T04:52:41.784 INFO:tasks.workunit.client.0.smithi077.stdout:1/25: mkdir d5 0 2024-04-26T04:52:41.786 INFO:tasks.workunit.client.0.smithi077.stdout:2/11: write f2 [4256533,23299] 0 2024-04-26T04:52:41.790 INFO:tasks.workunit.client.0.smithi077.stdout:2/12: symlink l3 0 2024-04-26T04:52:41.793 INFO:tasks.workunit.client.0.smithi077.stdout:1/26: dwrite f3 [0,4194304] 0 2024-04-26T04:52:41.795 INFO:tasks.workunit.client.0.smithi077.stdout:2/13: rename l3 to l4 0 2024-04-26T04:52:41.798 INFO:tasks.workunit.client.0.smithi077.stdout:7/17: unlink f6 0 2024-04-26T04:52:41.802 INFO:tasks.workunit.client.0.smithi077.stdout:2/14: rename f2 to f5 0 2024-04-26T04:52:41.802 INFO:tasks.workunit.client.0.smithi077.stdout:8/10: fsync f2 0 2024-04-26T04:52:41.805 INFO:tasks.workunit.client.0.smithi077.stdout:7/18: creat f7 x:0 0 0 2024-04-26T04:52:41.809 INFO:tasks.workunit.client.0.smithi077.stdout:8/11: link c3 d4/c6 0 2024-04-26T04:52:41.811 INFO:tasks.workunit.client.0.smithi077.stdout:8/12: symlink d4/l7 0 2024-04-26T04:52:41.812 INFO:tasks.workunit.client.0.smithi077.stdout:8/13: symlink d4/l8 0 2024-04-26T04:52:41.821 INFO:tasks.workunit.client.0.smithi077.stdout:8/14: dwrite f0 [0,4194304] 0 2024-04-26T04:52:41.824 INFO:tasks.workunit.client.0.smithi077.stdout:8/15: rename f2 to d4/f9 0 2024-04-26T04:52:41.826 INFO:tasks.workunit.client.0.smithi077.stdout:8/16: symlink d4/la 0 2024-04-26T04:52:41.828 INFO:tasks.workunit.client.0.smithi077.stdout:6/34: getdents d8 0 2024-04-26T04:52:41.829 INFO:tasks.workunit.client.0.smithi077.stdout:8/17: symlink d4/lb 0 2024-04-26T04:52:41.831 INFO:tasks.workunit.client.0.smithi077.stdout:6/35: unlink d8/fc 0 2024-04-26T04:52:41.832 INFO:tasks.workunit.client.0.smithi077.stdout:6/36: truncate f3 111059 0 2024-04-26T04:52:41.832 INFO:tasks.workunit.client.0.smithi077.stdout:8/18: fdatasync d4/f9 0 2024-04-26T04:52:41.842 INFO:tasks.workunit.client.1.smithi078.stderr:++ readlink -f fsstress 2024-04-26T04:52:41.844 INFO:tasks.workunit.client.1.smithi078.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-26T04:52:41.844 INFO:tasks.workunit.client.1.smithi078.stderr:+ popd 2024-04-26T04:52:41.845 INFO:tasks.workunit.client.1.smithi078.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:52:41.845 INFO:tasks.workunit.client.1.smithi078.stderr:+ popd 2024-04-26T04:52:41.846 INFO:tasks.workunit.client.1.smithi078.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:52:41.846 INFO:tasks.workunit.client.1.smithi078.stderr:++ mktemp -d -p . 2024-04-26T04:52:41.849 INFO:tasks.workunit.client.0.smithi077.stdout:9/26: getdents d9 0 2024-04-26T04:52:41.849 INFO:tasks.workunit.client.0.smithi077.stdout:8/19: dread - f1 zero size 2024-04-26T04:52:41.850 INFO:tasks.workunit.client.1.smithi078.stderr:+ T=./tmp.U5lnxyJscE 2024-04-26T04:52:41.850 INFO:tasks.workunit.client.1.smithi078.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.U5lnxyJscE -l 1 -n 1000 -p 10 -v 2024-04-26T04:52:41.852 INFO:tasks.workunit.client.0.smithi077.stdout:8/20: dread f0 [0,4194304] 0 2024-04-26T04:52:41.857 INFO:tasks.workunit.client.1.smithi078.stdout:seed = 1713775977 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/0: dread - no filename 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/1: dread - no filename 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/2: unlink - no file 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/3: link - no file 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/4: dread - no filename 2024-04-26T04:52:41.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/5: truncate - no filename 2024-04-26T04:52:41.865 INFO:tasks.workunit.client.1.smithi078.stdout:1/0: write - no filename 2024-04-26T04:52:41.867 INFO:tasks.workunit.client.1.smithi078.stdout:2/0: rmdir - no directory 2024-04-26T04:52:41.870 INFO:tasks.workunit.client.1.smithi078.stdout:0/6: creat f0 x:0 0 0 2024-04-26T04:52:41.870 INFO:tasks.workunit.client.1.smithi078.stdout:0/7: write f0 [843705,41732] 0 2024-04-26T04:52:41.871 INFO:tasks.workunit.client.1.smithi078.stdout:0/8: chown f0 0 1 2024-04-26T04:52:41.873 INFO:tasks.workunit.client.1.smithi078.stdout:3/0: dwrite - no filename 2024-04-26T04:52:41.873 INFO:tasks.workunit.client.1.smithi078.stdout:3/1: rmdir - no directory 2024-04-26T04:52:41.875 INFO:tasks.workunit.client.1.smithi078.stdout:1/1: mknod c0 0 2024-04-26T04:52:41.877 INFO:tasks.workunit.client.1.smithi078.stdout:0/9: dwrite f0 [0,4194304] 0 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/0: dwrite - no filename 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/1: dread - no filename 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/2: dwrite - no filename 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/3: link - no file 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/4: dread - no filename 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/5: fsync - no filename 2024-04-26T04:52:41.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/6: write - no filename 2024-04-26T04:52:41.881 INFO:tasks.workunit.client.1.smithi078.stdout:2/1: symlink l0 0 2024-04-26T04:52:41.881 INFO:tasks.workunit.client.1.smithi078.stdout:2/2: chown l0 63373 1 2024-04-26T04:52:41.881 INFO:tasks.workunit.client.1.smithi078.stdout:2/3: chown l0 70862 1 2024-04-26T04:52:41.883 INFO:tasks.workunit.client.1.smithi078.stdout:1/2: creat f1 x:0 0 0 2024-04-26T04:52:41.885 INFO:tasks.workunit.client.1.smithi078.stdout:0/10: rename f0 to f1 0 2024-04-26T04:52:41.886 INFO:tasks.workunit.client.1.smithi078.stdout:0/11: read f1 [1080873,47878] 0 2024-04-26T04:52:41.889 INFO:tasks.workunit.client.1.smithi078.stdout:5/0: truncate - no filename 2024-04-26T04:52:41.891 INFO:tasks.workunit.client.1.smithi078.stdout:2/4: rename l0 to l1 0 2024-04-26T04:52:41.891 INFO:tasks.workunit.client.1.smithi078.stdout:2/5: dread - no filename 2024-04-26T04:52:41.892 INFO:tasks.workunit.client.1.smithi078.stdout:1/3: dwrite f1 [0,4194304] 0 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/2: getdents . 0 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/3: dread - no filename 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/4: stat - no entries 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/5: truncate - no filename 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/6: chown . 33 1 2024-04-26T04:52:41.893 INFO:tasks.workunit.client.1.smithi078.stdout:1/4: truncate f1 4379205 0 2024-04-26T04:52:41.894 INFO:tasks.workunit.client.1.smithi078.stdout:0/12: dread f1 [0,4194304] 0 2024-04-26T04:52:41.899 INFO:tasks.workunit.client.1.smithi078.stdout:6/0: read - no filename 2024-04-26T04:52:41.901 INFO:tasks.workunit.client.1.smithi078.stdout:4/7: mkdir d0 0 2024-04-26T04:52:41.901 INFO:tasks.workunit.client.1.smithi078.stdout:4/8: rename d0 to d0/d1 22 2024-04-26T04:52:41.902 INFO:tasks.workunit.client.1.smithi078.stdout:4/9: rename d0 to d0/d2 22 2024-04-26T04:52:41.902 INFO:tasks.workunit.client.1.smithi078.stdout:4/10: rename d0 to d0/d3 22 2024-04-26T04:52:41.907 INFO:tasks.workunit.client.1.smithi078.stdout:1/5: symlink l2 0 2024-04-26T04:52:41.908 INFO:tasks.workunit.client.1.smithi078.stdout:0/13: mkdir d2 0 2024-04-26T04:52:41.912 INFO:tasks.workunit.client.1.smithi078.stdout:7/0: read - no filename 2024-04-26T04:52:41.912 INFO:tasks.workunit.client.1.smithi078.stdout:7/1: rmdir - no directory 2024-04-26T04:52:41.917 INFO:tasks.workunit.client.1.smithi078.stdout:5/1: symlink l0 0 2024-04-26T04:52:41.919 INFO:tasks.workunit.client.1.smithi078.stdout:3/7: symlink l0 0 2024-04-26T04:52:41.921 INFO:tasks.workunit.client.1.smithi078.stdout:1/6: creat f3 x:0 0 0 2024-04-26T04:52:41.925 INFO:tasks.workunit.client.1.smithi078.stdout:8/0: dwrite - no filename 2024-04-26T04:52:41.925 INFO:tasks.workunit.client.1.smithi078.stdout:8/1: rmdir - no directory 2024-04-26T04:52:41.927 INFO:tasks.workunit.client.1.smithi078.stdout:6/1: mknod c0 0 2024-04-26T04:52:41.928 INFO:tasks.workunit.client.1.smithi078.stdout:4/11: symlink d0/l4 0 2024-04-26T04:52:41.928 INFO:tasks.workunit.client.1.smithi078.stdout:4/12: write - no filename 2024-04-26T04:52:41.928 INFO:tasks.workunit.client.1.smithi078.stdout:4/13: dread - no filename 2024-04-26T04:52:41.929 INFO:tasks.workunit.client.1.smithi078.stdout:0/14: dwrite f1 [0,4194304] 0 2024-04-26T04:52:41.930 INFO:tasks.workunit.client.1.smithi078.stdout:5/2: creat f1 x:0 0 0 2024-04-26T04:52:41.931 INFO:tasks.workunit.client.1.smithi078.stdout:3/8: symlink l1 0 2024-04-26T04:52:41.934 INFO:tasks.workunit.client.1.smithi078.stdout:9/0: dread - no filename 2024-04-26T04:52:41.934 INFO:tasks.workunit.client.1.smithi078.stdout:9/1: read - no filename 2024-04-26T04:52:41.936 INFO:tasks.workunit.client.1.smithi078.stdout:7/2: creat f0 x:0 0 0 2024-04-26T04:52:41.938 INFO:tasks.workunit.client.1.smithi078.stdout:6/2: creat f1 x:0 0 0 2024-04-26T04:52:41.940 INFO:tasks.workunit.client.1.smithi078.stdout:4/14: unlink d0/l4 0 2024-04-26T04:52:41.940 INFO:tasks.workunit.client.1.smithi078.stdout:4/15: dwrite - no filename 2024-04-26T04:52:41.945 INFO:tasks.workunit.client.1.smithi078.stdout:5/3: symlink l2 0 2024-04-26T04:52:41.945 INFO:tasks.workunit.client.1.smithi078.stdout:7/3: dwrite f0 [0,4194304] 0 2024-04-26T04:52:41.947 INFO:tasks.workunit.client.1.smithi078.stdout:3/9: creat f2 x:0 0 0 2024-04-26T04:52:41.948 INFO:tasks.workunit.client.1.smithi078.stdout:3/10: dread - f2 zero size 2024-04-26T04:52:41.950 INFO:tasks.workunit.client.1.smithi078.stdout:6/3: dwrite f1 [0,4194304] 0 2024-04-26T04:52:41.951 INFO:tasks.workunit.client.1.smithi078.stdout:8/2: mkdir d0 0 2024-04-26T04:52:41.951 INFO:tasks.workunit.client.1.smithi078.stdout:8/3: dread - no filename 2024-04-26T04:52:41.953 INFO:tasks.workunit.client.1.smithi078.stdout:0/15: fdatasync f1 0 2024-04-26T04:52:41.956 INFO:tasks.workunit.client.1.smithi078.stdout:4/16: creat d0/f5 x:0 0 0 2024-04-26T04:52:41.959 INFO:tasks.workunit.client.1.smithi078.stdout:7/4: dwrite f0 [0,4194304] 0 2024-04-26T04:52:41.965 INFO:tasks.workunit.client.1.smithi078.stdout:9/2: mkdir d0 0 2024-04-26T04:52:41.967 INFO:tasks.workunit.client.1.smithi078.stdout:3/11: creat f3 x:0 0 0 2024-04-26T04:52:41.968 INFO:tasks.workunit.client.1.smithi078.stdout:4/17: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:52:41.979 INFO:tasks.workunit.client.1.smithi078.stdout:3/12: symlink l4 0 2024-04-26T04:52:41.981 INFO:tasks.workunit.client.1.smithi078.stdout:8/4: creat d0/f1 x:0 0 0 2024-04-26T04:52:41.982 INFO:tasks.workunit.client.1.smithi078.stdout:8/5: write d0/f1 [432824,85013] 0 2024-04-26T04:52:41.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/5: fdatasync f0 0 2024-04-26T04:52:41.987 INFO:tasks.workunit.client.1.smithi078.stdout:0/16: creat d2/f3 x:0 0 0 2024-04-26T04:52:41.987 INFO:tasks.workunit.client.1.smithi078.stdout:0/17: readlink - no filename 2024-04-26T04:52:41.989 INFO:tasks.workunit.client.1.smithi078.stdout:2/6: chown l1 297489 1 2024-04-26T04:52:41.993 INFO:tasks.workunit.client.1.smithi078.stdout:1/7: getdents . 0 2024-04-26T04:52:41.993 INFO:tasks.workunit.client.1.smithi078.stdout:1/8: write f1 [1348640,109620] 0 2024-04-26T04:52:41.994 INFO:tasks.workunit.client.1.smithi078.stdout:1/9: chown c0 21011316 1 2024-04-26T04:52:41.996 INFO:tasks.workunit.client.1.smithi078.stdout:7/6: rename f0 to f1 0 2024-04-26T04:52:41.997 INFO:tasks.workunit.client.1.smithi078.stdout:1/10: dread f1 [0,4194304] 0 2024-04-26T04:52:41.998 INFO:tasks.workunit.client.1.smithi078.stdout:4/18: link d0/f5 d0/f6 0 2024-04-26T04:52:41.998 INFO:tasks.workunit.client.1.smithi078.stdout:1/11: fsync f1 0 2024-04-26T04:52:41.999 INFO:tasks.workunit.client.1.smithi078.stdout:1/12: fdatasync f1 0 2024-04-26T04:52:42.003 INFO:tasks.workunit.client.1.smithi078.stdout:1/13: dread f1 [0,4194304] 0 2024-04-26T04:52:42.004 INFO:tasks.workunit.client.1.smithi078.stdout:1/14: chown c0 203 1 2024-04-26T04:52:42.004 INFO:tasks.workunit.client.1.smithi078.stdout:1/15: chown l2 491599 1 2024-04-26T04:52:42.005 INFO:tasks.workunit.client.1.smithi078.stdout:2/7: unlink l1 0 2024-04-26T04:52:42.008 INFO:tasks.workunit.client.1.smithi078.stdout:9/3: rmdir d0 0 2024-04-26T04:52:42.008 INFO:tasks.workunit.client.1.smithi078.stdout:9/4: truncate - no filename 2024-04-26T04:52:42.008 INFO:tasks.workunit.client.1.smithi078.stdout:9/5: dread - no filename 2024-04-26T04:52:42.015 INFO:tasks.workunit.client.1.smithi078.stdout:7/7: creat f2 x:0 0 0 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: pgmap v10: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:42.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:42.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:42.019 INFO:tasks.workunit.client.1.smithi078.stdout:4/19: mknod d0/c7 0 2024-04-26T04:52:42.020 INFO:tasks.workunit.client.1.smithi078.stdout:5/4: getdents . 0 2024-04-26T04:52:42.020 INFO:tasks.workunit.client.1.smithi078.stdout:5/5: dread - f1 zero size 2024-04-26T04:52:42.023 INFO:tasks.workunit.client.1.smithi078.stdout:0/18: link f1 d2/f4 0 2024-04-26T04:52:42.027 INFO:tasks.workunit.client.1.smithi078.stdout:2/8: symlink l2 0 2024-04-26T04:52:42.027 INFO:tasks.workunit.client.1.smithi078.stdout:2/9: readlink l2 0 2024-04-26T04:52:42.027 INFO:tasks.workunit.client.1.smithi078.stdout:2/10: truncate - no filename 2024-04-26T04:52:42.029 INFO:tasks.workunit.client.1.smithi078.stdout:7/8: fdatasync f1 0 2024-04-26T04:52:42.029 INFO:tasks.workunit.client.1.smithi078.stdout:7/9: rmdir - no directory 2024-04-26T04:52:42.031 INFO:tasks.workunit.client.1.smithi078.stdout:4/20: write d0/f5 [548809,130566] 0 2024-04-26T04:52:42.032 INFO:tasks.workunit.client.1.smithi078.stdout:4/21: write d0/f6 [3321346,66678] 0 2024-04-26T04:52:42.033 INFO:tasks.workunit.client.1.smithi078.stdout:5/6: rename f1 to f3 0 2024-04-26T04:52:42.034 INFO:tasks.workunit.client.1.smithi078.stdout:0/19: creat d2/f5 x:0 0 0 2024-04-26T04:52:42.037 INFO:tasks.workunit.client.1.smithi078.stdout:2/11: creat f3 x:0 0 0 2024-04-26T04:52:42.043 INFO:tasks.workunit.client.1.smithi078.stdout:4/22: creat d0/f8 x:0 0 0 2024-04-26T04:52:42.045 INFO:tasks.workunit.client.1.smithi078.stdout:2/12: symlink l4 0 2024-04-26T04:52:42.048 INFO:tasks.workunit.client.1.smithi078.stdout:4/23: mknod d0/c9 0 2024-04-26T04:52:42.048 INFO:tasks.workunit.client.1.smithi078.stdout:8/6: rmdir d0 39 2024-04-26T04:52:42.050 INFO:tasks.workunit.client.1.smithi078.stdout:6/4: truncate f1 1605699 0 2024-04-26T04:52:42.050 INFO:tasks.workunit.client.1.smithi078.stdout:6/5: chown f1 9 1 2024-04-26T04:52:42.051 INFO:tasks.workunit.client.1.smithi078.stdout:6/6: chown c0 8253774 1 2024-04-26T04:52:42.051 INFO:tasks.workunit.client.1.smithi078.stdout:0/20: link f1 d2/f6 0 2024-04-26T04:52:42.051 INFO:tasks.workunit.client.1.smithi078.stdout:6/7: chown c0 6116 1 2024-04-26T04:52:42.051 INFO:tasks.workunit.client.1.smithi078.stdout:0/21: chown d2/f6 382 1 2024-04-26T04:52:42.052 INFO:tasks.workunit.client.1.smithi078.stdout:0/22: write d2/f5 [188828,17326] 0 2024-04-26T04:52:42.054 INFO:tasks.workunit.client.1.smithi078.stdout:4/24: creat d0/fa x:0 0 0 2024-04-26T04:52:42.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/8: mkdir d2 0 2024-04-26T04:52:42.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/9: chown f1 3235025 1 2024-04-26T04:52:42.058 INFO:tasks.workunit.client.1.smithi078.stdout:0/23: creat d2/f7 x:0 0 0 2024-04-26T04:52:42.070 INFO:tasks.workunit.client.1.smithi078.stdout:7/10: unlink f1 0 2024-04-26T04:52:42.075 INFO:tasks.workunit.client.1.smithi078.stdout:7/11: dwrite f2 [0,4194304] 0 2024-04-26T04:52:42.076 INFO:tasks.workunit.client.1.smithi078.stdout:0/24: link d2/f5 d2/f8 0 2024-04-26T04:52:42.077 INFO:tasks.workunit.client.1.smithi078.stdout:0/25: readlink - no filename 2024-04-26T04:52:42.077 INFO:tasks.workunit.client.1.smithi078.stdout:0/26: chown d2/f7 204200325 1 2024-04-26T04:52:42.078 INFO:tasks.workunit.client.1.smithi078.stdout:4/25: link d0/c7 d0/cb 0 2024-04-26T04:52:42.079 INFO:tasks.workunit.client.1.smithi078.stdout:4/26: read - d0/f8 zero size 2024-04-26T04:52:42.079 INFO:tasks.workunit.client.1.smithi078.stdout:7/12: dread f2 [0,4194304] 0 2024-04-26T04:52:42.080 INFO:tasks.workunit.client.1.smithi078.stdout:7/13: write f2 [373387,10608] 0 2024-04-26T04:52:42.081 INFO:tasks.workunit.client.1.smithi078.stdout:7/14: truncate f2 4739599 0 2024-04-26T04:52:42.082 INFO:tasks.workunit.client.1.smithi078.stdout:8/7: getdents d0 0 2024-04-26T04:52:42.084 INFO:tasks.workunit.client.1.smithi078.stdout:7/15: dread f2 [0,4194304] 0 2024-04-26T04:52:42.089 INFO:tasks.workunit.client.1.smithi078.stdout:4/27: creat d0/fc x:0 0 0 2024-04-26T04:52:42.092 INFO:tasks.workunit.client.1.smithi078.stdout:8/8: unlink d0/f1 0 2024-04-26T04:52:42.092 INFO:tasks.workunit.client.1.smithi078.stdout:8/9: dread - no filename 2024-04-26T04:52:42.095 INFO:tasks.workunit.client.1.smithi078.stdout:7/16: symlink l3 0 2024-04-26T04:52:42.097 INFO:tasks.workunit.client.1.smithi078.stdout:1/16: truncate f1 2779603 0 2024-04-26T04:52:42.097 INFO:tasks.workunit.client.1.smithi078.stdout:0/27: truncate d2/f4 3182914 0 2024-04-26T04:52:42.098 INFO:tasks.workunit.client.1.smithi078.stdout:0/28: dread - d2/f7 zero size 2024-04-26T04:52:42.099 INFO:tasks.workunit.client.1.smithi078.stdout:4/28: mkdir d0/dd 0 2024-04-26T04:52:42.102 INFO:tasks.workunit.client.1.smithi078.stdout:8/10: symlink d0/l2 0 2024-04-26T04:52:42.102 INFO:tasks.workunit.client.1.smithi078.stdout:8/11: truncate - no filename 2024-04-26T04:52:42.104 INFO:tasks.workunit.client.1.smithi078.stdout:7/17: creat f4 x:0 0 0 2024-04-26T04:52:42.107 INFO:tasks.workunit.client.1.smithi078.stdout:0/29: mkdir d2/d9 0 2024-04-26T04:52:42.109 INFO:tasks.workunit.client.1.smithi078.stdout:4/29: mkdir d0/de 0 2024-04-26T04:52:42.111 INFO:tasks.workunit.client.1.smithi078.stdout:6/10: truncate f1 2198089 0 2024-04-26T04:52:42.114 INFO:tasks.workunit.client.1.smithi078.stdout:7/18: write f2 [114495,102366] 0 2024-04-26T04:52:42.117 INFO:tasks.workunit.client.1.smithi078.stdout:0/30: dwrite d2/f3 [0,4194304] 0 2024-04-26T04:52:42.124 INFO:tasks.workunit.client.1.smithi078.stdout:4/30: creat d0/de/ff x:0 0 0 2024-04-26T04:52:42.126 INFO:tasks.workunit.client.1.smithi078.stdout:4/31: dread - d0/f8 zero size 2024-04-26T04:52:42.128 INFO:tasks.workunit.client.1.smithi078.stdout:7/19: dwrite f2 [0,4194304] 0 2024-04-26T04:52:42.130 INFO:tasks.workunit.client.1.smithi078.stdout:0/31: mkdir d2/d9/da 0 2024-04-26T04:52:42.133 INFO:tasks.workunit.client.1.smithi078.stdout:4/32: mkdir d0/d10 0 2024-04-26T04:52:42.135 INFO:tasks.workunit.client.1.smithi078.stdout:7/20: unlink l3 0 2024-04-26T04:52:42.137 INFO:tasks.workunit.client.1.smithi078.stdout:7/21: chown f2 4143485 1 2024-04-26T04:52:42.137 INFO:tasks.workunit.client.1.smithi078.stdout:7/22: readlink - no filename 2024-04-26T04:52:42.138 INFO:tasks.workunit.client.1.smithi078.stdout:0/32: creat d2/d9/fb x:0 0 0 2024-04-26T04:52:42.139 INFO:tasks.workunit.client.1.smithi078.stdout:7/23: chown f2 157944578 1 2024-04-26T04:52:42.140 INFO:tasks.workunit.client.1.smithi078.stdout:7/24: chown f4 188740 1 2024-04-26T04:52:42.141 INFO:tasks.workunit.client.1.smithi078.stdout:7/25: chown f2 39437 1 2024-04-26T04:52:42.142 INFO:tasks.workunit.client.1.smithi078.stdout:4/33: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:52:42.143 INFO:tasks.workunit.client.1.smithi078.stdout:0/33: dread d2/f5 [0,4194304] 0 2024-04-26T04:52:42.231 INFO:tasks.workunit.client.1.smithi078.stdout:4/34: symlink d0/l11 0 2024-04-26T04:52:42.232 INFO:tasks.workunit.client.1.smithi078.stdout:4/35: readlink d0/l11 0 2024-04-26T04:52:42.232 INFO:tasks.workunit.client.1.smithi078.stdout:4/36: fsync d0/f6 0 2024-04-26T04:52:42.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/34: fdatasync d2/f8 0 2024-04-26T04:52:42.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/35: read d2/f3 [2754644,14560] 0 2024-04-26T04:52:42.237 INFO:tasks.workunit.client.1.smithi078.stdout:0/36: fsync d2/f3 0 2024-04-26T04:52:42.239 INFO:tasks.workunit.client.1.smithi078.stdout:7/26: link f4 f5 0 2024-04-26T04:52:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: pgmap v10: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-26T04:52:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:42.300 INFO:tasks.workunit.client.0.smithi077.stdout:6/37: symlink d8/l10 0 2024-04-26T04:52:42.301 INFO:tasks.workunit.client.1.smithi078.stdout:8/12: rmdir d0 39 2024-04-26T04:52:42.302 INFO:tasks.workunit.client.0.smithi077.stdout:9/27: mkdir d9/dd 0 2024-04-26T04:52:42.303 INFO:tasks.workunit.client.1.smithi078.stdout:0/37: dwrite d2/f7 [0,4194304] 0 2024-04-26T04:52:42.304 INFO:tasks.workunit.client.0.smithi077.stdout:9/28: chown c4 1185427 1 2024-04-26T04:52:42.304 INFO:tasks.workunit.client.0.smithi077.stdout:8/21: rename d4/la to d4/lc 0 2024-04-26T04:52:42.305 INFO:tasks.workunit.client.1.smithi078.stdout:1/17: write f1 [807506,108590] 0 2024-04-26T04:52:42.306 INFO:tasks.workunit.client.1.smithi078.stdout:4/37: rename d0/cb to d0/d10/c12 0 2024-04-26T04:52:42.307 INFO:tasks.workunit.client.0.smithi077.stdout:2/15: rename l4 to l6 0 2024-04-26T04:52:42.310 INFO:tasks.workunit.client.1.smithi078.stdout:6/11: dwrite f1 [0,4194304] 0 2024-04-26T04:52:42.311 INFO:tasks.workunit.client.1.smithi078.stdout:8/13: chown d0/l2 90237 1 2024-04-26T04:52:42.311 INFO:tasks.workunit.client.1.smithi078.stdout:8/14: write - no filename 2024-04-26T04:52:42.311 INFO:tasks.workunit.client.0.smithi077.stdout:6/38: dwrite d8/fe [0,4194304] 0 2024-04-26T04:52:42.313 INFO:tasks.workunit.client.0.smithi077.stdout:8/22: creat d4/fd x:0 0 0 2024-04-26T04:52:42.316 INFO:tasks.workunit.client.1.smithi078.stdout:1/18: unlink c0 0 2024-04-26T04:52:42.318 INFO:tasks.workunit.client.0.smithi077.stdout:9/29: dwrite d9/fc [0,4194304] 0 2024-04-26T04:52:42.320 INFO:tasks.workunit.client.0.smithi077.stdout:9/30: readlink d9/la 0 2024-04-26T04:52:42.322 INFO:tasks.workunit.client.1.smithi078.stdout:8/15: mkdir d0/d3 0 2024-04-26T04:52:42.322 INFO:tasks.workunit.client.1.smithi078.stdout:8/16: dwrite - no filename 2024-04-26T04:52:42.322 INFO:tasks.workunit.client.0.smithi077.stdout:8/23: creat d4/fe x:0 0 0 2024-04-26T04:52:42.325 INFO:tasks.workunit.client.0.smithi077.stdout:2/16: dwrite f5 [0,4194304] 0 2024-04-26T04:52:42.326 INFO:tasks.workunit.client.1.smithi078.stdout:7/27: dwrite f4 [0,4194304] 0 2024-04-26T04:52:42.326 INFO:tasks.workunit.client.1.smithi078.stdout:7/28: rmdir - no directory 2024-04-26T04:52:42.327 INFO:tasks.workunit.client.0.smithi077.stdout:6/39: link f3 d8/f11 0 2024-04-26T04:52:42.332 INFO:tasks.workunit.client.1.smithi078.stdout:1/19: rename l2 to l4 0 2024-04-26T04:52:42.336 INFO:tasks.workunit.client.0.smithi077.stdout:8/24: mkdir d4/df 0 2024-04-26T04:52:42.338 INFO:tasks.workunit.client.0.smithi077.stdout:8/25: write f1 [319050,94798] 0 2024-04-26T04:52:42.338 INFO:tasks.workunit.client.1.smithi078.stdout:8/17: mknod d0/c4 0 2024-04-26T04:52:42.339 INFO:tasks.workunit.client.1.smithi078.stdout:6/12: fsync f1 0 2024-04-26T04:52:42.340 INFO:tasks.workunit.client.1.smithi078.stdout:6/13: readlink - no filename 2024-04-26T04:52:42.342 INFO:tasks.workunit.client.0.smithi077.stdout:6/40: symlink d8/l12 0 2024-04-26T04:52:42.343 INFO:tasks.workunit.client.0.smithi077.stdout:6/41: read d8/fe [367115,2012] 0 2024-04-26T04:52:42.344 INFO:tasks.workunit.client.1.smithi078.stdout:1/20: creat f5 x:0 0 0 2024-04-26T04:52:42.344 INFO:tasks.workunit.client.1.smithi078.stdout:1/21: fdatasync f1 0 2024-04-26T04:52:42.344 INFO:tasks.workunit.client.0.smithi077.stdout:9/31: symlink d9/dd/le 0 2024-04-26T04:52:42.345 INFO:tasks.workunit.client.0.smithi077.stdout:9/32: chown d9/dd 162949 1 2024-04-26T04:52:42.346 INFO:tasks.workunit.client.1.smithi078.stdout:8/18: creat d0/f5 x:0 0 0 2024-04-26T04:52:42.346 INFO:tasks.workunit.client.1.smithi078.stdout:8/19: readlink d0/l2 0 2024-04-26T04:52:42.347 INFO:tasks.workunit.client.1.smithi078.stdout:8/20: chown d0/f5 124242 1 2024-04-26T04:52:42.347 INFO:tasks.workunit.client.0.smithi077.stdout:8/26: creat d4/f10 x:0 0 0 2024-04-26T04:52:42.348 INFO:tasks.workunit.client.0.smithi077.stdout:8/27: write d4/fd [118895,72584] 0 2024-04-26T04:52:42.349 INFO:tasks.workunit.client.1.smithi078.stdout:7/29: link f4 f6 0 2024-04-26T04:52:42.352 INFO:tasks.workunit.client.0.smithi077.stdout:6/42: creat d8/f13 x:0 0 0 2024-04-26T04:52:42.353 INFO:tasks.workunit.client.0.smithi077.stdout:6/43: dread d8/f11 [0,4194304] 0 2024-04-26T04:52:42.354 INFO:tasks.workunit.client.0.smithi077.stdout:6/44: chown d8/f13 2 1 2024-04-26T04:52:42.356 INFO:tasks.workunit.client.0.smithi077.stdout:9/33: unlink d9/fb 0 2024-04-26T04:52:42.356 INFO:tasks.workunit.client.0.smithi077.stdout:9/34: stat d9/dd 0 2024-04-26T04:52:42.357 INFO:tasks.workunit.client.1.smithi078.stdout:1/22: mknod c6 0 2024-04-26T04:52:42.361 INFO:tasks.workunit.client.1.smithi078.stdout:6/14: mkdir d2/d3 0 2024-04-26T04:52:42.363 INFO:tasks.workunit.client.1.smithi078.stdout:1/23: mkdir d7 0 2024-04-26T04:52:42.364 INFO:tasks.workunit.client.0.smithi077.stdout:6/45: symlink d8/l14 0 2024-04-26T04:52:42.367 INFO:tasks.workunit.client.1.smithi078.stdout:7/30: link f2 f7 0 2024-04-26T04:52:42.367 INFO:tasks.workunit.client.0.smithi077.stdout:8/28: symlink d4/df/l11 0 2024-04-26T04:52:42.368 INFO:tasks.workunit.client.1.smithi078.stdout:6/15: symlink d2/l4 0 2024-04-26T04:52:42.370 INFO:tasks.workunit.client.1.smithi078.stdout:7/31: dread f4 [0,4194304] 0 2024-04-26T04:52:42.371 INFO:tasks.workunit.client.1.smithi078.stdout:8/21: link d0/l2 d0/d3/l6 0 2024-04-26T04:52:42.372 INFO:tasks.workunit.client.1.smithi078.stdout:8/22: dread - d0/f5 zero size 2024-04-26T04:52:42.373 INFO:tasks.workunit.client.0.smithi077.stdout:8/29: chown d4/c6 218704340 1 2024-04-26T04:52:42.375 INFO:tasks.workunit.client.0.smithi077.stdout:6/46: dwrite d8/fe [0,4194304] 0 2024-04-26T04:52:42.375 INFO:tasks.workunit.client.0.smithi077.stdout:9/35: link c6 d9/dd/cf 0 2024-04-26T04:52:42.377 INFO:tasks.workunit.client.1.smithi078.stdout:7/32: symlink l8 0 2024-04-26T04:52:42.378 INFO:tasks.workunit.client.1.smithi078.stdout:7/33: chown f5 42034 1 2024-04-26T04:52:42.380 INFO:tasks.workunit.client.1.smithi078.stdout:8/23: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:52:42.381 INFO:tasks.workunit.client.0.smithi077.stdout:9/36: creat d9/f10 x:0 0 0 2024-04-26T04:52:42.383 INFO:tasks.workunit.client.0.smithi077.stdout:8/30: dwrite d4/f9 [4194304,4194304] 0 2024-04-26T04:52:42.386 INFO:tasks.workunit.client.1.smithi078.stdout:7/34: mkdir d9 0 2024-04-26T04:52:42.388 INFO:tasks.workunit.client.1.smithi078.stdout:8/24: unlink d0/c4 0 2024-04-26T04:52:42.391 INFO:tasks.workunit.client.1.smithi078.stdout:7/35: write f6 [3430908,2130] 0 2024-04-26T04:52:42.392 INFO:tasks.workunit.client.1.smithi078.stdout:7/36: chown f7 2059771 1 2024-04-26T04:52:42.393 INFO:tasks.workunit.client.1.smithi078.stdout:8/25: mkdir d0/d7 0 2024-04-26T04:52:42.396 INFO:tasks.workunit.client.0.smithi077.stdout:9/37: dwrite f8 [0,4194304] 0 2024-04-26T04:52:42.399 INFO:tasks.workunit.client.0.smithi077.stdout:8/31: dwrite f1 [0,4194304] 0 2024-04-26T04:52:42.399 INFO:tasks.workunit.client.1.smithi078.stdout:7/37: dwrite f4 [0,4194304] 0 2024-04-26T04:52:42.399 INFO:tasks.workunit.client.1.smithi078.stdout:8/26: mknod d0/d3/c8 0 2024-04-26T04:52:42.401 INFO:tasks.workunit.client.0.smithi077.stdout:8/32: chown d4/f5 8 1 2024-04-26T04:52:42.403 INFO:tasks.workunit.client.0.smithi077.stdout:9/38: rename f8 to d9/f11 0 2024-04-26T04:52:42.413 INFO:tasks.workunit.client.0.smithi077.stdout:9/39: rmdir d9/dd 39 2024-04-26T04:52:42.415 INFO:tasks.workunit.client.0.smithi077.stdout:8/33: dread f0 [0,4194304] 0 2024-04-26T04:52:42.419 INFO:tasks.workunit.client.0.smithi077.stdout:8/34: creat d4/df/f12 x:0 0 0 2024-04-26T04:52:42.427 INFO:tasks.workunit.client.0.smithi077.stdout:8/35: dwrite d4/f5 [0,4194304] 0 2024-04-26T04:52:42.432 INFO:tasks.workunit.client.0.smithi077.stdout:2/17: chown l6 157 1 2024-04-26T04:52:42.433 INFO:tasks.workunit.client.1.smithi078.stdout:6/16: truncate f1 670172 0 2024-04-26T04:52:42.437 INFO:tasks.workunit.client.0.smithi077.stdout:2/18: dwrite f5 [0,4194304] 0 2024-04-26T04:52:42.441 INFO:tasks.workunit.client.0.smithi077.stdout:9/40: rename d9/dd/cf to d9/c12 0 2024-04-26T04:52:42.442 INFO:tasks.workunit.client.0.smithi077.stdout:9/41: dread - f7 zero size 2024-04-26T04:52:42.444 INFO:tasks.workunit.client.0.smithi077.stdout:2/19: creat f7 x:0 0 0 2024-04-26T04:52:42.446 INFO:tasks.workunit.client.0.smithi077.stdout:8/36: rename f1 to d4/f13 0 2024-04-26T04:52:42.448 INFO:tasks.workunit.client.0.smithi077.stdout:9/42: rmdir d9/dd 39 2024-04-26T04:52:42.448 INFO:tasks.workunit.client.0.smithi077.stdout:8/37: dread d4/fd [0,4194304] 0 2024-04-26T04:52:42.450 INFO:tasks.workunit.client.0.smithi077.stdout:2/20: creat f8 x:0 0 0 2024-04-26T04:52:42.457 INFO:tasks.workunit.client.0.smithi077.stdout:6/47: truncate f3 134762 0 2024-04-26T04:52:42.460 INFO:tasks.workunit.client.0.smithi077.stdout:2/21: mkdir d9 0 2024-04-26T04:52:42.463 INFO:tasks.workunit.client.0.smithi077.stdout:6/48: dwrite d8/fe [0,4194304] 0 2024-04-26T04:52:42.465 INFO:tasks.workunit.client.0.smithi077.stdout:9/43: symlink d9/dd/l13 0 2024-04-26T04:52:42.478 INFO:tasks.workunit.client.0.smithi077.stdout:2/22: mknod d9/ca 0 2024-04-26T04:52:42.484 INFO:tasks.workunit.client.0.smithi077.stdout:9/44: mknod d9/c14 0 2024-04-26T04:52:42.485 INFO:tasks.workunit.client.0.smithi077.stdout:2/23: dwrite f5 [0,4194304] 0 2024-04-26T04:52:42.485 INFO:tasks.workunit.client.0.smithi077.stdout:9/45: write f3 [767832,75931] 0 2024-04-26T04:52:42.485 INFO:tasks.workunit.client.0.smithi077.stdout:2/24: chown d9/ca 254 1 2024-04-26T04:52:42.494 INFO:tasks.workunit.client.0.smithi077.stdout:2/25: dwrite f7 [0,4194304] 0 2024-04-26T04:52:42.495 INFO:tasks.workunit.client.0.smithi077.stdout:2/26: readlink l6 0 2024-04-26T04:52:42.495 INFO:tasks.workunit.client.0.smithi077.stdout:2/27: chown l6 11962582 1 2024-04-26T04:52:42.503 INFO:tasks.workunit.client.0.smithi077.stdout:9/46: rename c6 to d9/c15 0 2024-04-26T04:52:42.504 INFO:tasks.workunit.client.0.smithi077.stdout:9/47: readlink d9/la 0 2024-04-26T04:52:42.511 INFO:tasks.workunit.client.0.smithi077.stdout:2/28: link d9/ca d9/cb 0 2024-04-26T04:52:42.512 INFO:tasks.workunit.client.0.smithi077.stdout:9/48: dwrite d9/fc [4194304,4194304] 0 2024-04-26T04:52:42.520 INFO:tasks.workunit.client.0.smithi077.stdout:9/49: rename c5 to d9/dd/c16 0 2024-04-26T04:52:42.521 INFO:tasks.workunit.client.0.smithi077.stdout:2/29: dwrite f8 [0,4194304] 0 2024-04-26T04:52:42.531 INFO:tasks.workunit.client.0.smithi077.stdout:8/38: truncate d4/f9 2155091 0 2024-04-26T04:52:42.532 INFO:tasks.workunit.client.0.smithi077.stdout:8/39: chown d4/df/l11 425131 1 2024-04-26T04:52:42.537 INFO:tasks.workunit.client.0.smithi077.stdout:9/50: link d9/f10 d9/f17 0 2024-04-26T04:52:42.541 INFO:tasks.workunit.client.0.smithi077.stdout:9/51: fdatasync d9/f17 0 2024-04-26T04:52:42.548 INFO:tasks.workunit.client.0.smithi077.stdout:9/52: mknod d9/c18 0 2024-04-26T04:52:42.552 INFO:tasks.workunit.client.0.smithi077.stdout:9/53: dread d9/fc [0,4194304] 0 2024-04-26T04:52:42.553 INFO:tasks.workunit.client.0.smithi077.stdout:9/54: stat d9/la 0 2024-04-26T04:52:42.559 INFO:tasks.workunit.client.0.smithi077.stdout:9/55: symlink d9/l19 0 2024-04-26T04:52:42.568 INFO:tasks.workunit.client.0.smithi077.stdout:6/49: write d8/f11 [1072272,112229] 0 2024-04-26T04:52:42.571 INFO:tasks.workunit.client.0.smithi077.stdout:6/50: mkdir d8/d15 0 2024-04-26T04:52:42.577 INFO:tasks.workunit.client.0.smithi077.stdout:2/30: truncate f5 2975009 0 2024-04-26T04:52:42.578 INFO:tasks.workunit.client.0.smithi077.stdout:6/51: dwrite d8/f11 [0,4194304] 0 2024-04-26T04:52:42.586 INFO:tasks.workunit.client.0.smithi077.stdout:6/52: dwrite f4 [0,4194304] 0 2024-04-26T04:52:42.596 INFO:tasks.workunit.client.0.smithi077.stdout:6/53: dwrite d8/ff [0,4194304] 0 2024-04-26T04:52:42.596 INFO:tasks.workunit.client.0.smithi077.stdout:6/54: fsync d8/f13 0 2024-04-26T04:52:42.601 INFO:tasks.workunit.client.0.smithi077.stdout:6/55: dread f4 [0,4194304] 0 2024-04-26T04:52:42.602 INFO:tasks.workunit.client.0.smithi077.stdout:6/56: stat d8/ff 0 2024-04-26T04:52:42.603 INFO:tasks.workunit.client.0.smithi077.stdout:6/57: write d8/f11 [2524427,110969] 0 2024-04-26T04:52:42.604 INFO:tasks.workunit.client.0.smithi077.stdout:9/56: rmdir d9 39 2024-04-26T04:52:42.616 INFO:tasks.workunit.client.0.smithi077.stdout:6/58: fsync d8/f11 0 2024-04-26T04:52:42.616 INFO:tasks.workunit.client.0.smithi077.stdout:6/59: dread - d8/fb zero size 2024-04-26T04:52:42.617 INFO:tasks.workunit.client.0.smithi077.stdout:6/60: write d8/fe [3655846,40889] 0 2024-04-26T04:52:42.617 INFO:tasks.workunit.client.0.smithi077.stdout:8/40: truncate d4/f9 1344590 0 2024-04-26T04:52:42.618 INFO:tasks.workunit.client.0.smithi077.stdout:6/61: dread - d8/f13 zero size 2024-04-26T04:52:42.625 INFO:tasks.workunit.client.0.smithi077.stdout:6/62: dwrite d8/ff [0,4194304] 0 2024-04-26T04:52:42.632 INFO:tasks.workunit.client.0.smithi077.stdout:6/63: mknod d8/c16 0 2024-04-26T04:52:42.632 INFO:tasks.workunit.client.0.smithi077.stdout:6/64: readlink l6 0 2024-04-26T04:52:42.635 INFO:tasks.workunit.client.0.smithi077.stdout:6/65: creat d8/f17 x:0 0 0 2024-04-26T04:52:42.644 INFO:tasks.workunit.client.0.smithi077.stdout:6/66: rmdir d8/d15 0 2024-04-26T04:52:42.689 INFO:tasks.workunit.client.0.smithi077.stdout:8/41: dwrite d4/f9 [0,4194304] 0 2024-04-26T04:52:42.695 INFO:tasks.workunit.client.0.smithi077.stdout:8/42: dwrite d4/f5 [0,4194304] 0 2024-04-26T04:52:42.700 INFO:tasks.workunit.client.0.smithi077.stdout:8/43: mknod d4/c14 0 2024-04-26T04:52:42.700 INFO:tasks.workunit.client.0.smithi077.stdout:8/44: fdatasync d4/f10 0 2024-04-26T04:52:42.767 INFO:tasks.workunit.client.1.smithi078.stdout:7/38: mkdir d9/da 0 2024-04-26T04:52:42.768 INFO:tasks.workunit.client.1.smithi078.stdout:4/38: unlink d0/d10/c12 0 2024-04-26T04:52:42.769 INFO:tasks.workunit.client.1.smithi078.stdout:1/24: rename l4 to d7/l8 0 2024-04-26T04:52:42.770 INFO:tasks.workunit.client.1.smithi078.stdout:7/39: dread f5 [0,4194304] 0 2024-04-26T04:52:42.773 INFO:tasks.workunit.client.1.smithi078.stdout:1/25: mkdir d7/d9 0 2024-04-26T04:52:42.773 INFO:tasks.workunit.client.1.smithi078.stdout:7/40: write f5 [3146976,19225] 0 2024-04-26T04:52:42.773 INFO:tasks.workunit.client.1.smithi078.stdout:4/39: stat d0/c7 0 2024-04-26T04:52:42.774 INFO:tasks.workunit.client.1.smithi078.stdout:4/40: dread - d0/fc zero size 2024-04-26T04:52:42.776 INFO:tasks.workunit.client.1.smithi078.stdout:8/27: rename d0/l2 to d0/d7/l9 0 2024-04-26T04:52:42.776 INFO:tasks.workunit.client.1.smithi078.stdout:8/28: chown d0/d3 89 1 2024-04-26T04:52:42.777 INFO:tasks.workunit.client.1.smithi078.stdout:4/41: creat d0/f13 x:0 0 0 2024-04-26T04:52:42.778 INFO:tasks.workunit.client.1.smithi078.stdout:1/26: getdents d7/d9 0 2024-04-26T04:52:42.778 INFO:tasks.workunit.client.1.smithi078.stdout:4/42: rename d0/dd to d0/dd/d14 22 2024-04-26T04:52:42.779 INFO:tasks.workunit.client.1.smithi078.stdout:1/27: write f1 [3417167,38976] 0 2024-04-26T04:52:42.780 INFO:tasks.workunit.client.1.smithi078.stdout:4/43: creat d0/d10/f15 x:0 0 0 2024-04-26T04:52:42.783 INFO:tasks.workunit.client.1.smithi078.stdout:4/44: mknod d0/c16 0 2024-04-26T04:52:42.783 INFO:tasks.workunit.client.1.smithi078.stdout:8/29: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:52:42.785 INFO:tasks.workunit.client.1.smithi078.stdout:4/45: truncate d0/f8 671936 0 2024-04-26T04:52:42.786 INFO:tasks.workunit.client.1.smithi078.stdout:4/46: read - d0/fa zero size 2024-04-26T04:52:42.788 INFO:tasks.workunit.client.1.smithi078.stdout:1/28: dwrite f3 [0,4194304] 0 2024-04-26T04:52:42.789 INFO:tasks.workunit.client.1.smithi078.stdout:4/47: symlink d0/dd/l17 0 2024-04-26T04:52:42.798 INFO:tasks.workunit.client.1.smithi078.stdout:4/48: dwrite d0/d10/f15 [0,4194304] 0 2024-04-26T04:52:42.800 INFO:tasks.workunit.client.1.smithi078.stdout:4/49: symlink d0/d10/l18 0 2024-04-26T04:52:42.802 INFO:tasks.workunit.client.1.smithi078.stdout:4/50: unlink d0/fa 0 2024-04-26T04:52:42.805 INFO:tasks.workunit.client.1.smithi078.stdout:4/51: creat d0/dd/f19 x:0 0 0 2024-04-26T04:52:42.805 INFO:tasks.workunit.client.1.smithi078.stdout:4/52: chown d0/c16 153 1 2024-04-26T04:52:42.807 INFO:tasks.workunit.client.1.smithi078.stdout:4/53: creat d0/de/f1a x:0 0 0 2024-04-26T04:52:42.808 INFO:tasks.workunit.client.1.smithi078.stdout:4/54: truncate d0/de/f1a 329500 0 2024-04-26T04:52:42.810 INFO:tasks.workunit.client.1.smithi078.stdout:4/55: creat d0/d10/f1b x:0 0 0 2024-04-26T04:52:42.817 INFO:tasks.workunit.client.1.smithi078.stdout:4/56: dwrite d0/fc [0,4194304] 0 2024-04-26T04:52:42.819 INFO:tasks.workunit.client.1.smithi078.stdout:4/57: creat d0/de/f1c x:0 0 0 2024-04-26T04:52:42.820 INFO:tasks.workunit.client.1.smithi078.stdout:4/58: dread - d0/de/ff zero size 2024-04-26T04:52:42.824 INFO:tasks.workunit.client.1.smithi078.stdout:4/59: unlink d0/dd/l17 0 2024-04-26T04:52:42.824 INFO:tasks.workunit.client.1.smithi078.stdout:4/60: chown d0/d10 3106 1 2024-04-26T04:52:42.826 INFO:tasks.workunit.client.1.smithi078.stdout:4/61: mkdir d0/d1d 0 2024-04-26T04:52:42.827 INFO:tasks.workunit.client.1.smithi078.stdout:4/62: readlink d0/l11 0 2024-04-26T04:52:42.830 INFO:tasks.workunit.client.1.smithi078.stdout:7/41: getdents d9 0 2024-04-26T04:52:42.832 INFO:tasks.workunit.client.1.smithi078.stdout:6/17: write f1 [29817,49162] 0 2024-04-26T04:52:42.833 INFO:tasks.workunit.client.1.smithi078.stdout:1/29: getdents d7 0 2024-04-26T04:52:42.833 INFO:tasks.workunit.client.1.smithi078.stdout:6/18: chown f1 1243 1 2024-04-26T04:52:42.834 INFO:tasks.workunit.client.1.smithi078.stdout:7/42: dread f4 [0,4194304] 0 2024-04-26T04:52:42.836 INFO:tasks.workunit.client.1.smithi078.stdout:8/30: rename d0/d7/l9 to d0/d7/la 0 2024-04-26T04:52:42.838 INFO:tasks.workunit.client.1.smithi078.stdout:1/30: creat d7/d9/fa x:0 0 0 2024-04-26T04:52:42.839 INFO:tasks.workunit.client.1.smithi078.stdout:1/31: chown d7/d9/fa 13088 1 2024-04-26T04:52:42.839 INFO:tasks.workunit.client.1.smithi078.stdout:8/31: creat d0/fb x:0 0 0 2024-04-26T04:52:42.840 INFO:tasks.workunit.client.1.smithi078.stdout:6/19: mkdir d2/d3/d5 0 2024-04-26T04:52:42.841 INFO:tasks.workunit.client.1.smithi078.stdout:6/20: chown f1 475015 1 2024-04-26T04:52:42.841 INFO:tasks.workunit.client.1.smithi078.stdout:6/21: chown d2 454305002 1 2024-04-26T04:52:42.841 INFO:tasks.workunit.client.1.smithi078.stdout:7/43: creat d9/da/fb x:0 0 0 2024-04-26T04:52:42.842 INFO:tasks.workunit.client.1.smithi078.stdout:8/32: mknod d0/cc 0 2024-04-26T04:52:42.843 INFO:tasks.workunit.client.1.smithi078.stdout:1/32: mkdir d7/d9/db 0 2024-04-26T04:52:42.843 INFO:tasks.workunit.client.1.smithi078.stdout:6/22: creat d2/d3/f6 x:0 0 0 2024-04-26T04:52:42.844 INFO:tasks.workunit.client.1.smithi078.stdout:7/44: symlink d9/da/lc 0 2024-04-26T04:52:42.845 INFO:tasks.workunit.client.1.smithi078.stdout:8/33: mknod d0/d3/cd 0 2024-04-26T04:52:42.846 INFO:tasks.workunit.client.1.smithi078.stdout:6/23: symlink d2/d3/l7 0 2024-04-26T04:52:42.846 INFO:tasks.workunit.client.1.smithi078.stdout:6/24: dread - d2/d3/f6 zero size 2024-04-26T04:52:42.847 INFO:tasks.workunit.client.1.smithi078.stdout:7/45: unlink d9/da/lc 0 2024-04-26T04:52:42.847 INFO:tasks.workunit.client.1.smithi078.stdout:8/34: symlink d0/d3/le 0 2024-04-26T04:52:42.848 INFO:tasks.workunit.client.1.smithi078.stdout:8/35: write d0/f5 [5138129,69139] 0 2024-04-26T04:52:42.850 INFO:tasks.workunit.client.1.smithi078.stdout:7/46: rename f2 to d9/da/fd 0 2024-04-26T04:52:42.851 INFO:tasks.workunit.client.1.smithi078.stdout:6/25: creat d2/d3/d5/f8 x:0 0 0 2024-04-26T04:52:42.851 INFO:tasks.workunit.client.1.smithi078.stdout:6/26: chown d2 127998 1 2024-04-26T04:52:42.852 INFO:tasks.workunit.client.1.smithi078.stdout:7/47: write d9/da/fb [406772,19602] 0 2024-04-26T04:52:42.854 INFO:tasks.workunit.client.1.smithi078.stdout:7/48: chown f5 24856 1 2024-04-26T04:52:42.856 INFO:tasks.workunit.client.1.smithi078.stdout:8/36: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:52:42.856 INFO:tasks.workunit.client.1.smithi078.stdout:7/49: creat d9/fe x:0 0 0 2024-04-26T04:52:42.856 INFO:tasks.workunit.client.1.smithi078.stdout:1/33: rmdir d7/d9 39 2024-04-26T04:52:42.858 INFO:tasks.workunit.client.1.smithi078.stdout:8/37: mknod d0/cf 0 2024-04-26T04:52:42.860 INFO:tasks.workunit.client.1.smithi078.stdout:8/38: write d0/fb [452024,95513] 0 2024-04-26T04:52:42.860 INFO:tasks.workunit.client.1.smithi078.stdout:6/27: dwrite d2/d3/d5/f8 [0,4194304] 0 2024-04-26T04:52:42.861 INFO:tasks.workunit.client.1.smithi078.stdout:7/50: symlink d9/lf 0 2024-04-26T04:52:42.862 INFO:tasks.workunit.client.1.smithi078.stdout:7/51: dread - d9/fe zero size 2024-04-26T04:52:42.865 INFO:tasks.workunit.client.1.smithi078.stdout:6/28: dread d2/d3/d5/f8 [0,4194304] 0 2024-04-26T04:52:42.866 INFO:tasks.workunit.client.1.smithi078.stdout:8/39: creat d0/f10 x:0 0 0 2024-04-26T04:52:42.871 INFO:tasks.workunit.client.1.smithi078.stdout:7/52: creat d9/f10 x:0 0 0 2024-04-26T04:52:42.873 INFO:tasks.workunit.client.1.smithi078.stdout:8/40: write d0/f10 [1186,34183] 0 2024-04-26T04:52:42.873 INFO:tasks.workunit.client.1.smithi078.stdout:6/29: dwrite f1 [0,4194304] 0 2024-04-26T04:52:42.874 INFO:tasks.workunit.client.1.smithi078.stdout:6/30: chown d2/l4 883042123 1 2024-04-26T04:52:42.875 INFO:tasks.workunit.client.1.smithi078.stdout:1/34: creat d7/fc x:0 0 0 2024-04-26T04:52:42.876 INFO:tasks.workunit.client.1.smithi078.stdout:8/41: read d0/fb [425218,9735] 0 2024-04-26T04:52:42.881 INFO:tasks.workunit.client.1.smithi078.stdout:8/42: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:42.883 INFO:tasks.workunit.client.1.smithi078.stdout:6/31: mknod d2/d3/d5/c9 0 2024-04-26T04:52:42.888 INFO:tasks.workunit.client.1.smithi078.stdout:8/43: dwrite d0/f10 [0,4194304] 0 2024-04-26T04:52:42.889 INFO:tasks.workunit.client.1.smithi078.stdout:6/32: mkdir d2/d3/d5/da 0 2024-04-26T04:52:42.890 INFO:tasks.workunit.client.1.smithi078.stdout:1/35: mkdir d7/d9/dd 0 2024-04-26T04:52:42.895 INFO:tasks.workunit.client.1.smithi078.stdout:8/44: rename d0/d3/cd to d0/c11 0 2024-04-26T04:52:42.897 INFO:tasks.workunit.client.1.smithi078.stdout:7/53: getdents d9/da 0 2024-04-26T04:52:42.898 INFO:tasks.workunit.client.1.smithi078.stdout:8/45: write d0/fb [725866,30476] 0 2024-04-26T04:52:42.899 INFO:tasks.workunit.client.1.smithi078.stdout:7/54: read d9/da/fb [384298,118529] 0 2024-04-26T04:52:42.906 INFO:tasks.workunit.client.1.smithi078.stdout:7/55: write f4 [384091,22573] 0 2024-04-26T04:52:42.910 INFO:tasks.workunit.client.1.smithi078.stdout:6/33: getdents d2 0 2024-04-26T04:52:42.911 INFO:tasks.workunit.client.1.smithi078.stdout:7/56: fsync d9/da/fb 0 2024-04-26T04:52:42.921 INFO:tasks.workunit.client.1.smithi078.stdout:6/34: dwrite d2/d3/f6 [0,4194304] 0 2024-04-26T04:52:42.922 INFO:tasks.workunit.client.1.smithi078.stdout:8/46: rmdir d0 39 2024-04-26T04:52:42.924 INFO:tasks.workunit.client.1.smithi078.stdout:7/57: dwrite f6 [4194304,4194304] 0 2024-04-26T04:52:42.929 INFO:tasks.workunit.client.1.smithi078.stdout:7/58: mkdir d9/da/d11 0 2024-04-26T04:52:42.929 INFO:tasks.workunit.client.1.smithi078.stdout:7/59: chown d9 63900621 1 2024-04-26T04:52:42.930 INFO:tasks.workunit.client.1.smithi078.stdout:1/36: truncate f1 125827 0 2024-04-26T04:52:42.933 INFO:tasks.workunit.client.1.smithi078.stdout:8/47: stat d0/c11 0 2024-04-26T04:52:42.956 INFO:tasks.workunit.client.1.smithi078.stdout:6/35: mknod d2/d3/d5/da/cb 0 2024-04-26T04:52:42.957 INFO:tasks.workunit.client.1.smithi078.stdout:1/37: creat d7/d9/fe x:0 0 0 2024-04-26T04:52:42.957 INFO:tasks.workunit.client.1.smithi078.stdout:1/38: dread - f5 zero size 2024-04-26T04:52:42.959 INFO:tasks.workunit.client.1.smithi078.stdout:7/60: dwrite d9/fe [0,4194304] 0 2024-04-26T04:52:42.960 INFO:tasks.workunit.client.1.smithi078.stdout:7/61: chown f7 1199 1 2024-04-26T04:52:42.960 INFO:tasks.workunit.client.1.smithi078.stdout:8/48: mknod d0/d7/c12 0 2024-04-26T04:52:42.965 INFO:tasks.workunit.client.1.smithi078.stdout:1/39: dwrite d7/fc [0,4194304] 0 2024-04-26T04:52:42.973 INFO:tasks.workunit.client.1.smithi078.stdout:7/62: creat d9/da/d11/f12 x:0 0 0 2024-04-26T04:52:42.984 INFO:tasks.workunit.client.1.smithi078.stdout:1/40: creat d7/ff x:0 0 0 2024-04-26T04:52:42.984 INFO:tasks.workunit.client.1.smithi078.stdout:7/63: fsync d9/fe 0 2024-04-26T04:52:42.988 INFO:tasks.workunit.client.1.smithi078.stdout:7/64: mkdir d9/da/d11/d13 0 2024-04-26T04:52:42.990 INFO:tasks.workunit.client.1.smithi078.stdout:1/41: dwrite d7/d9/fe [0,4194304] 0 2024-04-26T04:52:42.995 INFO:tasks.workunit.client.1.smithi078.stdout:1/42: mknod d7/d9/c10 0 2024-04-26T04:52:42.996 INFO:tasks.workunit.client.1.smithi078.stdout:1/43: dread - d7/ff zero size 2024-04-26T04:52:42.996 INFO:tasks.workunit.client.1.smithi078.stdout:7/65: dwrite d9/da/d11/f12 [0,4194304] 0 2024-04-26T04:52:43.003 INFO:tasks.workunit.client.1.smithi078.stdout:7/66: dread d9/fe [0,4194304] 0 2024-04-26T04:52:43.005 INFO:tasks.workunit.client.1.smithi078.stdout:1/44: dwrite f5 [0,4194304] 0 2024-04-26T04:52:43.006 INFO:tasks.workunit.client.1.smithi078.stdout:1/45: read f3 [1553527,5542] 0 2024-04-26T04:52:43.011 INFO:tasks.workunit.client.1.smithi078.stdout:7/67: rename d9/da/fd to d9/f14 0 2024-04-26T04:52:43.017 INFO:tasks.workunit.client.1.smithi078.stdout:8/49: getdents d0/d7 0 2024-04-26T04:52:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:42 smithi078 ceph-mon[33746]: Adjusting osd_memory_target on smithi078 to 2581M 2024-04-26T04:52:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:42 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:52:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:42 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:52:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:42 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:52:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:42 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:52:43.018 INFO:tasks.workunit.client.1.smithi078.stdout:8/50: chown d0/d3 91173 1 2024-04-26T04:52:43.021 INFO:tasks.workunit.client.1.smithi078.stdout:1/46: fsync f5 0 2024-04-26T04:52:43.025 INFO:tasks.workunit.client.1.smithi078.stdout:8/51: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:43.037 INFO:tasks.workunit.client.1.smithi078.stdout:7/68: rmdir d9/da/d11 39 2024-04-26T04:52:43.039 INFO:tasks.workunit.client.1.smithi078.stdout:7/69: chown d9/da/d11/f12 794348 1 2024-04-26T04:52:43.046 INFO:tasks.workunit.client.1.smithi078.stdout:7/70: link f4 d9/da/f15 0 2024-04-26T04:52:43.047 INFO:tasks.workunit.client.1.smithi078.stdout:7/71: chown d9/da/d11/d13 36 1 2024-04-26T04:52:43.049 INFO:tasks.workunit.client.1.smithi078.stdout:7/72: mkdir d9/d16 0 2024-04-26T04:52:43.051 INFO:tasks.workunit.client.1.smithi078.stdout:7/73: write d9/da/d11/f12 [555748,43672] 0 2024-04-26T04:52:43.052 INFO:tasks.workunit.client.1.smithi078.stdout:7/74: chown f4 16 1 2024-04-26T04:52:43.054 INFO:tasks.workunit.client.1.smithi078.stdout:1/47: dwrite f5 [4194304,4194304] 0 2024-04-26T04:52:43.067 INFO:tasks.workunit.client.1.smithi078.stdout:8/52: write d0/f5 [5268471,19539] 0 2024-04-26T04:52:43.069 INFO:tasks.workunit.client.1.smithi078.stdout:8/53: readlink d0/d7/la 0 2024-04-26T04:52:43.070 INFO:tasks.workunit.client.1.smithi078.stdout:8/54: write d0/fb [236030,56408] 0 2024-04-26T04:52:43.073 INFO:tasks.workunit.client.1.smithi078.stdout:8/55: dread d0/f5 [4194304,4194304] 0 2024-04-26T04:52:43.077 INFO:tasks.workunit.client.1.smithi078.stdout:8/56: creat d0/f13 x:0 0 0 2024-04-26T04:52:43.077 INFO:tasks.workunit.client.1.smithi078.stdout:8/57: write d0/f10 [1948345,118452] 0 2024-04-26T04:52:43.079 INFO:tasks.workunit.client.1.smithi078.stdout:8/58: write d0/f10 [1001046,40619] 0 2024-04-26T04:52:43.085 INFO:tasks.workunit.client.1.smithi078.stdout:7/75: rmdir d9 39 2024-04-26T04:52:43.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/59: link d0/cc d0/d7/c14 0 2024-04-26T04:52:43.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/60: stat d0/fb 0 2024-04-26T04:52:43.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/61: chown d0 1 1 2024-04-26T04:52:43.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/62: readlink d0/d3/le 0 2024-04-26T04:52:43.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/63: chown d0/f10 1725102 1 2024-04-26T04:52:43.090 INFO:tasks.workunit.client.1.smithi078.stdout:8/64: dread - d0/f13 zero size 2024-04-26T04:52:43.092 INFO:tasks.workunit.client.1.smithi078.stdout:7/76: link d9/da/fb d9/da/d11/d13/f17 0 2024-04-26T04:52:43.096 INFO:tasks.workunit.client.1.smithi078.stdout:8/65: dwrite d0/fb [0,4194304] 0 2024-04-26T04:52:43.097 INFO:tasks.workunit.client.1.smithi078.stdout:8/66: readlink d0/d7/la 0 2024-04-26T04:52:43.098 INFO:tasks.workunit.client.1.smithi078.stdout:7/77: mknod d9/da/d11/d13/c18 0 2024-04-26T04:52:43.104 INFO:tasks.workunit.client.1.smithi078.stdout:7/78: mknod d9/d16/c19 0 2024-04-26T04:52:43.104 INFO:tasks.workunit.client.1.smithi078.stdout:8/67: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:43.105 INFO:tasks.workunit.client.1.smithi078.stdout:7/79: stat d9/da/d11/d13 0 2024-04-26T04:52:43.107 INFO:tasks.workunit.client.1.smithi078.stdout:1/48: dwrite f1 [0,4194304] 0 2024-04-26T04:52:43.118 INFO:tasks.workunit.client.1.smithi078.stdout:8/68: link d0/d7/c14 d0/d7/c15 0 2024-04-26T04:52:43.118 INFO:tasks.workunit.client.1.smithi078.stdout:8/69: rename d0 to d0/d16 22 2024-04-26T04:52:43.120 INFO:tasks.workunit.client.1.smithi078.stdout:1/49: link d7/d9/fa d7/f11 0 2024-04-26T04:52:43.122 INFO:tasks.workunit.client.1.smithi078.stdout:8/70: rename d0/d3/c8 to d0/c17 0 2024-04-26T04:52:43.122 INFO:tasks.workunit.client.1.smithi078.stdout:8/71: chown d0/d3 5 1 2024-04-26T04:52:43.123 INFO:tasks.workunit.client.1.smithi078.stdout:8/72: dread - d0/f13 zero size 2024-04-26T04:52:43.125 INFO:tasks.workunit.client.1.smithi078.stdout:8/73: write d0/f10 [3898729,31523] 0 2024-04-26T04:52:43.126 INFO:tasks.workunit.client.1.smithi078.stdout:1/50: dwrite f1 [0,4194304] 0 2024-04-26T04:52:43.129 INFO:tasks.workunit.client.1.smithi078.stdout:8/74: unlink d0/f13 0 2024-04-26T04:52:43.131 INFO:tasks.workunit.client.1.smithi078.stdout:1/51: rename d7/d9/fa to d7/d9/f12 0 2024-04-26T04:52:43.131 INFO:tasks.workunit.client.1.smithi078.stdout:1/52: chown d7/d9/dd 691 1 2024-04-26T04:52:43.135 INFO:tasks.workunit.client.1.smithi078.stdout:8/75: creat d0/f18 x:0 0 0 2024-04-26T04:52:43.136 INFO:tasks.workunit.client.1.smithi078.stdout:8/76: read d0/f5 [3151679,5635] 0 2024-04-26T04:52:43.140 INFO:tasks.workunit.client.1.smithi078.stdout:8/77: mkdir d0/d19 0 2024-04-26T04:52:43.143 INFO:tasks.workunit.client.1.smithi078.stdout:1/53: creat d7/f13 x:0 0 0 2024-04-26T04:52:43.145 INFO:tasks.workunit.client.1.smithi078.stdout:8/78: rename d0/fb to d0/f1a 0 2024-04-26T04:52:43.145 INFO:tasks.workunit.client.1.smithi078.stdout:8/79: chown d0/c11 488 1 2024-04-26T04:52:43.161 INFO:tasks.workunit.client.1.smithi078.stdout:1/54: rmdir d7/d9 39 2024-04-26T04:52:43.164 INFO:tasks.workunit.client.1.smithi078.stdout:8/80: fsync d0/f18 0 2024-04-26T04:52:43.167 INFO:tasks.workunit.client.1.smithi078.stdout:1/55: dwrite f5 [0,4194304] 0 2024-04-26T04:52:43.171 INFO:tasks.workunit.client.0.smithi077.stdout:8/45: dread d4/fd [0,4194304] 0 2024-04-26T04:52:43.171 INFO:tasks.workunit.client.1.smithi078.stdout:1/56: dread f5 [4194304,4194304] 0 2024-04-26T04:52:43.171 INFO:tasks.workunit.client.0.smithi077.stdout:8/46: fdatasync d4/f9 0 2024-04-26T04:52:43.176 INFO:tasks.workunit.client.1.smithi078.stdout:8/81: link d0/d3/le d0/d7/l1b 0 2024-04-26T04:52:43.178 INFO:tasks.workunit.client.0.smithi077.stdout:8/47: creat d4/df/f15 x:0 0 0 2024-04-26T04:52:43.180 INFO:tasks.workunit.client.1.smithi078.stdout:8/82: dread d0/f10 [0,4194304] 0 2024-04-26T04:52:43.181 INFO:tasks.workunit.client.0.smithi077.stdout:8/48: creat d4/df/f16 x:0 0 0 2024-04-26T04:52:43.181 INFO:tasks.workunit.client.0.smithi077.stdout:8/49: read - d4/df/f12 zero size 2024-04-26T04:52:43.182 INFO:tasks.workunit.client.1.smithi078.stdout:1/57: creat d7/d9/db/f14 x:0 0 0 2024-04-26T04:52:43.182 INFO:tasks.workunit.client.0.smithi077.stdout:8/50: dread - d4/df/f16 zero size 2024-04-26T04:52:43.182 INFO:tasks.workunit.client.1.smithi078.stdout:1/58: dread - d7/ff zero size 2024-04-26T04:52:43.188 INFO:tasks.workunit.client.1.smithi078.stdout:1/59: getdents d7 0 2024-04-26T04:52:43.189 INFO:tasks.workunit.client.1.smithi078.stdout:1/60: creat d7/d9/db/f15 x:0 0 0 2024-04-26T04:52:43.191 INFO:tasks.workunit.client.1.smithi078.stdout:1/61: getdents d7 0 2024-04-26T04:52:43.192 INFO:tasks.workunit.client.1.smithi078.stdout:1/62: write f5 [6963797,86162] 0 2024-04-26T04:52:43.193 INFO:tasks.workunit.client.1.smithi078.stdout:1/63: fsync d7/f13 0 2024-04-26T04:52:43.193 INFO:tasks.workunit.client.0.smithi077.stdout:8/51: getdents d4/df 0 2024-04-26T04:52:43.195 INFO:tasks.workunit.client.1.smithi078.stdout:1/64: fdatasync f3 0 2024-04-26T04:52:43.195 INFO:tasks.workunit.client.1.smithi078.stdout:8/83: write d0/f5 [1330120,56522] 0 2024-04-26T04:52:43.197 INFO:tasks.workunit.client.1.smithi078.stdout:1/65: rmdir d7/d9/db 39 2024-04-26T04:52:43.200 INFO:tasks.workunit.client.0.smithi077.stdout:8/52: getdents d4 0 2024-04-26T04:52:43.200 INFO:tasks.workunit.client.1.smithi078.stdout:1/66: mknod d7/d9/db/c16 0 2024-04-26T04:52:43.202 INFO:tasks.workunit.client.1.smithi078.stdout:1/67: mkdir d7/d9/d17 0 2024-04-26T04:52:43.203 INFO:tasks.workunit.client.0.smithi077.stdout:8/53: truncate d4/f13 1534098 0 2024-04-26T04:52:43.204 INFO:tasks.workunit.client.0.smithi077.stdout:8/54: mkdir d4/d17 0 2024-04-26T04:52:43.205 INFO:tasks.workunit.client.1.smithi078.stdout:1/68: dread d7/fc [0,4194304] 0 2024-04-26T04:52:43.206 INFO:tasks.workunit.client.0.smithi077.stdout:8/55: mkdir d4/df/d18 0 2024-04-26T04:52:43.209 INFO:tasks.workunit.client.0.smithi077.stdout:8/56: unlink d4/df/f12 0 2024-04-26T04:52:43.212 INFO:tasks.workunit.client.1.smithi078.stdout:1/69: dwrite d7/d9/db/f14 [0,4194304] 0 2024-04-26T04:52:43.215 INFO:tasks.workunit.client.1.smithi078.stdout:1/70: mknod d7/c18 0 2024-04-26T04:52:43.216 INFO:tasks.workunit.client.1.smithi078.stdout:1/71: dread - d7/f11 zero size 2024-04-26T04:52:43.217 INFO:tasks.workunit.client.0.smithi077.stdout:8/57: getdents d4 0 2024-04-26T04:52:43.221 INFO:tasks.workunit.client.0.smithi077.stdout:8/58: getdents d4/d17 0 2024-04-26T04:52:43.222 INFO:tasks.workunit.client.0.smithi077.stdout:8/59: truncate d4/df/f15 563914 0 2024-04-26T04:52:43.224 INFO:tasks.workunit.client.1.smithi078.stdout:1/72: dwrite d7/ff [0,4194304] 0 2024-04-26T04:52:43.225 INFO:tasks.workunit.client.0.smithi077.stdout:8/60: creat d4/df/d18/f19 x:0 0 0 2024-04-26T04:52:43.228 INFO:tasks.workunit.client.1.smithi078.stdout:1/73: creat d7/f19 x:0 0 0 2024-04-26T04:52:43.228 INFO:tasks.workunit.client.0.smithi077.stdout:8/61: getdents d4 0 2024-04-26T04:52:43.229 INFO:tasks.workunit.client.1.smithi078.stdout:1/74: write d7/d9/db/f14 [1841907,34718] 0 2024-04-26T04:52:43.230 INFO:tasks.workunit.client.1.smithi078.stdout:1/75: chown d7/ff 322486315 1 2024-04-26T04:52:43.230 INFO:tasks.workunit.client.0.smithi077.stdout:8/62: mkdir d4/d1a 0 2024-04-26T04:52:43.233 INFO:tasks.workunit.client.1.smithi078.stdout:1/76: unlink d7/d9/fe 0 2024-04-26T04:52:43.233 INFO:tasks.workunit.client.1.smithi078.stdout:1/77: stat f3 0 2024-04-26T04:52:43.237 INFO:tasks.workunit.client.0.smithi077.stdout:8/63: creat d4/d1a/f1b x:0 0 0 2024-04-26T04:52:43.238 INFO:tasks.workunit.client.0.smithi077.stdout:8/64: creat d4/d1a/f1c x:0 0 0 2024-04-26T04:52:43.239 INFO:tasks.workunit.client.0.smithi077.stdout:8/65: dread - d4/df/d18/f19 zero size 2024-04-26T04:52:43.240 INFO:tasks.workunit.client.1.smithi078.stdout:1/78: dwrite f5 [0,4194304] 0 2024-04-26T04:52:43.241 INFO:tasks.workunit.client.0.smithi077.stdout:8/66: symlink d4/df/d18/l1d 0 2024-04-26T04:52:43.241 INFO:tasks.workunit.client.0.smithi077.stdout:8/67: fdatasync d4/df/f15 0 2024-04-26T04:52:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:42 smithi077 ceph-mon[26311]: Adjusting osd_memory_target on smithi078 to 2581M 2024-04-26T04:52:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:42 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:52:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:42 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:52:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:42 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:52:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:42 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:52:43.252 INFO:tasks.workunit.client.1.smithi078.stdout:1/79: truncate f5 802053 0 2024-04-26T04:52:43.254 INFO:tasks.workunit.client.1.smithi078.stdout:1/80: symlink d7/d9/dd/l1a 0 2024-04-26T04:52:43.255 INFO:tasks.workunit.client.0.smithi077.stdout:8/68: dwrite f0 [0,4194304] 0 2024-04-26T04:52:43.260 INFO:tasks.workunit.client.0.smithi077.stdout:8/69: getdents d4/d1a 0 2024-04-26T04:52:43.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/70: truncate d4/df/f16 171988 0 2024-04-26T04:52:43.262 INFO:tasks.workunit.client.0.smithi077.stdout:8/71: truncate d4/fe 120051 0 2024-04-26T04:52:43.266 INFO:tasks.workunit.client.1.smithi078.stdout:1/81: dwrite d7/d9/f12 [0,4194304] 0 2024-04-26T04:52:43.269 INFO:tasks.workunit.client.0.smithi077.stdout:8/72: dwrite d4/df/d18/f19 [0,4194304] 0 2024-04-26T04:52:43.270 INFO:tasks.workunit.client.1.smithi078.stdout:1/82: creat d7/f1b x:0 0 0 2024-04-26T04:52:43.272 INFO:tasks.workunit.client.0.smithi077.stdout:8/73: creat d4/d1a/f1e x:0 0 0 2024-04-26T04:52:43.273 INFO:tasks.workunit.client.1.smithi078.stdout:1/83: truncate f3 771713 0 2024-04-26T04:52:43.276 INFO:tasks.workunit.client.0.smithi077.stdout:8/74: creat d4/df/f1f x:0 0 0 2024-04-26T04:52:43.277 INFO:tasks.workunit.client.1.smithi078.stdout:1/84: dread d7/d9/f12 [0,4194304] 0 2024-04-26T04:52:43.280 INFO:tasks.workunit.client.0.smithi077.stdout:8/75: mknod d4/c20 0 2024-04-26T04:52:43.281 INFO:tasks.workunit.client.1.smithi078.stdout:1/85: creat d7/f1c x:0 0 0 2024-04-26T04:52:43.282 INFO:tasks.workunit.client.0.smithi077.stdout:8/76: unlink d4/df/d18/l1d 0 2024-04-26T04:52:43.288 INFO:tasks.workunit.client.1.smithi078.stdout:1/86: dwrite d7/d9/f12 [4194304,4194304] 0 2024-04-26T04:52:43.288 INFO:tasks.workunit.client.0.smithi077.stdout:8/77: dwrite d4/df/f1f [0,4194304] 0 2024-04-26T04:52:43.289 INFO:tasks.workunit.client.0.smithi077.stdout:8/78: dread - d4/d1a/f1e zero size 2024-04-26T04:52:43.291 INFO:tasks.workunit.client.1.smithi078.stdout:1/87: creat d7/d9/d17/f1d x:0 0 0 2024-04-26T04:52:43.294 INFO:tasks.workunit.client.0.smithi077.stdout:8/79: dread d4/f5 [0,4194304] 0 2024-04-26T04:52:43.294 INFO:tasks.workunit.client.1.smithi078.stdout:1/88: mkdir d7/d1e 0 2024-04-26T04:52:43.297 INFO:tasks.workunit.client.0.smithi077.stdout:8/80: rename d4/l7 to d4/d1a/l21 0 2024-04-26T04:52:43.298 INFO:tasks.workunit.client.1.smithi078.stdout:1/89: mkdir d7/d9/d1f 0 2024-04-26T04:52:43.301 INFO:tasks.workunit.client.0.smithi077.stdout:8/81: dread d4/f5 [0,4194304] 0 2024-04-26T04:52:43.302 INFO:tasks.workunit.client.0.smithi077.stdout:8/82: stat d4/f5 0 2024-04-26T04:52:43.303 INFO:tasks.workunit.client.1.smithi078.stdout:1/90: dread d7/f11 [4194304,4194304] 0 2024-04-26T04:52:43.305 INFO:tasks.workunit.client.1.smithi078.stdout:1/91: mkdir d7/d9/dd/d20 0 2024-04-26T04:52:43.305 INFO:tasks.workunit.client.0.smithi077.stdout:8/83: dread d4/f9 [0,4194304] 0 2024-04-26T04:52:43.307 INFO:tasks.workunit.client.1.smithi078.stdout:1/92: symlink d7/d9/d17/l21 0 2024-04-26T04:52:43.308 INFO:tasks.workunit.client.1.smithi078.stdout:1/93: symlink d7/d9/d17/l22 0 2024-04-26T04:52:43.308 INFO:tasks.workunit.client.1.smithi078.stdout:1/94: chown d7/d1e 6 1 2024-04-26T04:52:43.309 INFO:tasks.workunit.client.1.smithi078.stdout:1/95: readlink d7/d9/dd/l1a 0 2024-04-26T04:52:43.310 INFO:tasks.workunit.client.1.smithi078.stdout:1/96: unlink d7/d9/c10 0 2024-04-26T04:52:43.311 INFO:tasks.workunit.client.1.smithi078.stdout:1/97: fdatasync d7/f19 0 2024-04-26T04:52:43.311 INFO:tasks.workunit.client.1.smithi078.stdout:1/98: dread - d7/f1b zero size 2024-04-26T04:52:43.317 INFO:tasks.workunit.client.1.smithi078.stdout:1/99: dwrite d7/d9/db/f15 [0,4194304] 0 2024-04-26T04:52:43.318 INFO:tasks.workunit.client.1.smithi078.stdout:1/100: write f1 [3358510,122999] 0 2024-04-26T04:52:43.319 INFO:tasks.workunit.client.1.smithi078.stdout:1/101: readlink d7/d9/dd/l1a 0 2024-04-26T04:52:43.320 INFO:tasks.workunit.client.1.smithi078.stdout:1/102: write d7/ff [3029312,82541] 0 2024-04-26T04:52:43.321 INFO:tasks.workunit.client.1.smithi078.stdout:1/103: chown d7/c18 5275 1 2024-04-26T04:52:43.325 INFO:tasks.workunit.client.1.smithi078.stdout:1/104: getdents d7/d9 0 2024-04-26T04:52:43.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/105: truncate d7/d9/f12 9039622 0 2024-04-26T04:52:43.327 INFO:tasks.workunit.client.1.smithi078.stdout:1/106: symlink d7/d9/d17/l23 0 2024-04-26T04:52:43.328 INFO:tasks.workunit.client.1.smithi078.stdout:1/107: mknod d7/d9/dd/c24 0 2024-04-26T04:52:43.331 INFO:tasks.workunit.client.1.smithi078.stdout:1/108: link d7/d9/dd/l1a d7/d9/d17/l25 0 2024-04-26T04:52:43.334 INFO:tasks.workunit.client.1.smithi078.stdout:1/109: link d7/d9/db/f15 d7/d9/d1f/f26 0 2024-04-26T04:52:43.338 INFO:tasks.workunit.client.1.smithi078.stdout:1/110: link d7/d9/d17/l22 d7/d9/d17/l27 0 2024-04-26T04:52:43.341 INFO:tasks.workunit.client.1.smithi078.stdout:1/111: creat d7/d1e/f28 x:0 0 0 2024-04-26T04:52:43.343 INFO:tasks.workunit.client.1.smithi078.stdout:1/112: dread f1 [0,4194304] 0 2024-04-26T04:52:43.346 INFO:tasks.workunit.client.1.smithi078.stdout:1/113: link d7/d9/d17/l22 d7/d9/dd/d20/l29 0 2024-04-26T04:52:43.347 INFO:tasks.workunit.client.1.smithi078.stdout:1/114: rmdir d7/d9/dd/d20 39 2024-04-26T04:52:43.349 INFO:tasks.workunit.client.1.smithi078.stdout:1/115: mkdir d7/d9/db/d2a 0 2024-04-26T04:52:43.356 INFO:tasks.workunit.client.1.smithi078.stdout:1/116: fsync d7/d9/f12 0 2024-04-26T04:52:43.357 INFO:tasks.workunit.client.1.smithi078.stdout:1/117: symlink d7/d9/db/d2a/l2b 0 2024-04-26T04:52:43.358 INFO:tasks.workunit.client.1.smithi078.stdout:1/118: dread - d7/f1c zero size 2024-04-26T04:52:43.364 INFO:tasks.workunit.client.1.smithi078.stdout:1/119: dwrite d7/f19 [0,4194304] 0 2024-04-26T04:52:43.372 INFO:tasks.workunit.client.1.smithi078.stdout:1/120: dwrite d7/d9/db/f15 [4194304,4194304] 0 2024-04-26T04:52:43.375 INFO:tasks.workunit.client.1.smithi078.stdout:1/121: creat d7/d9/f2c x:0 0 0 2024-04-26T04:52:43.385 INFO:tasks.workunit.client.1.smithi078.stdout:1/122: rename d7/f19 to d7/d9/dd/f2d 0 2024-04-26T04:52:43.386 INFO:tasks.workunit.client.1.smithi078.stdout:1/123: write d7/f1b [317777,105528] 0 2024-04-26T04:52:43.388 INFO:tasks.workunit.client.1.smithi078.stdout:1/124: unlink f5 0 2024-04-26T04:52:43.395 INFO:tasks.workunit.client.1.smithi078.stdout:1/125: dwrite d7/d9/f2c [0,4194304] 0 2024-04-26T04:52:43.399 INFO:tasks.workunit.client.1.smithi078.stdout:1/126: dread d7/f11 [0,4194304] 0 2024-04-26T04:52:43.401 INFO:tasks.workunit.client.1.smithi078.stdout:1/127: write d7/d9/d17/f1d [177257,71790] 0 2024-04-26T04:52:43.403 INFO:tasks.workunit.client.1.smithi078.stdout:1/128: stat d7/d9/d17/l22 0 2024-04-26T04:52:43.403 INFO:tasks.workunit.client.1.smithi078.stdout:1/129: truncate d7/d1e/f28 548187 0 2024-04-26T04:52:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: pgmap v11: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:43 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: pgmap v11: 65 pgs: 65 active+clean; 198 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:43 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:44.320 INFO:tasks.workunit.client.1.smithi078.stdout:7/80: sync 2024-04-26T04:52:44.320 INFO:tasks.workunit.client.1.smithi078.stdout:3/13: sync 2024-04-26T04:52:44.320 INFO:tasks.workunit.client.1.smithi078.stdout:1/130: sync 2024-04-26T04:52:44.320 INFO:tasks.workunit.client.1.smithi078.stdout:5/7: sync 2024-04-26T04:52:44.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/63: sync 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:2/13: sync 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:0/38: sync 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:0/39: readlink - no filename 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:8/84: sync 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:9/6: sync 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:9/7: read - no filename 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:9/8: rename - no filename 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:9/9: fdatasync - no filename 2024-04-26T04:52:44.321 INFO:tasks.workunit.client.1.smithi078.stdout:6/36: sync 2024-04-26T04:52:44.322 INFO:tasks.workunit.client.1.smithi078.stdout:9/10: creat f1 x:0 0 0 2024-04-26T04:52:44.322 INFO:tasks.workunit.client.1.smithi078.stdout:2/14: write f3 [204378,18607] 0 2024-04-26T04:52:44.323 INFO:tasks.workunit.client.1.smithi078.stdout:8/85: write d0/f18 [816035,27936] 0 2024-04-26T04:52:44.323 INFO:tasks.workunit.client.1.smithi078.stdout:5/8: fsync f3 0 2024-04-26T04:52:44.324 INFO:tasks.workunit.client.1.smithi078.stdout:3/14: rename l0 to l5 0 2024-04-26T04:52:44.324 INFO:tasks.workunit.client.1.smithi078.stdout:1/131: mkdir d7/d2e 0 2024-04-26T04:52:44.324 INFO:tasks.workunit.client.1.smithi078.stdout:5/9: chown f3 507114 1 2024-04-26T04:52:44.325 INFO:tasks.workunit.client.1.smithi078.stdout:1/132: chown d7/ff 98662 1 2024-04-26T04:52:44.325 INFO:tasks.workunit.client.1.smithi078.stdout:5/10: write f3 [746614,28885] 0 2024-04-26T04:52:44.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/133: chown d7/d9/d17/l27 145 1 2024-04-26T04:52:44.453 INFO:tasks.workunit.client.1.smithi078.stdout:0/40: mknod d2/d9/cc 0 2024-04-26T04:52:44.453 INFO:tasks.workunit.client.1.smithi078.stdout:4/64: creat d0/de/f1e x:0 0 0 2024-04-26T04:52:44.453 INFO:tasks.workunit.client.1.smithi078.stdout:0/41: write d2/f7 [79020,45189] 0 2024-04-26T04:52:44.454 INFO:tasks.workunit.client.1.smithi078.stdout:7/81: mkdir d9/da/d11/d13/d1a 0 2024-04-26T04:52:44.455 INFO:tasks.workunit.client.1.smithi078.stdout:9/11: creat f2 x:0 0 0 2024-04-26T04:52:44.455 INFO:tasks.workunit.client.1.smithi078.stdout:9/12: write f2 [213513,46266] 0 2024-04-26T04:52:44.456 INFO:tasks.workunit.client.1.smithi078.stdout:9/13: fdatasync f1 0 2024-04-26T04:52:44.456 INFO:tasks.workunit.client.1.smithi078.stdout:9/14: chown f2 209 1 2024-04-26T04:52:44.465 INFO:tasks.workunit.client.1.smithi078.stdout:3/15: rename f2 to f6 0 2024-04-26T04:52:44.465 INFO:tasks.workunit.client.1.smithi078.stdout:2/15: symlink l5 0 2024-04-26T04:52:44.466 INFO:tasks.workunit.client.1.smithi078.stdout:8/86: chown d0/d7/c14 405494102 1 2024-04-26T04:52:44.466 INFO:tasks.workunit.client.1.smithi078.stdout:8/87: chown d0/f1a 5071 1 2024-04-26T04:52:44.467 INFO:tasks.workunit.client.1.smithi078.stdout:8/88: truncate d0/f5 5542177 0 2024-04-26T04:52:44.467 INFO:tasks.workunit.client.1.smithi078.stdout:6/37: rename d2/d3/d5/da/cb to d2/d3/d5/da/cc 0 2024-04-26T04:52:44.468 INFO:tasks.workunit.client.1.smithi078.stdout:5/11: mkdir d4 0 2024-04-26T04:52:44.468 INFO:tasks.workunit.client.1.smithi078.stdout:1/134: creat d7/d1e/f2f x:0 0 0 2024-04-26T04:52:44.571 INFO:tasks.workunit.client.1.smithi078.stdout:4/65: symlink d0/d10/l1f 0 2024-04-26T04:52:44.571 INFO:tasks.workunit.client.1.smithi078.stdout:0/42: chown d2/f4 330944 1 2024-04-26T04:52:44.571 INFO:tasks.workunit.client.1.smithi078.stdout:0/43: stat d2/f4 0 2024-04-26T04:52:44.571 INFO:tasks.workunit.client.1.smithi078.stdout:4/66: chown d0/f8 32887219 1 2024-04-26T04:52:44.572 INFO:tasks.workunit.client.1.smithi078.stdout:7/82: creat d9/f1b x:0 0 0 2024-04-26T04:52:44.572 INFO:tasks.workunit.client.1.smithi078.stdout:9/15: creat f3 x:0 0 0 2024-04-26T04:52:44.572 INFO:tasks.workunit.client.1.smithi078.stdout:3/16: creat f7 x:0 0 0 2024-04-26T04:52:44.574 INFO:tasks.workunit.client.1.smithi078.stdout:3/17: read - f3 zero size 2024-04-26T04:52:44.575 INFO:tasks.workunit.client.1.smithi078.stdout:3/18: write f7 [888090,66745] 0 2024-04-26T04:52:44.576 INFO:tasks.workunit.client.1.smithi078.stdout:2/16: link f3 f6 0 2024-04-26T04:52:44.580 INFO:tasks.workunit.client.1.smithi078.stdout:2/17: dread f6 [0,4194304] 0 2024-04-26T04:52:44.582 INFO:tasks.workunit.client.1.smithi078.stdout:9/16: dwrite f2 [0,4194304] 0 2024-04-26T04:52:44.582 INFO:tasks.workunit.client.1.smithi078.stdout:3/19: rename l1 to l8 0 2024-04-26T04:52:44.583 INFO:tasks.workunit.client.1.smithi078.stdout:0/44: creat d2/d9/da/fd x:0 0 0 2024-04-26T04:52:44.586 INFO:tasks.workunit.client.1.smithi078.stdout:5/12: rmdir d4 0 2024-04-26T04:52:44.587 INFO:tasks.workunit.client.1.smithi078.stdout:9/17: dread f2 [0,4194304] 0 2024-04-26T04:52:44.587 INFO:tasks.workunit.client.1.smithi078.stdout:8/89: getdents d0 0 2024-04-26T04:52:44.588 INFO:tasks.workunit.client.1.smithi078.stdout:9/18: dread - f1 zero size 2024-04-26T04:52:44.588 INFO:tasks.workunit.client.1.smithi078.stdout:2/18: write f6 [963211,53033] 0 2024-04-26T04:52:44.589 INFO:tasks.workunit.client.1.smithi078.stdout:3/20: creat f9 x:0 0 0 2024-04-26T04:52:44.590 INFO:tasks.workunit.client.1.smithi078.stdout:3/21: dread - f9 zero size 2024-04-26T04:52:44.596 INFO:tasks.workunit.client.1.smithi078.stdout:9/19: rename f3 to f4 0 2024-04-26T04:52:44.598 INFO:tasks.workunit.client.1.smithi078.stdout:8/90: write d0/f18 [257999,90682] 0 2024-04-26T04:52:44.598 INFO:tasks.workunit.client.1.smithi078.stdout:2/19: dwrite f3 [0,4194304] 0 2024-04-26T04:52:44.598 INFO:tasks.workunit.client.1.smithi078.stdout:3/22: unlink f3 0 2024-04-26T04:52:44.601 INFO:tasks.workunit.client.1.smithi078.stdout:9/20: creat f5 x:0 0 0 2024-04-26T04:52:44.603 INFO:tasks.workunit.client.1.smithi078.stdout:2/20: creat f7 x:0 0 0 2024-04-26T04:52:44.604 INFO:tasks.workunit.client.1.smithi078.stdout:3/23: mknod ca 0 2024-04-26T04:52:44.604 INFO:tasks.workunit.client.1.smithi078.stdout:3/24: rmdir - no directory 2024-04-26T04:52:44.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/21: write f3 [4095108,34652] 0 2024-04-26T04:52:44.605 INFO:tasks.workunit.client.1.smithi078.stdout:3/25: write f7 [1187498,67477] 0 2024-04-26T04:52:44.606 INFO:tasks.workunit.client.1.smithi078.stdout:9/21: write f2 [3137032,9416] 0 2024-04-26T04:52:44.607 INFO:tasks.workunit.client.1.smithi078.stdout:9/22: dread - f1 zero size 2024-04-26T04:52:44.608 INFO:tasks.workunit.client.1.smithi078.stdout:8/91: getdents d0/d19 0 2024-04-26T04:52:44.609 INFO:tasks.workunit.client.1.smithi078.stdout:3/26: mknod cb 0 2024-04-26T04:52:44.611 INFO:tasks.workunit.client.1.smithi078.stdout:3/27: chown l4 967010423 1 2024-04-26T04:52:44.613 INFO:tasks.workunit.client.1.smithi078.stdout:8/92: getdents d0 0 2024-04-26T04:52:44.619 INFO:tasks.workunit.client.1.smithi078.stdout:9/23: dwrite f5 [0,4194304] 0 2024-04-26T04:52:44.625 INFO:tasks.workunit.client.1.smithi078.stdout:3/28: dwrite f6 [0,4194304] 0 2024-04-26T04:52:44.627 INFO:tasks.workunit.client.1.smithi078.stdout:8/93: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:44.628 INFO:tasks.workunit.client.1.smithi078.stdout:3/29: creat fc x:0 0 0 2024-04-26T04:52:44.630 INFO:tasks.workunit.client.1.smithi078.stdout:3/30: creat fd x:0 0 0 2024-04-26T04:52:44.632 INFO:tasks.workunit.client.1.smithi078.stdout:9/24: dwrite f5 [0,4194304] 0 2024-04-26T04:52:44.632 INFO:tasks.workunit.client.1.smithi078.stdout:8/94: creat d0/d19/f1c x:0 0 0 2024-04-26T04:52:44.633 INFO:tasks.workunit.client.1.smithi078.stdout:9/25: dread - f1 zero size 2024-04-26T04:52:44.634 INFO:tasks.workunit.client.1.smithi078.stdout:3/31: creat fe x:0 0 0 2024-04-26T04:52:44.634 INFO:tasks.workunit.client.1.smithi078.stdout:3/32: stat l4 0 2024-04-26T04:52:44.636 INFO:tasks.workunit.client.1.smithi078.stdout:8/95: mknod d0/d3/c1d 0 2024-04-26T04:52:44.639 INFO:tasks.workunit.client.1.smithi078.stdout:8/96: fdatasync d0/f5 0 2024-04-26T04:52:44.641 INFO:tasks.workunit.client.1.smithi078.stdout:9/26: dwrite f2 [0,4194304] 0 2024-04-26T04:52:44.646 INFO:tasks.workunit.client.1.smithi078.stdout:8/97: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:44.646 INFO:tasks.workunit.client.1.smithi078.stdout:9/27: dread f5 [0,4194304] 0 2024-04-26T04:52:44.650 INFO:tasks.workunit.client.1.smithi078.stdout:8/98: dread d0/f1a [0,4194304] 0 2024-04-26T04:52:44.653 INFO:tasks.workunit.client.1.smithi078.stdout:8/99: symlink d0/d3/l1e 0 2024-04-26T04:52:44.653 INFO:tasks.workunit.client.1.smithi078.stdout:9/28: dwrite f4 [0,4194304] 0 2024-04-26T04:52:44.654 INFO:tasks.workunit.client.1.smithi078.stdout:9/29: write f4 [3154410,14904] 0 2024-04-26T04:52:44.655 INFO:tasks.workunit.client.1.smithi078.stdout:8/100: chown d0/d3/l6 116024 1 2024-04-26T04:52:44.655 INFO:tasks.workunit.client.1.smithi078.stdout:9/30: write f4 [792742,76677] 0 2024-04-26T04:52:44.656 INFO:tasks.workunit.client.1.smithi078.stdout:9/31: chown f4 10 1 2024-04-26T04:52:44.657 INFO:tasks.workunit.client.1.smithi078.stdout:8/101: symlink d0/l1f 0 2024-04-26T04:52:44.659 INFO:tasks.workunit.client.1.smithi078.stdout:8/102: symlink d0/l20 0 2024-04-26T04:52:44.662 INFO:tasks.workunit.client.1.smithi078.stdout:9/32: dwrite f4 [0,4194304] 0 2024-04-26T04:52:44.663 INFO:tasks.workunit.client.1.smithi078.stdout:8/103: rename d0/d7 to d0/d21 0 2024-04-26T04:52:44.664 INFO:tasks.workunit.client.1.smithi078.stdout:9/33: write f1 [912012,105866] 0 2024-04-26T04:52:44.665 INFO:tasks.workunit.client.1.smithi078.stdout:4/67: rename d0/d10 to d0/dd/d20 0 2024-04-26T04:52:44.667 INFO:tasks.workunit.client.1.smithi078.stdout:9/34: fdatasync f5 0 2024-04-26T04:52:44.667 INFO:tasks.workunit.client.1.smithi078.stdout:6/38: write f1 [5092304,24184] 0 2024-04-26T04:52:44.669 INFO:tasks.workunit.client.1.smithi078.stdout:8/104: dread d0/f1a [0,4194304] 0 2024-04-26T04:52:44.671 INFO:tasks.workunit.client.1.smithi078.stdout:9/35: creat f6 x:0 0 0 2024-04-26T04:52:44.672 INFO:tasks.workunit.client.1.smithi078.stdout:9/36: fsync f2 0 2024-04-26T04:52:44.673 INFO:tasks.workunit.client.1.smithi078.stdout:6/39: dread f1 [0,4194304] 0 2024-04-26T04:52:44.675 INFO:tasks.workunit.client.1.smithi078.stdout:5/13: getdents . 0 2024-04-26T04:52:44.677 INFO:tasks.workunit.client.1.smithi078.stdout:6/40: creat d2/d3/fd x:0 0 0 2024-04-26T04:52:44.678 INFO:tasks.workunit.client.1.smithi078.stdout:7/83: dwrite d9/da/f15 [0,4194304] 0 2024-04-26T04:52:44.679 INFO:tasks.workunit.client.1.smithi078.stdout:7/84: chown d9/da/f15 131372 1 2024-04-26T04:52:44.679 INFO:tasks.workunit.client.1.smithi078.stdout:6/41: write d2/d3/f6 [3445542,64879] 0 2024-04-26T04:52:44.679 INFO:tasks.workunit.client.1.smithi078.stdout:5/14: creat f5 x:0 0 0 2024-04-26T04:52:44.680 INFO:tasks.workunit.client.1.smithi078.stdout:0/45: write d2/f8 [1053636,129750] 0 2024-04-26T04:52:44.680 INFO:tasks.workunit.client.1.smithi078.stdout:1/135: truncate d7/d9/db/f15 6500191 0 2024-04-26T04:52:44.681 INFO:tasks.workunit.client.1.smithi078.stdout:6/42: readlink d2/d3/l7 0 2024-04-26T04:52:44.682 INFO:tasks.workunit.client.1.smithi078.stdout:8/105: getdents d0/d21 0 2024-04-26T04:52:44.684 INFO:tasks.workunit.client.1.smithi078.stdout:1/136: truncate d7/d1e/f2f 1017207 0 2024-04-26T04:52:44.685 INFO:tasks.workunit.client.1.smithi078.stdout:6/43: mkdir d2/de 0 2024-04-26T04:52:44.686 INFO:tasks.workunit.client.1.smithi078.stdout:6/44: dread - d2/d3/fd zero size 2024-04-26T04:52:44.686 INFO:tasks.workunit.client.1.smithi078.stdout:6/45: chown d2/de 214 1 2024-04-26T04:52:44.687 INFO:tasks.workunit.client.1.smithi078.stdout:0/46: mknod d2/d9/da/ce 0 2024-04-26T04:52:44.688 INFO:tasks.workunit.client.1.smithi078.stdout:8/106: symlink d0/d19/l22 0 2024-04-26T04:52:44.689 INFO:tasks.workunit.client.1.smithi078.stdout:8/107: chown d0/d19/l22 308 1 2024-04-26T04:52:44.690 INFO:tasks.workunit.client.1.smithi078.stdout:6/46: mknod d2/cf 0 2024-04-26T04:52:44.691 INFO:tasks.workunit.client.1.smithi078.stdout:7/85: creat d9/da/d11/d13/d1a/f1c x:0 0 0 2024-04-26T04:52:44.692 INFO:tasks.workunit.client.1.smithi078.stdout:1/137: mknod d7/d2e/c30 0 2024-04-26T04:52:44.693 INFO:tasks.workunit.client.1.smithi078.stdout:0/47: dwrite d2/f5 [0,4194304] 0 2024-04-26T04:52:44.693 INFO:tasks.workunit.client.1.smithi078.stdout:8/108: creat d0/f23 x:0 0 0 2024-04-26T04:52:44.695 INFO:tasks.workunit.client.1.smithi078.stdout:3/33: fsync fc 0 2024-04-26T04:52:44.696 INFO:tasks.workunit.client.1.smithi078.stdout:6/47: chown d2/d3/d5/da/cc 137761049 1 2024-04-26T04:52:44.698 INFO:tasks.workunit.client.1.smithi078.stdout:1/138: rename d7/d9/db/f14 to d7/d1e/f31 0 2024-04-26T04:52:44.701 INFO:tasks.workunit.client.1.smithi078.stdout:6/48: creat d2/f10 x:0 0 0 2024-04-26T04:52:44.703 INFO:tasks.workunit.client.1.smithi078.stdout:8/109: truncate d0/f5 1645260 0 2024-04-26T04:52:44.704 INFO:tasks.workunit.client.1.smithi078.stdout:8/110: chown d0/d19 2864 1 2024-04-26T04:52:44.704 INFO:tasks.workunit.client.1.smithi078.stdout:3/34: dwrite f9 [0,4194304] 0 2024-04-26T04:52:44.706 INFO:tasks.workunit.client.1.smithi078.stdout:3/35: read - fe zero size 2024-04-26T04:52:44.707 INFO:tasks.workunit.client.1.smithi078.stdout:7/86: getdents d9/da 0 2024-04-26T04:52:44.707 INFO:tasks.workunit.client.1.smithi078.stdout:6/49: write f1 [1294169,82660] 0 2024-04-26T04:52:44.708 INFO:tasks.workunit.client.1.smithi078.stdout:1/139: dread d7/d9/f2c [0,4194304] 0 2024-04-26T04:52:44.708 INFO:tasks.workunit.client.1.smithi078.stdout:8/111: creat d0/d21/f24 x:0 0 0 2024-04-26T04:52:44.710 INFO:tasks.workunit.client.1.smithi078.stdout:6/50: chown d2/d3/fd 1360 1 2024-04-26T04:52:44.711 INFO:tasks.workunit.client.1.smithi078.stdout:3/36: creat ff x:0 0 0 2024-04-26T04:52:44.711 INFO:tasks.workunit.client.1.smithi078.stdout:4/68: truncate d0/fc 3852845 0 2024-04-26T04:52:44.711 INFO:tasks.workunit.client.1.smithi078.stdout:3/37: truncate fc 7592 0 2024-04-26T04:52:44.712 INFO:tasks.workunit.client.1.smithi078.stdout:1/140: mknod d7/d2e/c32 0 2024-04-26T04:52:44.712 INFO:tasks.workunit.client.1.smithi078.stdout:7/87: truncate d9/da/d11/f12 4734698 0 2024-04-26T04:52:44.713 INFO:tasks.workunit.client.1.smithi078.stdout:8/112: mkdir d0/d19/d25 0 2024-04-26T04:52:44.713 INFO:tasks.workunit.client.1.smithi078.stdout:7/88: readlink d9/lf 0 2024-04-26T04:52:44.714 INFO:tasks.workunit.client.1.smithi078.stdout:9/37: truncate f2 1929092 0 2024-04-26T04:52:44.714 INFO:tasks.workunit.client.1.smithi078.stdout:1/141: read d7/d9/f2c [3986116,37693] 0 2024-04-26T04:52:44.715 INFO:tasks.workunit.client.1.smithi078.stdout:7/89: dread - d9/da/d11/d13/d1a/f1c zero size 2024-04-26T04:52:44.715 INFO:tasks.workunit.client.1.smithi078.stdout:5/15: fsync f5 0 2024-04-26T04:52:44.716 INFO:tasks.workunit.client.1.smithi078.stdout:4/69: rename d0/c9 to d0/d1d/c21 0 2024-04-26T04:52:44.717 INFO:tasks.workunit.client.1.smithi078.stdout:7/90: fdatasync d9/da/f15 0 2024-04-26T04:52:44.719 INFO:tasks.workunit.client.1.smithi078.stdout:0/48: rmdir d2/d9 39 2024-04-26T04:52:44.720 INFO:tasks.workunit.client.1.smithi078.stdout:1/142: creat d7/d2e/f33 x:0 0 0 2024-04-26T04:52:44.721 INFO:tasks.workunit.client.1.smithi078.stdout:5/16: mknod c6 0 2024-04-26T04:52:44.722 INFO:tasks.workunit.client.1.smithi078.stdout:1/143: chown d7/d9/d17/f1d 106 1 2024-04-26T04:52:44.723 INFO:tasks.workunit.client.1.smithi078.stdout:4/70: mkdir d0/de/d22 0 2024-04-26T04:52:44.724 INFO:tasks.workunit.client.1.smithi078.stdout:8/113: dwrite d0/d19/f1c [0,4194304] 0 2024-04-26T04:52:44.724 INFO:tasks.workunit.client.1.smithi078.stdout:8/114: readlink d0/d21/la 0 2024-04-26T04:52:44.724 INFO:tasks.workunit.client.1.smithi078.stdout:7/91: symlink d9/d16/l1d 0 2024-04-26T04:52:44.731 INFO:tasks.workunit.client.1.smithi078.stdout:1/144: unlink d7/d1e/f28 0 2024-04-26T04:52:44.731 INFO:tasks.workunit.client.1.smithi078.stdout:8/115: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:52:44.733 INFO:tasks.workunit.client.1.smithi078.stdout:7/92: rmdir d9/da/d11/d13 39 2024-04-26T04:52:44.737 INFO:tasks.workunit.client.1.smithi078.stdout:3/38: getdents . 0 2024-04-26T04:52:44.741 INFO:tasks.workunit.client.1.smithi078.stdout:0/49: creat d2/ff x:0 0 0 2024-04-26T04:52:44.742 INFO:tasks.workunit.client.1.smithi078.stdout:9/38: rename f2 to f7 0 2024-04-26T04:52:44.742 INFO:tasks.workunit.client.1.smithi078.stdout:9/39: fsync f1 0 2024-04-26T04:52:44.743 INFO:tasks.workunit.client.1.smithi078.stdout:2/22: sync 2024-04-26T04:52:44.743 INFO:tasks.workunit.client.1.smithi078.stdout:6/51: truncate f1 4441160 0 2024-04-26T04:52:44.743 INFO:tasks.workunit.client.1.smithi078.stdout:2/23: fsync f3 0 2024-04-26T04:52:44.744 INFO:tasks.workunit.client.1.smithi078.stdout:2/24: readlink l2 0 2024-04-26T04:52:44.744 INFO:tasks.workunit.client.1.smithi078.stdout:2/25: stat l4 0 2024-04-26T04:52:44.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/26: dread - f7 zero size 2024-04-26T04:52:44.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/27: stat l2 0 2024-04-26T04:52:44.745 INFO:tasks.workunit.client.1.smithi078.stdout:3/39: rename l4 to l10 0 2024-04-26T04:52:44.747 INFO:tasks.workunit.client.1.smithi078.stdout:1/145: unlink d7/d9/f2c 0 2024-04-26T04:52:44.747 INFO:tasks.workunit.client.1.smithi078.stdout:8/116: stat d0/f5 0 2024-04-26T04:52:44.748 INFO:tasks.workunit.client.1.smithi078.stdout:8/117: chown d0/f23 234555 1 2024-04-26T04:52:44.749 INFO:tasks.workunit.client.1.smithi078.stdout:8/118: chown d0/cc 8951557 1 2024-04-26T04:52:44.751 INFO:tasks.workunit.client.1.smithi078.stdout:5/17: getdents . 0 2024-04-26T04:52:44.752 INFO:tasks.workunit.client.1.smithi078.stdout:5/18: dread - f5 zero size 2024-04-26T04:52:44.752 INFO:tasks.workunit.client.1.smithi078.stdout:5/19: rmdir - no directory 2024-04-26T04:52:44.752 INFO:tasks.workunit.client.1.smithi078.stdout:9/40: creat f8 x:0 0 0 2024-04-26T04:52:44.752 INFO:tasks.workunit.client.1.smithi078.stdout:9/41: readlink - no filename 2024-04-26T04:52:44.753 INFO:tasks.workunit.client.1.smithi078.stdout:5/20: read f3 [284986,100650] 0 2024-04-26T04:52:44.753 INFO:tasks.workunit.client.1.smithi078.stdout:1/146: dwrite d7/f1b [0,4194304] 0 2024-04-26T04:52:44.754 INFO:tasks.workunit.client.1.smithi078.stdout:6/52: write d2/d3/d5/f8 [1287286,93756] 0 2024-04-26T04:52:44.756 INFO:tasks.workunit.client.1.smithi078.stdout:7/93: creat d9/da/d11/d13/d1a/f1e x:0 0 0 2024-04-26T04:52:44.756 INFO:tasks.workunit.client.1.smithi078.stdout:2/28: mkdir d8 0 2024-04-26T04:52:44.757 INFO:tasks.workunit.client.1.smithi078.stdout:2/29: write f3 [3190250,66038] 0 2024-04-26T04:52:44.762 INFO:tasks.workunit.client.1.smithi078.stdout:1/147: dread d7/f11 [4194304,4194304] 0 2024-04-26T04:52:44.762 INFO:tasks.workunit.client.1.smithi078.stdout:8/119: creat d0/d3/f26 x:0 0 0 2024-04-26T04:52:44.763 INFO:tasks.workunit.client.1.smithi078.stdout:9/42: creat f9 x:0 0 0 2024-04-26T04:52:44.764 INFO:tasks.workunit.client.1.smithi078.stdout:9/43: readlink - no filename 2024-04-26T04:52:44.764 INFO:tasks.workunit.client.1.smithi078.stdout:5/21: mkdir d7 0 2024-04-26T04:52:44.764 INFO:tasks.workunit.client.1.smithi078.stdout:6/53: mkdir d2/d3/d5/d11 0 2024-04-26T04:52:44.766 INFO:tasks.workunit.client.1.smithi078.stdout:5/22: write f5 [993580,44654] 0 2024-04-26T04:52:44.767 INFO:tasks.workunit.client.1.smithi078.stdout:0/50: creat d2/f10 x:0 0 0 2024-04-26T04:52:44.768 INFO:tasks.workunit.client.1.smithi078.stdout:7/94: symlink d9/da/d11/l1f 0 2024-04-26T04:52:44.768 INFO:tasks.workunit.client.1.smithi078.stdout:8/120: symlink d0/d19/d25/l27 0 2024-04-26T04:52:44.824 INFO:tasks.workunit.client.1.smithi078.stdout:1/148: creat d7/d9/db/f34 x:0 0 0 2024-04-26T04:52:44.825 INFO:tasks.workunit.client.1.smithi078.stdout:2/30: dread f6 [0,4194304] 0 2024-04-26T04:52:44.826 INFO:tasks.workunit.client.1.smithi078.stdout:3/40: dwrite fd [0,4194304] 0 2024-04-26T04:52:44.832 INFO:tasks.workunit.client.1.smithi078.stdout:8/121: rename d0/d3/l6 to d0/l28 0 2024-04-26T04:52:44.834 INFO:tasks.workunit.client.1.smithi078.stdout:5/23: creat d7/f8 x:0 0 0 2024-04-26T04:52:44.834 INFO:tasks.workunit.client.1.smithi078.stdout:1/149: readlink d7/d9/d17/l21 0 2024-04-26T04:52:44.834 INFO:tasks.workunit.client.1.smithi078.stdout:6/54: dwrite d2/f10 [0,4194304] 0 2024-04-26T04:52:44.836 INFO:tasks.workunit.client.1.smithi078.stdout:0/51: dread d2/f3 [0,4194304] 0 2024-04-26T04:52:44.836 INFO:tasks.workunit.client.1.smithi078.stdout:2/31: rename f7 to d8/f9 0 2024-04-26T04:52:44.836 INFO:tasks.workunit.client.1.smithi078.stdout:7/95: dread f5 [4194304,4194304] 0 2024-04-26T04:52:44.837 INFO:tasks.workunit.client.1.smithi078.stdout:1/150: chown d7/ff 965298843 1 2024-04-26T04:52:44.837 INFO:tasks.workunit.client.1.smithi078.stdout:6/55: write d2/f10 [1571687,26745] 0 2024-04-26T04:52:44.837 INFO:tasks.workunit.client.1.smithi078.stdout:2/32: read f3 [2633687,49024] 0 2024-04-26T04:52:44.837 INFO:tasks.workunit.client.1.smithi078.stdout:3/41: mknod c11 0 2024-04-26T04:52:44.839 INFO:tasks.workunit.client.1.smithi078.stdout:8/122: rmdir d0/d19 39 2024-04-26T04:52:44.841 INFO:tasks.workunit.client.1.smithi078.stdout:5/24: fsync f3 0 2024-04-26T04:52:44.845 INFO:tasks.workunit.client.1.smithi078.stdout:3/42: write fc [171583,82324] 0 2024-04-26T04:52:44.845 INFO:tasks.workunit.client.1.smithi078.stdout:1/151: mkdir d7/d1e/d35 0 2024-04-26T04:52:44.846 INFO:tasks.workunit.client.1.smithi078.stdout:2/33: fsync d8/f9 0 2024-04-26T04:52:44.847 INFO:tasks.workunit.client.1.smithi078.stdout:8/123: creat d0/d21/f29 x:0 0 0 2024-04-26T04:52:44.848 INFO:tasks.workunit.client.1.smithi078.stdout:0/52: write f1 [3827307,110607] 0 2024-04-26T04:52:44.848 INFO:tasks.workunit.client.1.smithi078.stdout:2/34: dread - d8/f9 zero size 2024-04-26T04:52:44.850 INFO:tasks.workunit.client.1.smithi078.stdout:6/56: dwrite d2/d3/fd [0,4194304] 0 2024-04-26T04:52:44.851 INFO:tasks.workunit.client.1.smithi078.stdout:3/43: dread - fe zero size 2024-04-26T04:52:44.852 INFO:tasks.workunit.client.1.smithi078.stdout:1/152: creat d7/d1e/f36 x:0 0 0 2024-04-26T04:52:44.852 INFO:tasks.workunit.client.1.smithi078.stdout:7/96: getdents d9/da 0 2024-04-26T04:52:44.854 INFO:tasks.workunit.client.1.smithi078.stdout:0/53: truncate d2/f7 4529229 0 2024-04-26T04:52:44.854 INFO:tasks.workunit.client.1.smithi078.stdout:8/124: write d0/f10 [1964321,52229] 0 2024-04-26T04:52:44.855 INFO:tasks.workunit.client.1.smithi078.stdout:1/153: chown d7/d1e/f2f 183286975 1 2024-04-26T04:52:44.859 INFO:tasks.workunit.client.1.smithi078.stdout:7/97: write d9/fe [1682141,59884] 0 2024-04-26T04:52:44.861 INFO:tasks.workunit.client.1.smithi078.stdout:6/57: write d2/f10 [2250860,45296] 0 2024-04-26T04:52:44.869 INFO:tasks.workunit.client.1.smithi078.stdout:5/25: dwrite f3 [0,4194304] 0 2024-04-26T04:52:44.870 INFO:tasks.workunit.client.1.smithi078.stdout:6/58: write d2/d3/fd [1484446,43305] 0 2024-04-26T04:52:44.872 INFO:tasks.workunit.client.1.smithi078.stdout:8/125: creat d0/d19/f2a x:0 0 0 2024-04-26T04:52:44.874 INFO:tasks.workunit.client.1.smithi078.stdout:4/71: sync 2024-04-26T04:52:44.878 INFO:tasks.workunit.client.1.smithi078.stdout:6/59: fdatasync d2/d3/f6 0 2024-04-26T04:52:44.878 INFO:tasks.workunit.client.1.smithi078.stdout:8/126: readlink d0/l28 0 2024-04-26T04:52:44.879 INFO:tasks.workunit.client.1.smithi078.stdout:2/35: dwrite f6 [0,4194304] 0 2024-04-26T04:52:44.880 INFO:tasks.workunit.client.1.smithi078.stdout:1/154: write f3 [70496,109292] 0 2024-04-26T04:52:44.881 INFO:tasks.workunit.client.1.smithi078.stdout:0/54: dwrite d2/f4 [0,4194304] 0 2024-04-26T04:52:44.881 INFO:tasks.workunit.client.1.smithi078.stdout:3/44: dwrite f9 [0,4194304] 0 2024-04-26T04:52:44.886 INFO:tasks.workunit.client.1.smithi078.stdout:2/36: write d8/f9 [1024302,90310] 0 2024-04-26T04:52:44.886 INFO:tasks.workunit.client.1.smithi078.stdout:3/45: read fd [2356466,27709] 0 2024-04-26T04:52:44.889 INFO:tasks.workunit.client.1.smithi078.stdout:3/46: chown fc 864909611 1 2024-04-26T04:52:44.890 INFO:tasks.workunit.client.1.smithi078.stdout:3/47: stat f9 0 2024-04-26T04:52:44.890 INFO:tasks.workunit.client.1.smithi078.stdout:7/98: dread f4 [0,4194304] 0 2024-04-26T04:52:44.890 INFO:tasks.workunit.client.1.smithi078.stdout:3/48: fsync fd 0 2024-04-26T04:52:44.891 INFO:tasks.workunit.client.1.smithi078.stdout:7/99: dread - d9/f10 zero size 2024-04-26T04:52:44.892 INFO:tasks.workunit.client.1.smithi078.stdout:7/100: fdatasync d9/fe 0 2024-04-26T04:52:44.892 INFO:tasks.workunit.client.1.smithi078.stdout:7/101: fsync f4 0 2024-04-26T04:52:44.893 INFO:tasks.workunit.client.1.smithi078.stdout:8/127: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:52:44.956 INFO:tasks.workunit.client.1.smithi078.stdout:9/44: sync 2024-04-26T04:52:44.977 INFO:tasks.workunit.client.1.smithi078.stdout:1/155: mkdir d7/d9/db/d37 0 2024-04-26T04:52:44.982 INFO:tasks.workunit.client.1.smithi078.stdout:2/37: fsync f6 0 2024-04-26T04:52:44.982 INFO:tasks.workunit.client.1.smithi078.stdout:1/156: dwrite d7/f11 [0,4194304] 0 2024-04-26T04:52:44.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/102: mkdir d9/da/d20 0 2024-04-26T04:52:44.985 INFO:tasks.workunit.client.1.smithi078.stdout:8/128: creat d0/d19/d25/f2b x:0 0 0 2024-04-26T04:52:44.986 INFO:tasks.workunit.client.1.smithi078.stdout:9/45: creat fa x:0 0 0 2024-04-26T04:52:44.987 INFO:tasks.workunit.client.1.smithi078.stdout:9/46: read - f8 zero size 2024-04-26T04:52:44.988 INFO:tasks.workunit.client.1.smithi078.stdout:9/47: fsync f9 0 2024-04-26T04:52:44.988 INFO:tasks.workunit.client.1.smithi078.stdout:6/60: creat d2/de/f12 x:0 0 0 2024-04-26T04:52:44.990 INFO:tasks.workunit.client.1.smithi078.stdout:9/48: fdatasync f4 0 2024-04-26T04:52:44.992 INFO:tasks.workunit.client.1.smithi078.stdout:2/38: dwrite d8/f9 [0,4194304] 0 2024-04-26T04:52:44.994 INFO:tasks.workunit.client.1.smithi078.stdout:0/55: rename d2/d9/cc to d2/d9/c11 0 2024-04-26T04:52:44.994 INFO:tasks.workunit.client.1.smithi078.stdout:6/61: truncate d2/de/f12 402415 0 2024-04-26T04:52:44.997 INFO:tasks.workunit.client.1.smithi078.stdout:5/26: sync 2024-04-26T04:52:44.999 INFO:tasks.workunit.client.1.smithi078.stdout:8/129: dwrite d0/f18 [0,4194304] 0 2024-04-26T04:52:45.002 INFO:tasks.workunit.client.1.smithi078.stdout:0/56: truncate d2/f7 5326540 0 2024-04-26T04:52:45.002 INFO:tasks.workunit.client.1.smithi078.stdout:8/130: fsync d0/d21/f24 0 2024-04-26T04:52:45.003 INFO:tasks.workunit.client.1.smithi078.stdout:4/72: getdents d0/d1d 0 2024-04-26T04:52:45.006 INFO:tasks.workunit.client.1.smithi078.stdout:3/49: creat f12 x:0 0 0 2024-04-26T04:52:45.007 INFO:tasks.workunit.client.1.smithi078.stdout:0/57: stat d2/f5 0 2024-04-26T04:52:45.008 INFO:tasks.workunit.client.1.smithi078.stdout:7/103: creat d9/da/d20/f21 x:0 0 0 2024-04-26T04:52:45.009 INFO:tasks.workunit.client.1.smithi078.stdout:8/131: chown d0/cc 482 1 2024-04-26T04:52:45.013 INFO:tasks.workunit.client.1.smithi078.stdout:9/49: dwrite f1 [0,4194304] 0 2024-04-26T04:52:45.015 INFO:tasks.workunit.client.1.smithi078.stdout:2/39: dwrite f3 [0,4194304] 0 2024-04-26T04:52:45.019 INFO:tasks.workunit.client.1.smithi078.stdout:6/62: dwrite d2/de/f12 [0,4194304] 0 2024-04-26T04:52:45.020 INFO:tasks.workunit.client.1.smithi078.stdout:5/27: dread f3 [0,4194304] 0 2024-04-26T04:52:45.020 INFO:tasks.workunit.client.1.smithi078.stdout:3/50: write fd [4760736,17565] 0 2024-04-26T04:52:45.020 INFO:tasks.workunit.client.1.smithi078.stdout:4/73: creat d0/dd/d20/f23 x:0 0 0 2024-04-26T04:52:45.021 INFO:tasks.workunit.client.1.smithi078.stdout:1/157: link d7/d2e/c32 d7/c38 0 2024-04-26T04:52:45.022 INFO:tasks.workunit.client.1.smithi078.stdout:6/63: fsync d2/d3/fd 0 2024-04-26T04:52:45.023 INFO:tasks.workunit.client.1.smithi078.stdout:1/158: readlink d7/d9/db/d2a/l2b 0 2024-04-26T04:52:45.030 INFO:tasks.workunit.client.1.smithi078.stdout:5/28: dread f3 [0,4194304] 0 2024-04-26T04:52:45.032 INFO:tasks.workunit.client.1.smithi078.stdout:2/40: creat d8/fa x:0 0 0 2024-04-26T04:52:45.032 INFO:tasks.workunit.client.1.smithi078.stdout:6/64: dwrite d2/f10 [0,4194304] 0 2024-04-26T04:52:45.040 INFO:tasks.workunit.client.1.smithi078.stdout:1/159: rename d7/d9 to d7/d39 0 2024-04-26T04:52:45.048 INFO:tasks.workunit.client.1.smithi078.stdout:2/41: rename f6 to d8/fb 0 2024-04-26T04:52:45.052 INFO:tasks.workunit.client.1.smithi078.stdout:6/65: rename d2/d3/l7 to d2/d3/l13 0 2024-04-26T04:52:45.052 INFO:tasks.workunit.client.1.smithi078.stdout:6/66: chown d2/d3/d5/d11 2 1 2024-04-26T04:52:45.053 INFO:tasks.workunit.client.1.smithi078.stdout:6/67: chown d2/l4 9477 1 2024-04-26T04:52:45.054 INFO:tasks.workunit.client.1.smithi078.stdout:1/160: rmdir d7/d39/d17 39 2024-04-26T04:52:45.063 INFO:tasks.workunit.client.1.smithi078.stdout:2/42: symlink d8/lc 0 2024-04-26T04:52:45.066 INFO:tasks.workunit.client.1.smithi078.stdout:3/51: rename fe to f13 0 2024-04-26T04:52:45.069 INFO:tasks.workunit.client.1.smithi078.stdout:6/68: rmdir d2/d3/d5 39 2024-04-26T04:52:45.070 INFO:tasks.workunit.client.1.smithi078.stdout:4/74: rename d0/dd to d0/d24 0 2024-04-26T04:52:45.073 INFO:tasks.workunit.client.1.smithi078.stdout:8/132: link d0/d3/le d0/d3/l2c 0 2024-04-26T04:52:45.075 INFO:tasks.workunit.client.1.smithi078.stdout:7/104: write d9/f14 [2957961,100515] 0 2024-04-26T04:52:45.077 INFO:tasks.workunit.client.1.smithi078.stdout:7/105: write d9/f1b [105738,73254] 0 2024-04-26T04:52:45.077 INFO:tasks.workunit.client.1.smithi078.stdout:4/75: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:52:45.078 INFO:tasks.workunit.client.1.smithi078.stdout:2/43: mknod d8/cd 0 2024-04-26T04:52:45.079 INFO:tasks.workunit.client.1.smithi078.stdout:4/76: write d0/d24/d20/f1b [443251,44824] 0 2024-04-26T04:52:45.082 INFO:tasks.workunit.client.1.smithi078.stdout:2/44: dread f3 [0,4194304] 0 2024-04-26T04:52:45.087 INFO:tasks.workunit.client.1.smithi078.stdout:3/52: dwrite f13 [0,4194304] 0 2024-04-26T04:52:45.088 INFO:tasks.workunit.client.1.smithi078.stdout:1/161: mknod d7/d1e/d35/c3a 0 2024-04-26T04:52:45.090 INFO:tasks.workunit.client.1.smithi078.stdout:1/162: truncate d7/d39/db/f34 848109 0 2024-04-26T04:52:45.090 INFO:tasks.workunit.client.1.smithi078.stdout:5/29: getdents d7 0 2024-04-26T04:52:45.091 INFO:tasks.workunit.client.1.smithi078.stdout:6/69: dwrite d2/d3/d5/f8 [0,4194304] 0 2024-04-26T04:52:45.091 INFO:tasks.workunit.client.1.smithi078.stdout:5/30: write f5 [369840,15595] 0 2024-04-26T04:52:45.092 INFO:tasks.workunit.client.1.smithi078.stdout:7/106: creat d9/d16/f22 x:0 0 0 2024-04-26T04:52:45.094 INFO:tasks.workunit.client.1.smithi078.stdout:7/107: write f7 [953661,35869] 0 2024-04-26T04:52:45.095 INFO:tasks.workunit.client.1.smithi078.stdout:5/31: chown d7/f8 114223959 1 2024-04-26T04:52:45.098 INFO:tasks.workunit.client.1.smithi078.stdout:2/45: symlink d8/le 0 2024-04-26T04:52:45.101 INFO:tasks.workunit.client.1.smithi078.stdout:2/46: truncate d8/fa 341408 0 2024-04-26T04:52:45.102 INFO:tasks.workunit.client.1.smithi078.stdout:1/163: dwrite d7/f1c [0,4194304] 0 2024-04-26T04:52:45.104 INFO:tasks.workunit.client.1.smithi078.stdout:6/70: dwrite d2/d3/f6 [0,4194304] 0 2024-04-26T04:52:45.105 INFO:tasks.workunit.client.1.smithi078.stdout:6/71: chown d2/cf 6737 1 2024-04-26T04:52:45.106 INFO:tasks.workunit.client.1.smithi078.stdout:8/133: mknod d0/c2d 0 2024-04-26T04:52:45.112 INFO:tasks.workunit.client.1.smithi078.stdout:4/77: rename d0/d24 to d0/d25 0 2024-04-26T04:52:45.113 INFO:tasks.workunit.client.1.smithi078.stdout:4/78: dread - d0/de/f1c zero size 2024-04-26T04:52:45.119 INFO:tasks.workunit.client.1.smithi078.stdout:4/79: dwrite d0/d25/d20/f1b [0,4194304] 0 2024-04-26T04:52:45.122 INFO:tasks.workunit.client.1.smithi078.stdout:2/47: creat d8/ff x:0 0 0 2024-04-26T04:52:45.123 INFO:tasks.workunit.client.1.smithi078.stdout:0/58: sync 2024-04-26T04:52:45.126 INFO:tasks.workunit.client.1.smithi078.stdout:2/48: dread f3 [0,4194304] 0 2024-04-26T04:52:45.126 INFO:tasks.workunit.client.1.smithi078.stdout:0/59: dread d2/f6 [0,4194304] 0 2024-04-26T04:52:45.128 INFO:tasks.workunit.client.1.smithi078.stdout:6/72: creat d2/f14 x:0 0 0 2024-04-26T04:52:45.133 INFO:tasks.workunit.client.1.smithi078.stdout:0/60: dwrite d2/f4 [0,4194304] 0 2024-04-26T04:52:45.135 INFO:tasks.workunit.client.1.smithi078.stdout:7/108: link d9/f10 d9/da/f23 0 2024-04-26T04:52:45.141 INFO:tasks.workunit.client.1.smithi078.stdout:1/164: creat d7/d39/dd/d20/f3b x:0 0 0 2024-04-26T04:52:45.143 INFO:tasks.workunit.client.1.smithi078.stdout:3/53: creat f14 x:0 0 0 2024-04-26T04:52:45.145 INFO:tasks.workunit.client.1.smithi078.stdout:2/49: write f3 [1582265,36407] 0 2024-04-26T04:52:45.145 INFO:tasks.workunit.client.1.smithi078.stdout:2/50: read d8/f9 [1127237,74727] 0 2024-04-26T04:52:45.148 INFO:tasks.workunit.client.1.smithi078.stdout:5/32: link c6 d7/c9 0 2024-04-26T04:52:45.152 INFO:tasks.workunit.client.1.smithi078.stdout:2/51: dwrite d8/fb [4194304,4194304] 0 2024-04-26T04:52:45.153 INFO:tasks.workunit.client.1.smithi078.stdout:2/52: dread d8/fa [0,4194304] 0 2024-04-26T04:52:45.153 INFO:tasks.workunit.client.1.smithi078.stdout:4/80: mkdir d0/de/d22/d26 0 2024-04-26T04:52:45.155 INFO:tasks.workunit.client.1.smithi078.stdout:2/53: write d8/fb [4650084,46659] 0 2024-04-26T04:52:45.156 INFO:tasks.workunit.client.1.smithi078.stdout:2/54: stat d8/le 0 2024-04-26T04:52:45.156 INFO:tasks.workunit.client.1.smithi078.stdout:1/165: chown d7/d39/d17/l23 11441 1 2024-04-26T04:52:45.157 INFO:tasks.workunit.client.1.smithi078.stdout:2/55: write d8/ff [443022,65999] 0 2024-04-26T04:52:45.161 INFO:tasks.workunit.client.1.smithi078.stdout:6/73: link d2/d3/d5/f8 d2/d3/d5/d11/f15 0 2024-04-26T04:52:45.162 INFO:tasks.workunit.client.1.smithi078.stdout:5/33: mkdir d7/da 0 2024-04-26T04:52:45.162 INFO:tasks.workunit.client.1.smithi078.stdout:5/34: chown l2 3579 1 2024-04-26T04:52:45.166 INFO:tasks.workunit.client.1.smithi078.stdout:4/81: mkdir d0/de/d22/d27 0 2024-04-26T04:52:45.168 INFO:tasks.workunit.client.1.smithi078.stdout:3/54: mknod c15 0 2024-04-26T04:52:45.168 INFO:tasks.workunit.client.1.smithi078.stdout:4/82: dread d0/de/f1a [0,4194304] 0 2024-04-26T04:52:45.168 INFO:tasks.workunit.client.1.smithi078.stdout:9/50: sync 2024-04-26T04:52:45.169 INFO:tasks.workunit.client.1.smithi078.stdout:1/166: creat d7/d39/d17/f3c x:0 0 0 2024-04-26T04:52:45.169 INFO:tasks.workunit.client.1.smithi078.stdout:1/167: stat d7/d39/d17/l22 0 2024-04-26T04:52:45.170 INFO:tasks.workunit.client.1.smithi078.stdout:1/168: read f3 [588513,112807] 0 2024-04-26T04:52:45.172 INFO:tasks.workunit.client.1.smithi078.stdout:8/134: getdents d0/d19/d25 0 2024-04-26T04:52:45.173 INFO:tasks.workunit.client.1.smithi078.stdout:8/135: write d0/d19/d25/f2b [388197,104425] 0 2024-04-26T04:52:45.173 INFO:tasks.workunit.client.1.smithi078.stdout:5/35: fsync f3 0 2024-04-26T04:52:45.174 INFO:tasks.workunit.client.1.smithi078.stdout:8/136: write d0/d19/f2a [350981,19183] 0 2024-04-26T04:52:45.175 INFO:tasks.workunit.client.1.smithi078.stdout:0/61: creat d2/f12 x:0 0 0 2024-04-26T04:52:45.176 INFO:tasks.workunit.client.1.smithi078.stdout:0/62: read d2/f3 [4060753,57419] 0 2024-04-26T04:52:45.176 INFO:tasks.workunit.client.1.smithi078.stdout:7/109: link d9/da/fb d9/da/d20/f24 0 2024-04-26T04:52:45.182 INFO:tasks.workunit.client.1.smithi078.stdout:9/51: mknod cb 0 2024-04-26T04:52:45.190 INFO:tasks.workunit.client.1.smithi078.stdout:8/137: symlink d0/d19/d25/l2e 0 2024-04-26T04:52:45.191 INFO:tasks.workunit.client.1.smithi078.stdout:0/63: creat d2/d9/da/f13 x:0 0 0 2024-04-26T04:52:45.194 INFO:tasks.workunit.client.1.smithi078.stdout:3/55: mkdir d16 0 2024-04-26T04:52:45.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/52: unlink f6 0 2024-04-26T04:52:45.196 INFO:tasks.workunit.client.1.smithi078.stdout:4/83: symlink d0/de/d22/d26/l28 0 2024-04-26T04:52:45.198 INFO:tasks.workunit.client.1.smithi078.stdout:2/56: link d8/lc d8/l10 0 2024-04-26T04:52:45.199 INFO:tasks.workunit.client.1.smithi078.stdout:5/36: rename c6 to d7/cb 0 2024-04-26T04:52:45.200 INFO:tasks.workunit.client.1.smithi078.stdout:5/37: write f5 [1622143,7929] 0 2024-04-26T04:52:45.202 INFO:tasks.workunit.client.1.smithi078.stdout:7/110: unlink d9/da/fb 0 2024-04-26T04:52:45.206 INFO:tasks.workunit.client.1.smithi078.stdout:9/53: unlink f4 0 2024-04-26T04:52:45.206 INFO:tasks.workunit.client.1.smithi078.stdout:5/38: dwrite d7/f8 [0,4194304] 0 2024-04-26T04:52:45.208 INFO:tasks.workunit.client.1.smithi078.stdout:2/57: mkdir d8/d11 0 2024-04-26T04:52:45.209 INFO:tasks.workunit.client.1.smithi078.stdout:9/54: dread f1 [0,4194304] 0 2024-04-26T04:52:45.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/169: rename d7/d39/db/d2a to d7/d39/d1f/d3d 0 2024-04-26T04:52:45.217 INFO:tasks.workunit.client.1.smithi078.stdout:1/170: dwrite d7/f11 [4194304,4194304] 0 2024-04-26T04:52:45.218 INFO:tasks.workunit.client.1.smithi078.stdout:4/84: creat d0/de/d22/d27/f29 x:0 0 0 2024-04-26T04:52:45.220 INFO:tasks.workunit.client.1.smithi078.stdout:5/39: mknod d7/cc 0 2024-04-26T04:52:45.221 INFO:tasks.workunit.client.1.smithi078.stdout:5/40: chown d7/f8 24 1 2024-04-26T04:52:45.223 INFO:tasks.workunit.client.1.smithi078.stdout:1/171: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:45.223 INFO:tasks.workunit.client.1.smithi078.stdout:9/55: mkdir dc 0 2024-04-26T04:52:45.228 INFO:tasks.workunit.client.1.smithi078.stdout:8/138: creat d0/f2f x:0 0 0 2024-04-26T04:52:45.230 INFO:tasks.workunit.client.1.smithi078.stdout:0/64: creat d2/f14 x:0 0 0 2024-04-26T04:52:45.230 INFO:tasks.workunit.client.1.smithi078.stdout:0/65: stat d2/f6 0 2024-04-26T04:52:45.232 INFO:tasks.workunit.client.1.smithi078.stdout:6/74: sync 2024-04-26T04:52:45.234 INFO:tasks.workunit.client.1.smithi078.stdout:6/75: chown d2/d3/d5 11035571 1 2024-04-26T04:52:45.235 INFO:tasks.workunit.client.1.smithi078.stdout:6/76: fdatasync d2/f14 0 2024-04-26T04:52:45.237 INFO:tasks.workunit.client.1.smithi078.stdout:0/66: dwrite d2/d9/da/f13 [0,4194304] 0 2024-04-26T04:52:45.237 INFO:tasks.workunit.client.1.smithi078.stdout:0/67: dread - d2/f12 zero size 2024-04-26T04:52:45.238 INFO:tasks.workunit.client.1.smithi078.stdout:4/85: rmdir d0/de/d22 39 2024-04-26T04:52:45.240 INFO:tasks.workunit.client.1.smithi078.stdout:2/58: mkdir d8/d11/d12 0 2024-04-26T04:52:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:44 smithi077 ceph-mon[26311]: Reconfiguring node-exporter.smithi077 (dependencies changed)... 2024-04-26T04:52:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:44 smithi077 ceph-mon[26311]: Reconfiguring daemon node-exporter.smithi077 on smithi077 2024-04-26T04:52:45.253 INFO:tasks.workunit.client.1.smithi078.stdout:6/77: creat d2/f16 x:0 0 0 2024-04-26T04:52:45.256 INFO:tasks.workunit.client.1.smithi078.stdout:0/68: unlink d2/f5 0 2024-04-26T04:52:45.261 INFO:tasks.workunit.client.1.smithi078.stdout:9/56: truncate f7 2267413 0 2024-04-26T04:52:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:44 smithi078 ceph-mon[33746]: Reconfiguring node-exporter.smithi077 (dependencies changed)... 2024-04-26T04:52:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:44 smithi078 ceph-mon[33746]: Reconfiguring daemon node-exporter.smithi077 on smithi077 2024-04-26T04:52:45.276 INFO:tasks.workunit.client.1.smithi078.stdout:7/111: write d9/da/f23 [639503,71309] 0 2024-04-26T04:52:45.277 INFO:tasks.workunit.client.1.smithi078.stdout:7/112: fsync d9/da/d11/f12 0 2024-04-26T04:52:45.278 INFO:tasks.workunit.client.1.smithi078.stdout:8/139: rename d0/l1f to d0/l30 0 2024-04-26T04:52:45.278 INFO:tasks.workunit.client.1.smithi078.stdout:8/140: chown d0/d19/d25/f2b 0 1 2024-04-26T04:52:45.281 INFO:tasks.workunit.client.1.smithi078.stdout:4/86: link d0/de/f1a d0/d25/d20/f2a 0 2024-04-26T04:52:45.283 INFO:tasks.workunit.client.1.smithi078.stdout:2/59: rmdir d8/d11/d12 0 2024-04-26T04:52:45.285 INFO:tasks.workunit.client.1.smithi078.stdout:5/41: getdents d7 0 2024-04-26T04:52:45.286 INFO:tasks.workunit.client.1.smithi078.stdout:9/57: mkdir dc/dd 0 2024-04-26T04:52:45.290 INFO:tasks.workunit.client.1.smithi078.stdout:8/141: mkdir d0/d19/d25/d31 0 2024-04-26T04:52:45.292 INFO:tasks.workunit.client.1.smithi078.stdout:4/87: creat d0/d25/d20/f2b x:0 0 0 2024-04-26T04:52:45.293 INFO:tasks.workunit.client.1.smithi078.stdout:4/88: read - d0/d25/d20/f23 zero size 2024-04-26T04:52:45.294 INFO:tasks.workunit.client.1.smithi078.stdout:3/56: truncate f6 490711 0 2024-04-26T04:52:45.294 INFO:tasks.workunit.client.1.smithi078.stdout:4/89: chown d0/f8 27 1 2024-04-26T04:52:45.295 INFO:tasks.workunit.client.1.smithi078.stdout:5/42: symlink d7/ld 0 2024-04-26T04:52:45.295 INFO:tasks.workunit.client.1.smithi078.stdout:4/90: fsync d0/f5 0 2024-04-26T04:52:45.296 INFO:tasks.workunit.client.1.smithi078.stdout:5/43: fdatasync f3 0 2024-04-26T04:52:45.296 INFO:tasks.workunit.client.1.smithi078.stdout:8/142: dwrite d0/d19/d25/f2b [0,4194304] 0 2024-04-26T04:52:45.296 INFO:tasks.workunit.client.1.smithi078.stdout:9/58: rmdir dc 39 2024-04-26T04:52:45.307 INFO:tasks.workunit.client.1.smithi078.stdout:3/57: creat d16/f17 x:0 0 0 2024-04-26T04:52:45.307 INFO:tasks.workunit.client.1.smithi078.stdout:3/58: rename d16 to d16/d18 22 2024-04-26T04:52:45.328 INFO:tasks.workunit.client.1.smithi078.stdout:0/69: rmdir d2 39 2024-04-26T04:52:45.328 INFO:tasks.workunit.client.1.smithi078.stdout:8/143: dwrite d0/f1a [4194304,4194304] 0 2024-04-26T04:52:45.329 INFO:tasks.workunit.client.1.smithi078.stdout:1/172: write d7/d39/d1f/f26 [5856520,38595] 0 2024-04-26T04:52:45.330 INFO:tasks.workunit.client.1.smithi078.stdout:2/60: getdents d8/d11 0 2024-04-26T04:52:45.331 INFO:tasks.workunit.client.1.smithi078.stdout:1/173: write d7/d39/d17/f3c [553886,43107] 0 2024-04-26T04:52:45.332 INFO:tasks.workunit.client.1.smithi078.stdout:2/61: write d8/ff [748345,20198] 0 2024-04-26T04:52:45.336 INFO:tasks.workunit.client.1.smithi078.stdout:2/62: dread f3 [4194304,4194304] 0 2024-04-26T04:52:45.339 INFO:tasks.workunit.client.1.smithi078.stdout:2/63: dread d8/ff [0,4194304] 0 2024-04-26T04:52:45.343 INFO:tasks.workunit.client.1.smithi078.stdout:5/44: getdents d7 0 2024-04-26T04:52:45.347 INFO:tasks.workunit.client.1.smithi078.stdout:5/45: dwrite f3 [0,4194304] 0 2024-04-26T04:52:45.348 INFO:tasks.workunit.client.1.smithi078.stdout:5/46: write f5 [181495,97058] 0 2024-04-26T04:52:45.354 INFO:tasks.workunit.client.1.smithi078.stdout:6/78: sync 2024-04-26T04:52:45.356 INFO:tasks.workunit.client.1.smithi078.stdout:6/79: chown d2/de 29135 1 2024-04-26T04:52:45.357 INFO:tasks.workunit.client.1.smithi078.stdout:5/47: dwrite f3 [0,4194304] 0 2024-04-26T04:52:45.358 INFO:tasks.workunit.client.1.smithi078.stdout:5/48: stat d7/f8 0 2024-04-26T04:52:45.359 INFO:tasks.workunit.client.1.smithi078.stdout:1/174: dwrite d7/fc [0,4194304] 0 2024-04-26T04:52:45.363 INFO:tasks.workunit.client.1.smithi078.stdout:3/59: getdents d16 0 2024-04-26T04:52:45.367 INFO:tasks.workunit.client.1.smithi078.stdout:6/80: dread d2/d3/fd [0,4194304] 0 2024-04-26T04:52:45.368 INFO:tasks.workunit.client.1.smithi078.stdout:2/64: getdents d8/d11 0 2024-04-26T04:52:45.369 INFO:tasks.workunit.client.1.smithi078.stdout:6/81: chown d2/d3/d5/d11 459325 1 2024-04-26T04:52:45.369 INFO:tasks.workunit.client.1.smithi078.stdout:5/49: dwrite d7/f8 [4194304,4194304] 0 2024-04-26T04:52:45.369 INFO:tasks.workunit.client.1.smithi078.stdout:2/65: chown d8/fb 53 1 2024-04-26T04:52:45.370 INFO:tasks.workunit.client.1.smithi078.stdout:6/82: write d2/de/f12 [26245,89452] 0 2024-04-26T04:52:45.370 INFO:tasks.workunit.client.1.smithi078.stdout:2/66: chown l2 35003 1 2024-04-26T04:52:45.371 INFO:tasks.workunit.client.1.smithi078.stdout:7/113: creat d9/da/d11/d13/f25 x:0 0 0 2024-04-26T04:52:45.378 INFO:tasks.workunit.client.1.smithi078.stdout:3/60: mknod d16/c19 0 2024-04-26T04:52:45.382 INFO:tasks.workunit.client.1.smithi078.stdout:5/50: mkdir d7/de 0 2024-04-26T04:52:45.384 INFO:tasks.workunit.client.1.smithi078.stdout:2/67: mknod d8/c13 0 2024-04-26T04:52:45.386 INFO:tasks.workunit.client.1.smithi078.stdout:6/83: creat d2/d3/d5/da/f17 x:0 0 0 2024-04-26T04:52:45.387 INFO:tasks.workunit.client.1.smithi078.stdout:6/84: chown d2/d3/d5/da 47345 1 2024-04-26T04:52:45.388 INFO:tasks.workunit.client.1.smithi078.stdout:7/114: creat d9/d16/f26 x:0 0 0 2024-04-26T04:52:45.389 INFO:tasks.workunit.client.1.smithi078.stdout:8/144: getdents d0/d19 0 2024-04-26T04:52:45.389 INFO:tasks.workunit.client.1.smithi078.stdout:8/145: chown d0/f5 89791033 1 2024-04-26T04:52:45.390 INFO:tasks.workunit.client.1.smithi078.stdout:3/61: rmdir d16 39 2024-04-26T04:52:45.391 INFO:tasks.workunit.client.1.smithi078.stdout:8/146: write d0/f2f [959780,95001] 0 2024-04-26T04:52:45.391 INFO:tasks.workunit.client.1.smithi078.stdout:0/70: write d2/f10 [186700,56102] 0 2024-04-26T04:52:45.396 INFO:tasks.workunit.client.1.smithi078.stdout:8/147: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:45.401 INFO:tasks.workunit.client.1.smithi078.stdout:1/175: rename d7/f1b to d7/d39/f3e 0 2024-04-26T04:52:45.441 INFO:tasks.workunit.client.1.smithi078.stdout:5/51: unlink d7/cb 0 2024-04-26T04:52:45.441 INFO:tasks.workunit.client.1.smithi078.stdout:5/52: readlink l2 0 2024-04-26T04:52:45.450 INFO:tasks.workunit.client.1.smithi078.stdout:4/91: sync 2024-04-26T04:52:45.454 INFO:tasks.workunit.client.1.smithi078.stdout:3/62: unlink ca 0 2024-04-26T04:52:45.456 INFO:tasks.workunit.client.1.smithi078.stdout:1/176: dwrite d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:45.475 INFO:tasks.workunit.client.1.smithi078.stdout:9/59: sync 2024-04-26T04:52:45.475 INFO:tasks.workunit.client.1.smithi078.stdout:7/115: sync 2024-04-26T04:52:45.476 INFO:tasks.workunit.client.1.smithi078.stdout:9/60: write f9 [26579,2498] 0 2024-04-26T04:52:45.481 INFO:tasks.workunit.client.1.smithi078.stdout:8/148: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:52:45.483 INFO:tasks.workunit.client.1.smithi078.stdout:6/85: dwrite f1 [0,4194304] 0 2024-04-26T04:52:45.483 INFO:tasks.workunit.client.1.smithi078.stdout:8/149: write d0/f1a [1965771,54808] 0 2024-04-26T04:52:45.574 INFO:tasks.workunit.client.1.smithi078.stdout:1/177: rmdir d7 39 2024-04-26T04:52:45.574 INFO:tasks.workunit.client.1.smithi078.stdout:5/53: symlink d7/da/lf 0 2024-04-26T04:52:45.581 INFO:tasks.workunit.client.1.smithi078.stdout:8/150: mkdir d0/d19/d25/d32 0 2024-04-26T04:52:45.581 INFO:tasks.workunit.client.1.smithi078.stdout:2/68: getdents d8 0 2024-04-26T04:52:45.582 INFO:tasks.workunit.client.1.smithi078.stdout:4/92: mknod d0/de/c2c 0 2024-04-26T04:52:45.583 INFO:tasks.workunit.client.1.smithi078.stdout:4/93: fdatasync d0/de/f1c 0 2024-04-26T04:52:45.585 INFO:tasks.workunit.client.1.smithi078.stdout:7/116: truncate f4 6695387 0 2024-04-26T04:52:45.587 INFO:tasks.workunit.client.1.smithi078.stdout:2/69: mkdir d8/d11/d14 0 2024-04-26T04:52:45.589 INFO:tasks.workunit.client.1.smithi078.stdout:4/94: mknod d0/c2d 0 2024-04-26T04:52:45.590 INFO:tasks.workunit.client.1.smithi078.stdout:1/178: chown d7/d39/f12 2 1 2024-04-26T04:52:45.591 INFO:tasks.workunit.client.1.smithi078.stdout:0/71: rename d2/d9/fb to d2/f15 0 2024-04-26T04:52:45.592 INFO:tasks.workunit.client.1.smithi078.stdout:9/61: link f8 dc/dd/fe 0 2024-04-26T04:52:45.593 INFO:tasks.workunit.client.1.smithi078.stdout:9/62: chown dc/dd/fe 0 1 2024-04-26T04:52:45.597 INFO:tasks.workunit.client.1.smithi078.stdout:4/95: dread d0/d25/d20/f2a [0,4194304] 0 2024-04-26T04:52:45.597 INFO:tasks.workunit.client.1.smithi078.stdout:1/179: write d7/d39/f3e [5040129,76471] 0 2024-04-26T04:52:45.598 INFO:tasks.workunit.client.1.smithi078.stdout:0/72: creat d2/d9/f16 x:0 0 0 2024-04-26T04:52:45.598 INFO:tasks.workunit.client.1.smithi078.stdout:0/73: read f1 [728924,36245] 0 2024-04-26T04:52:45.601 INFO:tasks.workunit.client.1.smithi078.stdout:6/86: getdents d2/d3/d5 0 2024-04-26T04:52:45.602 INFO:tasks.workunit.client.1.smithi078.stdout:8/151: creat d0/f33 x:0 0 0 2024-04-26T04:52:45.602 INFO:tasks.workunit.client.1.smithi078.stdout:3/63: getdents d16 0 2024-04-26T04:52:45.604 INFO:tasks.workunit.client.1.smithi078.stdout:1/180: mknod d7/d39/d1f/d3d/c3f 0 2024-04-26T04:52:45.605 INFO:tasks.workunit.client.1.smithi078.stdout:0/74: creat d2/d9/f17 x:0 0 0 2024-04-26T04:52:45.606 INFO:tasks.workunit.client.1.smithi078.stdout:7/117: symlink d9/da/d11/d13/d1a/l27 0 2024-04-26T04:52:45.608 INFO:tasks.workunit.client.1.smithi078.stdout:3/64: creat d16/f1a x:0 0 0 2024-04-26T04:52:45.609 INFO:tasks.workunit.client.1.smithi078.stdout:3/65: dread - d16/f17 zero size 2024-04-26T04:52:45.609 INFO:tasks.workunit.client.1.smithi078.stdout:4/96: mkdir d0/de/d22/d26/d2e 0 2024-04-26T04:52:45.610 INFO:tasks.workunit.client.1.smithi078.stdout:1/181: creat d7/d39/dd/d20/f40 x:0 0 0 2024-04-26T04:52:45.611 INFO:tasks.workunit.client.1.smithi078.stdout:0/75: fdatasync f1 0 2024-04-26T04:52:45.611 INFO:tasks.workunit.client.1.smithi078.stdout:8/152: mknod d0/d19/d25/d32/c34 0 2024-04-26T04:52:45.613 INFO:tasks.workunit.client.1.smithi078.stdout:3/66: chown f7 358370848 1 2024-04-26T04:52:45.613 INFO:tasks.workunit.client.1.smithi078.stdout:4/97: mkdir d0/d2f 0 2024-04-26T04:52:45.615 INFO:tasks.workunit.client.1.smithi078.stdout:8/153: mkdir d0/d21/d35 0 2024-04-26T04:52:45.616 INFO:tasks.workunit.client.1.smithi078.stdout:3/67: creat d16/f1b x:0 0 0 2024-04-26T04:52:45.617 INFO:tasks.workunit.client.1.smithi078.stdout:1/182: dwrite d7/f13 [0,4194304] 0 2024-04-26T04:52:45.618 INFO:tasks.workunit.client.1.smithi078.stdout:3/68: fdatasync d16/f1b 0 2024-04-26T04:52:45.618 INFO:tasks.workunit.client.1.smithi078.stdout:4/98: symlink d0/de/d22/d26/d2e/l30 0 2024-04-26T04:52:45.619 INFO:tasks.workunit.client.1.smithi078.stdout:8/154: mkdir d0/d19/d25/d36 0 2024-04-26T04:52:45.620 INFO:tasks.workunit.client.1.smithi078.stdout:4/99: write d0/de/ff [26064,84226] 0 2024-04-26T04:52:45.622 INFO:tasks.workunit.client.1.smithi078.stdout:3/69: rename d16/f1b to d16/f1c 0 2024-04-26T04:52:45.623 INFO:tasks.workunit.client.1.smithi078.stdout:0/76: dread d2/f3 [0,4194304] 0 2024-04-26T04:52:45.624 INFO:tasks.workunit.client.1.smithi078.stdout:3/70: mknod d16/c1d 0 2024-04-26T04:52:45.625 INFO:tasks.workunit.client.1.smithi078.stdout:3/71: fdatasync f13 0 2024-04-26T04:52:45.625 INFO:tasks.workunit.client.1.smithi078.stdout:4/100: mkdir d0/d25/d20/d31 0 2024-04-26T04:52:45.626 INFO:tasks.workunit.client.1.smithi078.stdout:1/183: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:45.627 INFO:tasks.workunit.client.1.smithi078.stdout:0/77: stat d2/f6 0 2024-04-26T04:52:45.627 INFO:tasks.workunit.client.1.smithi078.stdout:0/78: chown d2/f15 1059433602 1 2024-04-26T04:52:45.627 INFO:tasks.workunit.client.1.smithi078.stdout:0/79: readlink - no filename 2024-04-26T04:52:45.628 INFO:tasks.workunit.client.1.smithi078.stdout:0/80: stat d2/d9/da 0 2024-04-26T04:52:45.628 INFO:tasks.workunit.client.1.smithi078.stdout:0/81: readlink - no filename 2024-04-26T04:52:45.628 INFO:tasks.workunit.client.1.smithi078.stdout:0/82: chown d2 365982 1 2024-04-26T04:52:45.629 INFO:tasks.workunit.client.1.smithi078.stdout:3/72: rename d16/f1a to d16/f1e 0 2024-04-26T04:52:45.630 INFO:tasks.workunit.client.1.smithi078.stdout:8/155: getdents d0/d19 0 2024-04-26T04:52:45.631 INFO:tasks.workunit.client.1.smithi078.stdout:4/101: creat d0/de/d22/f32 x:0 0 0 2024-04-26T04:52:45.632 INFO:tasks.workunit.client.1.smithi078.stdout:0/83: fdatasync d2/f12 0 2024-04-26T04:52:45.633 INFO:tasks.workunit.client.1.smithi078.stdout:3/73: rename d16/c19 to d16/c1f 0 2024-04-26T04:52:45.634 INFO:tasks.workunit.client.1.smithi078.stdout:1/184: dwrite d7/d39/dd/d20/f3b [0,4194304] 0 2024-04-26T04:52:45.635 INFO:tasks.workunit.client.1.smithi078.stdout:4/102: mkdir d0/d25/d20/d31/d33 0 2024-04-26T04:52:45.636 INFO:tasks.workunit.client.1.smithi078.stdout:3/74: creat d16/f20 x:0 0 0 2024-04-26T04:52:45.638 INFO:tasks.workunit.client.1.smithi078.stdout:0/84: rename d2/d9/da to d2/d9/d18 0 2024-04-26T04:52:45.639 INFO:tasks.workunit.client.1.smithi078.stdout:0/85: write d2/f10 [922039,20133] 0 2024-04-26T04:52:45.639 INFO:tasks.workunit.client.1.smithi078.stdout:3/75: creat d16/f21 x:0 0 0 2024-04-26T04:52:45.640 INFO:tasks.workunit.client.1.smithi078.stdout:3/76: dread - d16/f1e zero size 2024-04-26T04:52:45.642 INFO:tasks.workunit.client.1.smithi078.stdout:4/103: creat d0/de/f34 x:0 0 0 2024-04-26T04:52:45.643 INFO:tasks.workunit.client.1.smithi078.stdout:4/104: stat d0/d25/d20 0 2024-04-26T04:52:45.644 INFO:tasks.workunit.client.1.smithi078.stdout:3/77: rename f14 to d16/f22 0 2024-04-26T04:52:45.646 INFO:tasks.workunit.client.1.smithi078.stdout:3/78: rename c15 to d16/c23 0 2024-04-26T04:52:45.649 INFO:tasks.workunit.client.1.smithi078.stdout:4/105: link d0/f6 d0/de/d22/d26/d2e/f35 0 2024-04-26T04:52:45.650 INFO:tasks.workunit.client.1.smithi078.stdout:3/79: symlink d16/l24 0 2024-04-26T04:52:45.651 INFO:tasks.workunit.client.1.smithi078.stdout:4/106: creat d0/d1d/f36 x:0 0 0 2024-04-26T04:52:45.651 INFO:tasks.workunit.client.1.smithi078.stdout:4/107: fsync d0/d25/f19 0 2024-04-26T04:52:45.652 INFO:tasks.workunit.client.1.smithi078.stdout:4/108: chown d0/l11 48243 1 2024-04-26T04:52:45.655 INFO:tasks.workunit.client.1.smithi078.stdout:4/109: rename d0/de/d22/f32 to d0/f37 0 2024-04-26T04:52:45.660 INFO:tasks.workunit.client.1.smithi078.stdout:4/110: dwrite d0/de/d22/d27/f29 [0,4194304] 0 2024-04-26T04:52:45.664 INFO:tasks.workunit.client.1.smithi078.stdout:5/54: sync 2024-04-26T04:52:45.669 INFO:tasks.workunit.client.1.smithi078.stdout:4/111: dwrite d0/f37 [0,4194304] 0 2024-04-26T04:52:45.674 INFO:tasks.workunit.client.1.smithi078.stdout:4/112: dread d0/f5 [4194304,4194304] 0 2024-04-26T04:52:45.677 INFO:tasks.workunit.client.1.smithi078.stdout:4/113: rename d0/d25/d20/l1f to d0/d25/d20/d31/d33/l38 0 2024-04-26T04:52:45.692 INFO:tasks.workunit.client.1.smithi078.stdout:5/55: creat d7/da/f10 x:0 0 0 2024-04-26T04:52:45.697 INFO:tasks.workunit.client.1.smithi078.stdout:5/56: mkdir d7/d11 0 2024-04-26T04:52:45.699 INFO:tasks.workunit.client.1.smithi078.stdout:9/63: write f1 [357603,66325] 0 2024-04-26T04:52:45.702 INFO:tasks.workunit.client.1.smithi078.stdout:9/64: mkdir dc/df 0 2024-04-26T04:52:45.705 INFO:tasks.workunit.client.1.smithi078.stdout:9/65: mknod dc/c10 0 2024-04-26T04:52:45.706 INFO:tasks.workunit.client.1.smithi078.stdout:9/66: rename dc to dc/d11 22 2024-04-26T04:52:45.706 INFO:tasks.workunit.client.1.smithi078.stdout:2/70: dwrite d8/ff [0,4194304] 0 2024-04-26T04:52:45.708 INFO:tasks.workunit.client.1.smithi078.stdout:6/87: dwrite d2/d3/d5/d11/f15 [0,4194304] 0 2024-04-26T04:52:45.717 INFO:tasks.workunit.client.1.smithi078.stdout:4/114: chown d0/d25/d20/d31/d33/l38 95933268 1 2024-04-26T04:52:45.718 INFO:tasks.workunit.client.1.smithi078.stdout:6/88: dwrite d2/d3/d5/f8 [0,4194304] 0 2024-04-26T04:52:45.720 INFO:tasks.workunit.client.1.smithi078.stdout:6/89: dread - d2/f14 zero size 2024-04-26T04:52:45.721 INFO:tasks.workunit.client.1.smithi078.stdout:5/57: creat d7/da/f12 x:0 0 0 2024-04-26T04:52:45.722 INFO:tasks.workunit.client.1.smithi078.stdout:3/80: dwrite f6 [0,4194304] 0 2024-04-26T04:52:45.722 INFO:tasks.workunit.client.1.smithi078.stdout:4/115: mkdir d0/d25/d20/d31/d33/d39 0 2024-04-26T04:52:45.722 INFO:tasks.workunit.client.1.smithi078.stdout:6/90: chown d2/d3/d5/da 9241 1 2024-04-26T04:52:45.723 INFO:tasks.workunit.client.1.smithi078.stdout:2/71: write f3 [4424832,60938] 0 2024-04-26T04:52:45.725 INFO:tasks.workunit.client.1.smithi078.stdout:3/81: readlink l8 0 2024-04-26T04:52:45.725 INFO:tasks.workunit.client.1.smithi078.stdout:6/91: rename d2/d3 to d2/d3/d5/d18 22 2024-04-26T04:52:45.727 INFO:tasks.workunit.client.1.smithi078.stdout:3/82: readlink d16/l24 0 2024-04-26T04:52:45.727 INFO:tasks.workunit.client.1.smithi078.stdout:4/116: chown d0/d25/d20/d31/d33/l38 5654 1 2024-04-26T04:52:45.729 INFO:tasks.workunit.client.1.smithi078.stdout:3/83: write f12 [616137,11308] 0 2024-04-26T04:52:45.729 INFO:tasks.workunit.client.1.smithi078.stdout:2/72: mkdir d8/d11/d14/d15 0 2024-04-26T04:52:45.731 INFO:tasks.workunit.client.1.smithi078.stdout:4/117: chown d0/d25/d20/f1b 27062 1 2024-04-26T04:52:45.734 INFO:tasks.workunit.client.1.smithi078.stdout:3/84: creat d16/f25 x:0 0 0 2024-04-26T04:52:45.734 INFO:tasks.workunit.client.1.smithi078.stdout:9/67: dwrite dc/dd/fe [0,4194304] 0 2024-04-26T04:52:45.735 INFO:tasks.workunit.client.1.smithi078.stdout:5/58: getdents d7/da 0 2024-04-26T04:52:45.737 INFO:tasks.workunit.client.1.smithi078.stdout:2/73: symlink d8/d11/l16 0 2024-04-26T04:52:45.739 INFO:tasks.workunit.client.1.smithi078.stdout:2/74: write d8/ff [4912969,115802] 0 2024-04-26T04:52:45.741 INFO:tasks.workunit.client.1.smithi078.stdout:6/92: dwrite d2/f14 [0,4194304] 0 2024-04-26T04:52:45.743 INFO:tasks.workunit.client.1.smithi078.stdout:3/85: mkdir d16/d26 0 2024-04-26T04:52:45.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/75: mknod d8/c17 0 2024-04-26T04:52:45.746 INFO:tasks.workunit.client.1.smithi078.stdout:6/93: unlink f1 0 2024-04-26T04:52:45.747 INFO:tasks.workunit.client.1.smithi078.stdout:4/118: dwrite d0/d25/d20/f15 [0,4194304] 0 2024-04-26T04:52:45.747 INFO:tasks.workunit.client.1.smithi078.stdout:3/86: symlink d16/l27 0 2024-04-26T04:52:45.747 INFO:tasks.workunit.client.1.smithi078.stdout:9/68: dread f1 [0,4194304] 0 2024-04-26T04:52:45.748 INFO:tasks.workunit.client.1.smithi078.stdout:5/59: getdents d7/de 0 2024-04-26T04:52:45.748 INFO:tasks.workunit.client.1.smithi078.stdout:3/87: dread - d16/f21 zero size 2024-04-26T04:52:45.749 INFO:tasks.workunit.client.1.smithi078.stdout:2/76: mkdir d8/d11/d18 0 2024-04-26T04:52:45.749 INFO:tasks.workunit.client.1.smithi078.stdout:4/119: fsync d0/de/d22/d27/f29 0 2024-04-26T04:52:45.749 INFO:tasks.workunit.client.1.smithi078.stdout:7/118: sync 2024-04-26T04:52:45.750 INFO:tasks.workunit.client.1.smithi078.stdout:6/94: write d2/d3/d5/f8 [3534057,81835] 0 2024-04-26T04:52:45.752 INFO:tasks.workunit.client.1.smithi078.stdout:6/95: chown d2/cf 0 1 2024-04-26T04:52:45.754 INFO:tasks.workunit.client.1.smithi078.stdout:3/88: mknod d16/c28 0 2024-04-26T04:52:45.754 INFO:tasks.workunit.client.1.smithi078.stdout:6/96: write d2/f14 [3104685,48394] 0 2024-04-26T04:52:45.755 INFO:tasks.workunit.client.1.smithi078.stdout:5/60: creat d7/d11/f13 x:0 0 0 2024-04-26T04:52:45.756 INFO:tasks.workunit.client.1.smithi078.stdout:3/89: chown f12 2034223 1 2024-04-26T04:52:45.756 INFO:tasks.workunit.client.1.smithi078.stdout:7/119: mkdir d9/d16/d28 0 2024-04-26T04:52:45.757 INFO:tasks.workunit.client.1.smithi078.stdout:4/120: creat d0/d25/f3a x:0 0 0 2024-04-26T04:52:45.758 INFO:tasks.workunit.client.1.smithi078.stdout:7/120: chown l8 10 1 2024-04-26T04:52:45.758 INFO:tasks.workunit.client.1.smithi078.stdout:2/77: creat d8/d11/d14/f19 x:0 0 0 2024-04-26T04:52:45.759 INFO:tasks.workunit.client.1.smithi078.stdout:9/69: dwrite f5 [0,4194304] 0 2024-04-26T04:52:45.760 INFO:tasks.workunit.client.1.smithi078.stdout:6/97: rename d2/cf to d2/de/c19 0 2024-04-26T04:52:45.761 INFO:tasks.workunit.client.1.smithi078.stdout:3/90: truncate ff 933553 0 2024-04-26T04:52:45.761 INFO:tasks.workunit.client.1.smithi078.stdout:5/61: mknod d7/d11/c14 0 2024-04-26T04:52:45.762 INFO:tasks.workunit.client.1.smithi078.stdout:4/121: mkdir d0/de/d22/d27/d3b 0 2024-04-26T04:52:45.763 INFO:tasks.workunit.client.1.smithi078.stdout:7/121: write d9/da/d11/d13/d1a/f1c [597994,33266] 0 2024-04-26T04:52:45.764 INFO:tasks.workunit.client.1.smithi078.stdout:6/98: rmdir d2/d3/d5 39 2024-04-26T04:52:45.765 INFO:tasks.workunit.client.1.smithi078.stdout:4/122: fdatasync d0/de/f1e 0 2024-04-26T04:52:45.766 INFO:tasks.workunit.client.1.smithi078.stdout:5/62: write d7/da/f10 [186272,118598] 0 2024-04-26T04:52:45.767 INFO:tasks.workunit.client.1.smithi078.stdout:5/63: fsync f3 0 2024-04-26T04:52:45.768 INFO:tasks.workunit.client.1.smithi078.stdout:4/123: write d0/d1d/f36 [98228,10471] 0 2024-04-26T04:52:45.770 INFO:tasks.workunit.client.1.smithi078.stdout:4/124: chown d0 321 1 2024-04-26T04:52:45.771 INFO:tasks.workunit.client.1.smithi078.stdout:9/70: dread f7 [0,4194304] 0 2024-04-26T04:52:45.773 INFO:tasks.workunit.client.1.smithi078.stdout:3/91: creat d16/d26/f29 x:0 0 0 2024-04-26T04:52:45.775 INFO:tasks.workunit.client.1.smithi078.stdout:5/64: dread f5 [0,4194304] 0 2024-04-26T04:52:45.775 INFO:tasks.workunit.client.1.smithi078.stdout:6/99: unlink d2/d3/d5/da/f17 0 2024-04-26T04:52:45.776 INFO:tasks.workunit.client.1.smithi078.stdout:6/100: stat d2/de 0 2024-04-26T04:52:45.777 INFO:tasks.workunit.client.1.smithi078.stdout:3/92: creat d16/d26/f2a x:0 0 0 2024-04-26T04:52:45.778 INFO:tasks.workunit.client.1.smithi078.stdout:4/125: rmdir d0/d25/d20/d31/d33/d39 0 2024-04-26T04:52:45.779 INFO:tasks.workunit.client.1.smithi078.stdout:7/122: link d9/da/f15 d9/da/d11/d13/d1a/f29 0 2024-04-26T04:52:45.782 INFO:tasks.workunit.client.1.smithi078.stdout:4/126: readlink d0/de/d22/d26/d2e/l30 0 2024-04-26T04:52:45.786 INFO:tasks.workunit.client.1.smithi078.stdout:9/71: dwrite f1 [0,4194304] 0 2024-04-26T04:52:45.787 INFO:tasks.workunit.client.1.smithi078.stdout:4/127: read - d0/de/f1c zero size 2024-04-26T04:52:45.788 INFO:tasks.workunit.client.1.smithi078.stdout:5/65: dwrite d7/da/f12 [0,4194304] 0 2024-04-26T04:52:45.790 INFO:tasks.workunit.client.1.smithi078.stdout:6/101: dread d2/d3/d5/d11/f15 [0,4194304] 0 2024-04-26T04:52:45.790 INFO:tasks.workunit.client.1.smithi078.stdout:9/72: unlink cb 0 2024-04-26T04:52:45.791 INFO:tasks.workunit.client.1.smithi078.stdout:4/128: unlink d0/de/ff 0 2024-04-26T04:52:45.792 INFO:tasks.workunit.client.1.smithi078.stdout:3/93: getdents d16/d26 0 2024-04-26T04:52:45.793 INFO:tasks.workunit.client.1.smithi078.stdout:9/73: chown dc/dd/fe 1777589 1 2024-04-26T04:52:45.795 INFO:tasks.workunit.client.1.smithi078.stdout:5/66: symlink d7/d11/l15 0 2024-04-26T04:52:45.795 INFO:tasks.workunit.client.1.smithi078.stdout:2/78: dwrite d8/f9 [0,4194304] 0 2024-04-26T04:52:45.796 INFO:tasks.workunit.client.1.smithi078.stdout:5/67: dread - d7/d11/f13 zero size 2024-04-26T04:52:45.797 INFO:tasks.workunit.client.1.smithi078.stdout:7/123: write f6 [2744000,889] 0 2024-04-26T04:52:45.798 INFO:tasks.workunit.client.1.smithi078.stdout:3/94: mkdir d16/d26/d2b 0 2024-04-26T04:52:45.799 INFO:tasks.workunit.client.1.smithi078.stdout:6/102: dread d2/f14 [0,4194304] 0 2024-04-26T04:52:45.799 INFO:tasks.workunit.client.1.smithi078.stdout:4/129: rmdir d0/de/d22/d26/d2e 39 2024-04-26T04:52:45.801 INFO:tasks.workunit.client.1.smithi078.stdout:3/95: chown d16/f1c 126018 1 2024-04-26T04:52:45.803 INFO:tasks.workunit.client.1.smithi078.stdout:6/103: stat d2/de/f12 0 2024-04-26T04:52:45.803 INFO:tasks.workunit.client.1.smithi078.stdout:9/74: mknod dc/df/c12 0 2024-04-26T04:52:45.806 INFO:tasks.workunit.client.1.smithi078.stdout:3/96: creat d16/d26/f2c x:0 0 0 2024-04-26T04:52:45.807 INFO:tasks.workunit.client.1.smithi078.stdout:5/68: creat d7/f16 x:0 0 0 2024-04-26T04:52:45.807 INFO:tasks.workunit.client.1.smithi078.stdout:2/79: dwrite d8/f9 [4194304,4194304] 0 2024-04-26T04:52:45.807 INFO:tasks.workunit.client.1.smithi078.stdout:7/124: dread f6 [0,4194304] 0 2024-04-26T04:52:45.807 INFO:tasks.workunit.client.1.smithi078.stdout:3/97: write d16/f21 [722476,70468] 0 2024-04-26T04:52:45.808 INFO:tasks.workunit.client.1.smithi078.stdout:8/156: sync 2024-04-26T04:52:45.808 INFO:tasks.workunit.client.1.smithi078.stdout:4/130: rename d0/de/c2c to d0/d25/d20/d31/d33/c3c 0 2024-04-26T04:52:45.812 INFO:tasks.workunit.client.1.smithi078.stdout:4/131: fsync d0/d25/d20/f23 0 2024-04-26T04:52:45.813 INFO:tasks.workunit.client.1.smithi078.stdout:5/69: symlink d7/l17 0 2024-04-26T04:52:45.813 INFO:tasks.workunit.client.1.smithi078.stdout:2/80: mkdir d8/d11/d14/d15/d1a 0 2024-04-26T04:52:45.815 INFO:tasks.workunit.client.1.smithi078.stdout:3/98: symlink d16/d26/d2b/l2d 0 2024-04-26T04:52:45.815 INFO:tasks.workunit.client.1.smithi078.stdout:9/75: dread f1 [0,4194304] 0 2024-04-26T04:52:45.815 INFO:tasks.workunit.client.1.smithi078.stdout:8/157: truncate d0/d21/f24 25073 0 2024-04-26T04:52:45.816 INFO:tasks.workunit.client.1.smithi078.stdout:9/76: read f1 [3935102,82514] 0 2024-04-26T04:52:45.816 INFO:tasks.workunit.client.1.smithi078.stdout:2/81: truncate d8/d11/d14/f19 587843 0 2024-04-26T04:52:45.817 INFO:tasks.workunit.client.1.smithi078.stdout:5/70: unlink d7/d11/l15 0 2024-04-26T04:52:45.819 INFO:tasks.workunit.client.1.smithi078.stdout:7/125: getdents d9 0 2024-04-26T04:52:45.822 INFO:tasks.workunit.client.1.smithi078.stdout:9/77: creat dc/dd/f13 x:0 0 0 2024-04-26T04:52:45.823 INFO:tasks.workunit.client.1.smithi078.stdout:7/126: fdatasync d9/da/d11/d13/d1a/f1e 0 2024-04-26T04:52:45.823 INFO:tasks.workunit.client.1.smithi078.stdout:2/82: creat d8/d11/d14/d15/f1b x:0 0 0 2024-04-26T04:52:45.825 INFO:tasks.workunit.client.1.smithi078.stdout:5/71: mkdir d7/d18 0 2024-04-26T04:52:45.826 INFO:tasks.workunit.client.1.smithi078.stdout:9/78: write f5 [1781475,126600] 0 2024-04-26T04:52:45.828 INFO:tasks.workunit.client.1.smithi078.stdout:3/99: dwrite d16/d26/f2c [0,4194304] 0 2024-04-26T04:52:45.828 INFO:tasks.workunit.client.1.smithi078.stdout:4/132: dwrite d0/d25/f19 [0,4194304] 0 2024-04-26T04:52:45.830 INFO:tasks.workunit.client.1.smithi078.stdout:4/133: readlink d0/d25/d20/d31/d33/l38 0 2024-04-26T04:52:45.831 INFO:tasks.workunit.client.1.smithi078.stdout:2/83: creat d8/d11/d14/f1c x:0 0 0 2024-04-26T04:52:45.833 INFO:tasks.workunit.client.1.smithi078.stdout:7/127: rename d9/f1b to d9/da/d11/f2a 0 2024-04-26T04:52:45.834 INFO:tasks.workunit.client.1.smithi078.stdout:5/72: dread f3 [0,4194304] 0 2024-04-26T04:52:45.835 INFO:tasks.workunit.client.1.smithi078.stdout:7/128: write d9/fe [304830,122745] 0 2024-04-26T04:52:45.837 INFO:tasks.workunit.client.1.smithi078.stdout:2/84: unlink d8/ff 0 2024-04-26T04:52:45.838 INFO:tasks.workunit.client.1.smithi078.stdout:2/85: read d8/d11/d14/f19 [188115,108991] 0 2024-04-26T04:52:45.839 INFO:tasks.workunit.client.1.smithi078.stdout:7/129: dread d9/da/f15 [0,4194304] 0 2024-04-26T04:52:45.840 INFO:tasks.workunit.client.1.smithi078.stdout:9/79: rename f9 to dc/f14 0 2024-04-26T04:52:45.842 INFO:tasks.workunit.client.1.smithi078.stdout:9/80: dread - dc/dd/f13 zero size 2024-04-26T04:52:45.844 INFO:tasks.workunit.client.1.smithi078.stdout:4/134: creat d0/de/d22/d27/d3b/f3d x:0 0 0 2024-04-26T04:52:45.845 INFO:tasks.workunit.client.1.smithi078.stdout:2/86: dread d8/fb [4194304,4194304] 0 2024-04-26T04:52:45.845 INFO:tasks.workunit.client.1.smithi078.stdout:8/158: truncate d0/f5 2014258 0 2024-04-26T04:52:45.846 INFO:tasks.workunit.client.1.smithi078.stdout:4/135: chown d0/de/d22/d27 175930434 1 2024-04-26T04:52:45.846 INFO:tasks.workunit.client.1.smithi078.stdout:8/159: dread - d0/d21/f29 zero size 2024-04-26T04:52:45.847 INFO:tasks.workunit.client.1.smithi078.stdout:7/130: dwrite d9/f14 [0,4194304] 0 2024-04-26T04:52:45.847 INFO:tasks.workunit.client.1.smithi078.stdout:7/131: readlink l8 0 2024-04-26T04:52:45.848 INFO:tasks.workunit.client.1.smithi078.stdout:5/73: mknod d7/c19 0 2024-04-26T04:52:45.850 INFO:tasks.workunit.client.1.smithi078.stdout:9/81: dread f8 [0,4194304] 0 2024-04-26T04:52:45.852 INFO:tasks.workunit.client.1.smithi078.stdout:2/87: mknod d8/d11/d14/d15/c1d 0 2024-04-26T04:52:45.855 INFO:tasks.workunit.client.1.smithi078.stdout:5/74: unlink l0 0 2024-04-26T04:52:45.856 INFO:tasks.workunit.client.1.smithi078.stdout:7/132: creat d9/d16/f2b x:0 0 0 2024-04-26T04:52:45.859 INFO:tasks.workunit.client.1.smithi078.stdout:9/82: unlink f7 0 2024-04-26T04:52:45.859 INFO:tasks.workunit.client.1.smithi078.stdout:4/136: creat d0/d2f/f3e x:0 0 0 2024-04-26T04:52:45.863 INFO:tasks.workunit.client.1.smithi078.stdout:0/86: sync 2024-04-26T04:52:45.863 INFO:tasks.workunit.client.1.smithi078.stdout:1/185: sync 2024-04-26T04:52:45.863 INFO:tasks.workunit.client.1.smithi078.stdout:8/160: creat d0/d19/d25/d36/f37 x:0 0 0 2024-04-26T04:52:45.865 INFO:tasks.workunit.client.1.smithi078.stdout:9/83: mknod dc/df/c15 0 2024-04-26T04:52:45.866 INFO:tasks.workunit.client.1.smithi078.stdout:5/75: creat d7/d18/f1a x:0 0 0 2024-04-26T04:52:45.867 INFO:tasks.workunit.client.1.smithi078.stdout:0/87: chown d2/d9/c11 519330780 1 2024-04-26T04:52:45.868 INFO:tasks.workunit.client.1.smithi078.stdout:7/133: mkdir d9/d16/d28/d2c 0 2024-04-26T04:52:45.869 INFO:tasks.workunit.client.1.smithi078.stdout:8/161: mknod d0/d19/c38 0 2024-04-26T04:52:45.869 INFO:tasks.workunit.client.1.smithi078.stdout:4/137: dwrite d0/de/d22/d27/d3b/f3d [0,4194304] 0 2024-04-26T04:52:45.869 INFO:tasks.workunit.client.1.smithi078.stdout:2/88: truncate d8/fb 5119652 0 2024-04-26T04:52:45.871 INFO:tasks.workunit.client.1.smithi078.stdout:1/186: rename d7/d39/d17/f1d to d7/d39/db/f41 0 2024-04-26T04:52:45.874 INFO:tasks.workunit.client.1.smithi078.stdout:2/89: truncate d8/d11/d14/f1c 1015833 0 2024-04-26T04:52:45.874 INFO:tasks.workunit.client.1.smithi078.stdout:2/90: rename d8 to d8/d1e 22 2024-04-26T04:52:45.876 INFO:tasks.workunit.client.1.smithi078.stdout:5/76: dwrite d7/da/f10 [0,4194304] 0 2024-04-26T04:52:45.877 INFO:tasks.workunit.client.1.smithi078.stdout:2/91: truncate d8/d11/d14/d15/f1b 256064 0 2024-04-26T04:52:45.877 INFO:tasks.workunit.client.1.smithi078.stdout:5/77: dread - d7/d18/f1a zero size 2024-04-26T04:52:45.878 INFO:tasks.workunit.client.1.smithi078.stdout:8/162: read d0/f2f [403780,46290] 0 2024-04-26T04:52:45.879 INFO:tasks.workunit.client.1.smithi078.stdout:4/138: creat d0/d25/d20/d31/f3f x:0 0 0 2024-04-26T04:52:45.879 INFO:tasks.workunit.client.1.smithi078.stdout:8/163: chown d0/cc 954296 1 2024-04-26T04:52:45.880 INFO:tasks.workunit.client.1.smithi078.stdout:8/164: read d0/f23 [583994,94829] 0 2024-04-26T04:52:45.883 INFO:tasks.workunit.client.1.smithi078.stdout:7/134: unlink d9/d16/f2b 0 2024-04-26T04:52:45.887 INFO:tasks.workunit.client.1.smithi078.stdout:5/78: mknod d7/d11/c1b 0 2024-04-26T04:52:45.888 INFO:tasks.workunit.client.1.smithi078.stdout:5/79: fsync d7/d11/f13 0 2024-04-26T04:52:45.890 INFO:tasks.workunit.client.1.smithi078.stdout:4/139: rename d0/f6 to d0/d25/d20/d31/d33/f40 0 2024-04-26T04:52:45.892 INFO:tasks.workunit.client.1.smithi078.stdout:5/80: stat d7/f8 0 2024-04-26T04:52:45.892 INFO:tasks.workunit.client.1.smithi078.stdout:4/140: dread - d0/de/f1e zero size 2024-04-26T04:52:45.893 INFO:tasks.workunit.client.1.smithi078.stdout:4/141: read - d0/f13 zero size 2024-04-26T04:52:45.893 INFO:tasks.workunit.client.1.smithi078.stdout:8/165: symlink d0/l39 0 2024-04-26T04:52:45.894 INFO:tasks.workunit.client.1.smithi078.stdout:8/166: read - d0/d21/f29 zero size 2024-04-26T04:52:45.896 INFO:tasks.workunit.client.1.smithi078.stdout:7/135: rename d9/d16/d28/d2c to d9/da/d11/d13/d1a/d2d 0 2024-04-26T04:52:45.896 INFO:tasks.workunit.client.1.smithi078.stdout:9/84: truncate f1 2598661 0 2024-04-26T04:52:45.896 INFO:tasks.workunit.client.1.smithi078.stdout:9/85: readlink - no filename 2024-04-26T04:52:45.898 INFO:tasks.workunit.client.1.smithi078.stdout:5/81: mkdir d7/d18/d1c 0 2024-04-26T04:52:45.899 INFO:tasks.workunit.client.1.smithi078.stdout:1/187: getdents d7/d1e/d35 0 2024-04-26T04:52:45.900 INFO:tasks.workunit.client.1.smithi078.stdout:4/142: mknod d0/de/d22/d27/d3b/c41 0 2024-04-26T04:52:45.901 INFO:tasks.workunit.client.1.smithi078.stdout:8/167: dwrite d0/d21/f29 [0,4194304] 0 2024-04-26T04:52:45.902 INFO:tasks.workunit.client.1.smithi078.stdout:4/143: write d0/d25/d20/f15 [4833609,62974] 0 2024-04-26T04:52:45.903 INFO:tasks.workunit.client.1.smithi078.stdout:8/168: write d0/d19/f2a [301827,115956] 0 2024-04-26T04:52:45.904 INFO:tasks.workunit.client.1.smithi078.stdout:7/136: symlink d9/d16/d28/l2e 0 2024-04-26T04:52:45.906 INFO:tasks.workunit.client.1.smithi078.stdout:9/86: creat dc/f16 x:0 0 0 2024-04-26T04:52:45.907 INFO:tasks.workunit.client.1.smithi078.stdout:5/82: creat d7/d11/f1d x:0 0 0 2024-04-26T04:52:45.907 INFO:tasks.workunit.client.1.smithi078.stdout:1/188: rmdir d7/d39/d17 39 2024-04-26T04:52:45.909 INFO:tasks.workunit.client.1.smithi078.stdout:8/169: truncate d0/d3/f26 841983 0 2024-04-26T04:52:45.910 INFO:tasks.workunit.client.1.smithi078.stdout:5/83: dread - d7/d11/f1d zero size 2024-04-26T04:52:45.910 INFO:tasks.workunit.client.1.smithi078.stdout:9/87: unlink fa 0 2024-04-26T04:52:45.910 INFO:tasks.workunit.client.1.smithi078.stdout:1/189: chown d7/d39/dd/f2d 15 1 2024-04-26T04:52:45.910 INFO:tasks.workunit.client.1.smithi078.stdout:8/170: fsync d0/f10 0 2024-04-26T04:52:45.911 INFO:tasks.workunit.client.1.smithi078.stdout:2/92: dwrite f3 [4194304,4194304] 0 2024-04-26T04:52:45.911 INFO:tasks.workunit.client.1.smithi078.stdout:7/137: fdatasync d9/da/d11/d13/d1a/f29 0 2024-04-26T04:52:45.913 INFO:tasks.workunit.client.1.smithi078.stdout:7/138: chown d9/d16/l1d 7039 1 2024-04-26T04:52:45.914 INFO:tasks.workunit.client.1.smithi078.stdout:7/139: chown d9/lf 6 1 2024-04-26T04:52:45.915 INFO:tasks.workunit.client.1.smithi078.stdout:4/144: mkdir d0/de/d22/d26/d2e/d42 0 2024-04-26T04:52:45.916 INFO:tasks.workunit.client.1.smithi078.stdout:5/84: dread f3 [0,4194304] 0 2024-04-26T04:52:45.921 INFO:tasks.workunit.client.1.smithi078.stdout:9/88: mkdir dc/df/d17 0 2024-04-26T04:52:45.922 INFO:tasks.workunit.client.1.smithi078.stdout:4/145: dwrite d0/f13 [0,4194304] 0 2024-04-26T04:52:45.922 INFO:tasks.workunit.client.1.smithi078.stdout:1/190: symlink d7/d1e/l42 0 2024-04-26T04:52:45.927 INFO:tasks.workunit.client.1.smithi078.stdout:5/85: chown d7/ld 1759883 1 2024-04-26T04:52:45.929 INFO:tasks.workunit.client.1.smithi078.stdout:9/89: mkdir dc/d18 0 2024-04-26T04:52:45.930 INFO:tasks.workunit.client.1.smithi078.stdout:4/146: mknod d0/d25/d20/d31/c43 0 2024-04-26T04:52:45.931 INFO:tasks.workunit.client.1.smithi078.stdout:5/86: dread d7/da/f12 [0,4194304] 0 2024-04-26T04:52:45.934 INFO:tasks.workunit.client.1.smithi078.stdout:4/147: dread d0/f37 [0,4194304] 0 2024-04-26T04:52:45.935 INFO:tasks.workunit.client.1.smithi078.stdout:4/148: fsync d0/de/d22/d27/d3b/f3d 0 2024-04-26T04:52:45.936 INFO:tasks.workunit.client.1.smithi078.stdout:4/149: dread - d0/de/f1c zero size 2024-04-26T04:52:45.942 INFO:tasks.workunit.client.1.smithi078.stdout:5/87: unlink d7/d11/f13 0 2024-04-26T04:52:45.944 INFO:tasks.workunit.client.1.smithi078.stdout:8/171: rename d0/d21/la to d0/l3a 0 2024-04-26T04:52:45.944 INFO:tasks.workunit.client.1.smithi078.stdout:8/172: write d0/d21/f29 [904734,94760] 0 2024-04-26T04:52:45.945 INFO:tasks.workunit.client.1.smithi078.stdout:7/140: creat d9/da/d11/f2f x:0 0 0 2024-04-26T04:52:45.946 INFO:tasks.workunit.client.1.smithi078.stdout:7/141: dread - d9/d16/f26 zero size 2024-04-26T04:52:45.947 INFO:tasks.workunit.client.1.smithi078.stdout:7/142: fsync d9/da/d11/d13/f25 0 2024-04-26T04:52:45.948 INFO:tasks.workunit.client.1.smithi078.stdout:8/173: dread d0/f18 [0,4194304] 0 2024-04-26T04:52:45.948 INFO:tasks.workunit.client.1.smithi078.stdout:2/93: write d8/fa [701154,44250] 0 2024-04-26T04:52:45.949 INFO:tasks.workunit.client.1.smithi078.stdout:2/94: chown d8/le 1 1 2024-04-26T04:52:45.954 INFO:tasks.workunit.client.1.smithi078.stdout:5/88: mkdir d7/da/d1e 0 2024-04-26T04:52:45.956 INFO:tasks.workunit.client.1.smithi078.stdout:4/150: rename d0/de/d22/d27/d3b to d0/d25/d20/d31/d33/d44 0 2024-04-26T04:52:45.961 INFO:tasks.workunit.client.1.smithi078.stdout:8/174: symlink d0/d19/l3b 0 2024-04-26T04:52:45.963 INFO:tasks.workunit.client.1.smithi078.stdout:8/175: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:45.963 INFO:tasks.workunit.client.1.smithi078.stdout:1/191: truncate d7/d39/d1f/f26 4789495 0 2024-04-26T04:52:45.963 INFO:tasks.workunit.client.1.smithi078.stdout:1/192: chown d7/d39/db/d37 46023 1 2024-04-26T04:52:45.964 INFO:tasks.workunit.client.1.smithi078.stdout:1/193: chown d7/ff 3 1 2024-04-26T04:52:45.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:45 smithi077 ceph-mon[26311]: pgmap v12: 65 pgs: 65 active+clean; 329 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 18 MiB/s wr, 462 op/s 2024-04-26T04:52:46.060 INFO:tasks.workunit.client.1.smithi078.stdout:4/151: sync 2024-04-26T04:52:46.060 INFO:tasks.workunit.client.1.smithi078.stdout:0/88: sync 2024-04-26T04:52:46.060 INFO:tasks.workunit.client.1.smithi078.stdout:3/100: sync 2024-04-26T04:52:46.060 INFO:tasks.workunit.client.1.smithi078.stdout:6/104: sync 2024-04-26T04:52:46.060 INFO:tasks.workunit.client.1.smithi078.stdout:8/176: sync 2024-04-26T04:52:46.061 INFO:tasks.workunit.client.1.smithi078.stdout:4/152: readlink d0/d25/d20/l18 0 2024-04-26T04:52:46.080 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:45 smithi078 ceph-mon[33746]: pgmap v12: 65 pgs: 65 active+clean; 329 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 18 MiB/s wr, 462 op/s 2024-04-26T04:52:46.116 INFO:tasks.workunit.client.1.smithi078.stdout:4/153: sync 2024-04-26T04:52:46.118 INFO:tasks.workunit.client.1.smithi078.stdout:5/89: creat d7/d11/f1f x:0 0 0 2024-04-26T04:52:46.121 INFO:tasks.workunit.client.1.smithi078.stdout:7/143: mknod d9/da/d11/c30 0 2024-04-26T04:52:46.129 INFO:tasks.workunit.client.1.smithi078.stdout:9/90: dwrite f1 [0,4194304] 0 2024-04-26T04:52:46.133 INFO:tasks.workunit.client.1.smithi078.stdout:3/101: mkdir d16/d26/d2e 0 2024-04-26T04:52:46.134 INFO:tasks.workunit.client.1.smithi078.stdout:3/102: chown d16/f20 16250718 1 2024-04-26T04:52:46.134 INFO:tasks.workunit.client.1.smithi078.stdout:1/194: dwrite d7/d39/db/f41 [0,4194304] 0 2024-04-26T04:52:46.135 INFO:tasks.workunit.client.1.smithi078.stdout:8/177: mkdir d0/d21/d3c 0 2024-04-26T04:52:46.139 INFO:tasks.workunit.client.1.smithi078.stdout:6/105: rename d2/d3 to d2/d1a 0 2024-04-26T04:52:46.140 INFO:tasks.workunit.client.1.smithi078.stdout:6/106: read - d2/f16 zero size 2024-04-26T04:52:46.140 INFO:tasks.workunit.client.1.smithi078.stdout:6/107: chown c0 4639665 1 2024-04-26T04:52:46.144 INFO:tasks.workunit.client.1.smithi078.stdout:4/154: creat d0/d1d/f45 x:0 0 0 2024-04-26T04:52:46.148 INFO:tasks.workunit.client.1.smithi078.stdout:7/144: read d9/da/d20/f24 [363995,101958] 0 2024-04-26T04:52:46.150 INFO:tasks.workunit.client.1.smithi078.stdout:9/91: fsync f8 0 2024-04-26T04:52:46.150 INFO:tasks.workunit.client.1.smithi078.stdout:5/90: fsync d7/d11/f1f 0 2024-04-26T04:52:46.156 INFO:tasks.workunit.client.1.smithi078.stdout:6/108: creat d2/d1a/d5/da/f1b x:0 0 0 2024-04-26T04:52:46.157 INFO:tasks.workunit.client.1.smithi078.stdout:4/155: rmdir d0/de/d22/d26/d2e 39 2024-04-26T04:52:46.159 INFO:tasks.workunit.client.1.smithi078.stdout:0/89: creat d2/f19 x:0 0 0 2024-04-26T04:52:46.160 INFO:tasks.workunit.client.1.smithi078.stdout:7/145: creat d9/da/d20/f31 x:0 0 0 2024-04-26T04:52:46.163 INFO:tasks.workunit.client.1.smithi078.stdout:2/95: truncate d8/d11/d14/d15/f1b 196626 0 2024-04-26T04:52:46.164 INFO:tasks.workunit.client.1.smithi078.stdout:9/92: rename dc/f16 to dc/df/f19 0 2024-04-26T04:52:46.166 INFO:tasks.workunit.client.1.smithi078.stdout:3/103: unlink cb 0 2024-04-26T04:52:46.171 INFO:tasks.workunit.client.1.smithi078.stdout:4/156: unlink d0/de/d22/d27/f29 0 2024-04-26T04:52:46.173 INFO:tasks.workunit.client.1.smithi078.stdout:0/90: creat d2/d9/f1a x:0 0 0 2024-04-26T04:52:46.175 INFO:tasks.workunit.client.1.smithi078.stdout:7/146: mknod d9/da/d20/c32 0 2024-04-26T04:52:46.176 INFO:tasks.workunit.client.1.smithi078.stdout:5/91: symlink d7/da/d1e/l20 0 2024-04-26T04:52:46.177 INFO:tasks.workunit.client.1.smithi078.stdout:5/92: write d7/d11/f1d [658642,37524] 0 2024-04-26T04:52:46.179 INFO:tasks.workunit.client.1.smithi078.stdout:9/93: write dc/f14 [777021,18643] 0 2024-04-26T04:52:46.179 INFO:tasks.workunit.client.1.smithi078.stdout:9/94: fdatasync dc/dd/fe 0 2024-04-26T04:52:46.181 INFO:tasks.workunit.client.1.smithi078.stdout:6/109: creat d2/d1a/d5/d11/f1c x:0 0 0 2024-04-26T04:52:46.185 INFO:tasks.workunit.client.1.smithi078.stdout:4/157: mkdir d0/d1d/d46 0 2024-04-26T04:52:46.185 INFO:tasks.workunit.client.1.smithi078.stdout:8/178: sync 2024-04-26T04:52:46.188 INFO:tasks.workunit.client.1.smithi078.stdout:7/147: mkdir d9/da/d33 0 2024-04-26T04:52:46.193 INFO:tasks.workunit.client.1.smithi078.stdout:3/104: symlink d16/d26/d2e/l2f 0 2024-04-26T04:52:46.194 INFO:tasks.workunit.client.1.smithi078.stdout:9/95: mkdir dc/df/d1a 0 2024-04-26T04:52:46.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/96: write f1 [1204966,40683] 0 2024-04-26T04:52:46.196 INFO:tasks.workunit.client.1.smithi078.stdout:6/110: readlink d2/d1a/l13 0 2024-04-26T04:52:46.197 INFO:tasks.workunit.client.1.smithi078.stdout:6/111: readlink d2/d1a/l13 0 2024-04-26T04:52:46.198 INFO:tasks.workunit.client.1.smithi078.stdout:4/158: mkdir d0/d25/d20/d31/d47 0 2024-04-26T04:52:46.200 INFO:tasks.workunit.client.1.smithi078.stdout:8/179: rename d0/d19/c38 to d0/d19/c3d 0 2024-04-26T04:52:46.201 INFO:tasks.workunit.client.1.smithi078.stdout:8/180: write d0/d19/f1c [1727742,72010] 0 2024-04-26T04:52:46.202 INFO:tasks.workunit.client.1.smithi078.stdout:0/91: mknod d2/c1b 0 2024-04-26T04:52:46.204 INFO:tasks.workunit.client.1.smithi078.stdout:7/148: creat d9/f34 x:0 0 0 2024-04-26T04:52:46.206 INFO:tasks.workunit.client.1.smithi078.stdout:5/93: mkdir d7/d21 0 2024-04-26T04:52:46.210 INFO:tasks.workunit.client.1.smithi078.stdout:9/97: rename dc/df/c15 to dc/df/d17/c1b 0 2024-04-26T04:52:46.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/195: chown d7/d39/d1f/f26 919155944 1 2024-04-26T04:52:46.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/196: stat d7/d1e/f2f 0 2024-04-26T04:52:46.217 INFO:tasks.workunit.client.1.smithi078.stdout:7/149: fsync d9/da/d20/f24 0 2024-04-26T04:52:46.220 INFO:tasks.workunit.client.1.smithi078.stdout:5/94: write f5 [655831,62282] 0 2024-04-26T04:52:46.220 INFO:tasks.workunit.client.1.smithi078.stdout:6/112: dwrite d2/d1a/d5/d11/f15 [4194304,4194304] 0 2024-04-26T04:52:46.222 INFO:tasks.workunit.client.1.smithi078.stdout:5/95: read - d7/d11/f1f zero size 2024-04-26T04:52:46.225 INFO:tasks.workunit.client.1.smithi078.stdout:5/96: read d7/da/f10 [3120958,55143] 0 2024-04-26T04:52:46.225 INFO:tasks.workunit.client.1.smithi078.stdout:8/181: dread d0/f18 [0,4194304] 0 2024-04-26T04:52:46.225 INFO:tasks.workunit.client.1.smithi078.stdout:5/97: chown d7/d11 1571 1 2024-04-26T04:52:46.229 INFO:tasks.workunit.client.1.smithi078.stdout:2/96: dwrite d8/d11/d14/d15/f1b [0,4194304] 0 2024-04-26T04:52:46.229 INFO:tasks.workunit.client.1.smithi078.stdout:7/150: mknod d9/d16/c35 0 2024-04-26T04:52:46.231 INFO:tasks.workunit.client.1.smithi078.stdout:7/151: readlink d9/d16/l1d 0 2024-04-26T04:52:46.232 INFO:tasks.workunit.client.1.smithi078.stdout:1/197: dread d7/ff [0,4194304] 0 2024-04-26T04:52:46.233 INFO:tasks.workunit.client.1.smithi078.stdout:9/98: creat dc/df/d1a/f1c x:0 0 0 2024-04-26T04:52:46.237 INFO:tasks.workunit.client.1.smithi078.stdout:8/182: dwrite d0/f33 [0,4194304] 0 2024-04-26T04:52:46.237 INFO:tasks.workunit.client.1.smithi078.stdout:6/113: creat d2/f1d x:0 0 0 2024-04-26T04:52:46.238 INFO:tasks.workunit.client.1.smithi078.stdout:6/114: chown d2/de/f12 181523 1 2024-04-26T04:52:46.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/97: dwrite d8/d11/d14/d15/f1b [0,4194304] 0 2024-04-26T04:52:46.244 INFO:tasks.workunit.client.1.smithi078.stdout:2/98: truncate d8/fa 845566 0 2024-04-26T04:52:46.246 INFO:tasks.workunit.client.1.smithi078.stdout:6/115: dread d2/d1a/fd [0,4194304] 0 2024-04-26T04:52:46.251 INFO:tasks.workunit.client.1.smithi078.stdout:2/99: dwrite d8/d11/d14/f1c [0,4194304] 0 2024-04-26T04:52:46.256 INFO:tasks.workunit.client.1.smithi078.stdout:9/99: creat dc/df/f1d x:0 0 0 2024-04-26T04:52:46.257 INFO:tasks.workunit.client.1.smithi078.stdout:2/100: dread d8/fa [0,4194304] 0 2024-04-26T04:52:46.258 INFO:tasks.workunit.client.1.smithi078.stdout:8/183: mknod d0/d19/d25/d36/c3e 0 2024-04-26T04:52:46.262 INFO:tasks.workunit.client.1.smithi078.stdout:1/198: dwrite d7/ff [0,4194304] 0 2024-04-26T04:52:46.264 INFO:tasks.workunit.client.1.smithi078.stdout:4/159: rmdir d0/de/d22/d26/d2e/d42 0 2024-04-26T04:52:46.267 INFO:tasks.workunit.client.1.smithi078.stdout:2/101: dwrite d8/f9 [8388608,4194304] 0 2024-04-26T04:52:46.268 INFO:tasks.workunit.client.1.smithi078.stdout:6/116: creat d2/f1e x:0 0 0 2024-04-26T04:52:46.268 INFO:tasks.workunit.client.1.smithi078.stdout:4/160: write d0/d2f/f3e [284440,2463] 0 2024-04-26T04:52:46.269 INFO:tasks.workunit.client.1.smithi078.stdout:4/161: dread - d0/d1d/f45 zero size 2024-04-26T04:52:46.271 INFO:tasks.workunit.client.1.smithi078.stdout:4/162: write d0/d25/f3a [567009,81429] 0 2024-04-26T04:52:46.273 INFO:tasks.workunit.client.1.smithi078.stdout:4/163: chown d0/d25/d20/d31/f3f 1677 1 2024-04-26T04:52:46.275 INFO:tasks.workunit.client.1.smithi078.stdout:4/164: dread d0/d25/d20/f2a [0,4194304] 0 2024-04-26T04:52:46.275 INFO:tasks.workunit.client.1.smithi078.stdout:4/165: chown d0/d1d/c21 10470379 1 2024-04-26T04:52:46.279 INFO:tasks.workunit.client.1.smithi078.stdout:5/98: creat d7/f22 x:0 0 0 2024-04-26T04:52:46.282 INFO:tasks.workunit.client.1.smithi078.stdout:2/102: creat d8/f1f x:0 0 0 2024-04-26T04:52:46.285 INFO:tasks.workunit.client.1.smithi078.stdout:5/99: dwrite d7/f22 [0,4194304] 0 2024-04-26T04:52:46.286 INFO:tasks.workunit.client.1.smithi078.stdout:5/100: stat d7/c19 0 2024-04-26T04:52:46.289 INFO:tasks.workunit.client.1.smithi078.stdout:4/166: mkdir d0/d2f/d48 0 2024-04-26T04:52:46.289 INFO:tasks.workunit.client.1.smithi078.stdout:4/167: dread - d0/d25/d20/f23 zero size 2024-04-26T04:52:46.290 INFO:tasks.workunit.client.1.smithi078.stdout:8/184: mkdir d0/d21/d35/d3f 0 2024-04-26T04:52:46.292 INFO:tasks.workunit.client.1.smithi078.stdout:8/185: write d0/d19/d25/f2b [696393,56957] 0 2024-04-26T04:52:46.294 INFO:tasks.workunit.client.1.smithi078.stdout:1/199: dread d7/f11 [0,4194304] 0 2024-04-26T04:52:46.295 INFO:tasks.workunit.client.1.smithi078.stdout:6/117: mknod d2/d1a/c1f 0 2024-04-26T04:52:46.295 INFO:tasks.workunit.client.1.smithi078.stdout:1/200: read d7/fc [257440,49987] 0 2024-04-26T04:52:46.298 INFO:tasks.workunit.client.1.smithi078.stdout:7/152: getdents d9/d16/d28 0 2024-04-26T04:52:46.299 INFO:tasks.workunit.client.1.smithi078.stdout:3/105: sync 2024-04-26T04:52:46.306 INFO:tasks.workunit.client.1.smithi078.stdout:8/186: rename d0/l20 to d0/d21/d3c/l40 0 2024-04-26T04:52:46.308 INFO:tasks.workunit.client.1.smithi078.stdout:6/118: symlink d2/d1a/d5/da/l20 0 2024-04-26T04:52:46.309 INFO:tasks.workunit.client.1.smithi078.stdout:6/119: readlink d2/d1a/d5/da/l20 0 2024-04-26T04:52:46.310 INFO:tasks.workunit.client.1.smithi078.stdout:6/120: fsync d2/d1a/d5/d11/f1c 0 2024-04-26T04:52:46.311 INFO:tasks.workunit.client.1.smithi078.stdout:2/103: mknod d8/d11/d18/c20 0 2024-04-26T04:52:46.313 INFO:tasks.workunit.client.1.smithi078.stdout:7/153: unlink d9/f34 0 2024-04-26T04:52:46.314 INFO:tasks.workunit.client.1.smithi078.stdout:3/106: creat d16/f30 x:0 0 0 2024-04-26T04:52:46.315 INFO:tasks.workunit.client.1.smithi078.stdout:9/100: getdents dc/df/d1a 0 2024-04-26T04:52:46.318 INFO:tasks.workunit.client.1.smithi078.stdout:5/101: link d7/f22 d7/da/f23 0 2024-04-26T04:52:46.319 INFO:tasks.workunit.client.1.smithi078.stdout:0/92: sync 2024-04-26T04:52:46.319 INFO:tasks.workunit.client.1.smithi078.stdout:0/93: readlink - no filename 2024-04-26T04:52:46.321 INFO:tasks.workunit.client.1.smithi078.stdout:3/107: dwrite fc [0,4194304] 0 2024-04-26T04:52:46.324 INFO:tasks.workunit.client.1.smithi078.stdout:3/108: write d16/f17 [1025733,38454] 0 2024-04-26T04:52:46.325 INFO:tasks.workunit.client.1.smithi078.stdout:8/187: fdatasync d0/f23 0 2024-04-26T04:52:46.326 INFO:tasks.workunit.client.1.smithi078.stdout:9/101: dwrite f1 [0,4194304] 0 2024-04-26T04:52:46.326 INFO:tasks.workunit.client.1.smithi078.stdout:9/102: readlink - no filename 2024-04-26T04:52:46.328 INFO:tasks.workunit.client.1.smithi078.stdout:1/201: mknod d7/d39/db/d37/c43 0 2024-04-26T04:52:46.329 INFO:tasks.workunit.client.1.smithi078.stdout:5/102: dread f3 [0,4194304] 0 2024-04-26T04:52:46.331 INFO:tasks.workunit.client.1.smithi078.stdout:5/103: dread - d7/d11/f1f zero size 2024-04-26T04:52:46.332 INFO:tasks.workunit.client.1.smithi078.stdout:2/104: dread d8/fa [0,4194304] 0 2024-04-26T04:52:46.333 INFO:tasks.workunit.client.1.smithi078.stdout:9/103: dread f1 [0,4194304] 0 2024-04-26T04:52:46.334 INFO:tasks.workunit.client.1.smithi078.stdout:9/104: write f8 [2438649,23440] 0 2024-04-26T04:52:46.338 INFO:tasks.workunit.client.1.smithi078.stdout:4/168: rmdir d0/d2f/d48 0 2024-04-26T04:52:46.339 INFO:tasks.workunit.client.1.smithi078.stdout:4/169: fdatasync d0/d25/d20/f2b 0 2024-04-26T04:52:46.339 INFO:tasks.workunit.client.1.smithi078.stdout:0/94: fdatasync d2/f7 0 2024-04-26T04:52:46.340 INFO:tasks.workunit.client.1.smithi078.stdout:0/95: stat d2/d9/d18 0 2024-04-26T04:52:46.341 INFO:tasks.workunit.client.1.smithi078.stdout:0/96: write d2/f12 [267425,11595] 0 2024-04-26T04:52:46.343 INFO:tasks.workunit.client.1.smithi078.stdout:8/188: mknod d0/d19/c41 0 2024-04-26T04:52:46.347 INFO:tasks.workunit.client.1.smithi078.stdout:5/104: creat d7/da/f24 x:0 0 0 2024-04-26T04:52:46.350 INFO:tasks.workunit.client.1.smithi078.stdout:3/109: dwrite d16/f22 [0,4194304] 0 2024-04-26T04:52:46.354 INFO:tasks.workunit.client.1.smithi078.stdout:3/110: fsync d16/f25 0 2024-04-26T04:52:46.355 INFO:tasks.workunit.client.1.smithi078.stdout:0/97: dwrite d2/f6 [0,4194304] 0 2024-04-26T04:52:46.356 INFO:tasks.workunit.client.1.smithi078.stdout:7/154: rename d9/da/d11/d13/d1a/l27 to d9/da/d11/d13/l36 0 2024-04-26T04:52:46.356 INFO:tasks.workunit.client.1.smithi078.stdout:0/98: readlink - no filename 2024-04-26T04:52:46.357 INFO:tasks.workunit.client.1.smithi078.stdout:2/105: mknod d8/d11/d14/d15/d1a/c21 0 2024-04-26T04:52:46.360 INFO:tasks.workunit.client.1.smithi078.stdout:8/189: dwrite d0/f10 [0,4194304] 0 2024-04-26T04:52:46.360 INFO:tasks.workunit.client.1.smithi078.stdout:7/155: read - d9/da/d11/f2f zero size 2024-04-26T04:52:46.360 INFO:tasks.workunit.client.1.smithi078.stdout:1/202: dwrite d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:46.361 INFO:tasks.workunit.client.1.smithi078.stdout:4/170: creat d0/d25/d20/d31/d47/f49 x:0 0 0 2024-04-26T04:52:46.361 INFO:tasks.workunit.client.1.smithi078.stdout:0/99: truncate d2/f15 630100 0 2024-04-26T04:52:46.364 INFO:tasks.workunit.client.1.smithi078.stdout:6/121: getdents d2/d1a/d5/d11 0 2024-04-26T04:52:46.365 INFO:tasks.workunit.client.1.smithi078.stdout:7/156: write f6 [1638809,76844] 0 2024-04-26T04:52:46.370 INFO:tasks.workunit.client.1.smithi078.stdout:5/105: dwrite d7/da/f24 [0,4194304] 0 2024-04-26T04:52:46.370 INFO:tasks.workunit.client.1.smithi078.stdout:7/157: chown d9/lf 0 1 2024-04-26T04:52:46.372 INFO:tasks.workunit.client.1.smithi078.stdout:9/105: rename dc/dd/fe to dc/df/f1e 0 2024-04-26T04:52:46.375 INFO:tasks.workunit.client.1.smithi078.stdout:8/190: dread d0/f1a [4194304,4194304] 0 2024-04-26T04:52:46.375 INFO:tasks.workunit.client.1.smithi078.stdout:9/106: dread - dc/df/f19 zero size 2024-04-26T04:52:46.376 INFO:tasks.workunit.client.1.smithi078.stdout:2/106: dwrite d8/f1f [0,4194304] 0 2024-04-26T04:52:46.381 INFO:tasks.workunit.client.1.smithi078.stdout:7/158: dread f6 [0,4194304] 0 2024-04-26T04:52:46.386 INFO:tasks.workunit.client.1.smithi078.stdout:0/100: creat d2/d9/d18/f1c x:0 0 0 2024-04-26T04:52:46.387 INFO:tasks.workunit.client.1.smithi078.stdout:0/101: chown d2/f7 671414 1 2024-04-26T04:52:46.387 INFO:tasks.workunit.client.1.smithi078.stdout:9/107: dread f5 [0,4194304] 0 2024-04-26T04:52:46.389 INFO:tasks.workunit.client.1.smithi078.stdout:6/122: creat d2/d1a/d5/d11/f21 x:0 0 0 2024-04-26T04:52:46.389 INFO:tasks.workunit.client.1.smithi078.stdout:8/191: dwrite d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:46.389 INFO:tasks.workunit.client.1.smithi078.stdout:9/108: write dc/dd/f13 [706241,37181] 0 2024-04-26T04:52:46.402 INFO:tasks.workunit.client.1.smithi078.stdout:1/203: symlink d7/d39/d17/l44 0 2024-04-26T04:52:46.405 INFO:tasks.workunit.client.1.smithi078.stdout:2/107: creat d8/d11/f22 x:0 0 0 2024-04-26T04:52:46.408 INFO:tasks.workunit.client.1.smithi078.stdout:4/171: rmdir d0/d25/d20 39 2024-04-26T04:52:46.410 INFO:tasks.workunit.client.1.smithi078.stdout:1/204: write d7/d39/db/f34 [1586,70694] 0 2024-04-26T04:52:46.416 INFO:tasks.workunit.client.1.smithi078.stdout:8/192: write d0/f1a [9285241,111525] 0 2024-04-26T04:52:46.417 INFO:tasks.workunit.client.1.smithi078.stdout:6/123: creat d2/d1a/d5/da/f22 x:0 0 0 2024-04-26T04:52:46.417 INFO:tasks.workunit.client.1.smithi078.stdout:8/193: rename d0/d19/d25/d36 to d0/d19/d25/d36/d42 22 2024-04-26T04:52:46.423 INFO:tasks.workunit.client.1.smithi078.stdout:6/124: dwrite d2/d1a/fd [0,4194304] 0 2024-04-26T04:52:46.427 INFO:tasks.workunit.client.1.smithi078.stdout:2/108: creat d8/d11/d14/f23 x:0 0 0 2024-04-26T04:52:46.428 INFO:tasks.workunit.client.1.smithi078.stdout:6/125: dwrite d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:46.429 INFO:tasks.workunit.client.1.smithi078.stdout:7/159: symlink d9/da/d11/l37 0 2024-04-26T04:52:46.430 INFO:tasks.workunit.client.1.smithi078.stdout:7/160: fdatasync d9/da/d20/f31 0 2024-04-26T04:52:46.434 INFO:tasks.workunit.client.1.smithi078.stdout:3/111: getdents d16 0 2024-04-26T04:52:46.435 INFO:tasks.workunit.client.1.smithi078.stdout:3/112: write d16/d26/f2c [1340011,84166] 0 2024-04-26T04:52:46.437 INFO:tasks.workunit.client.1.smithi078.stdout:1/205: creat d7/d39/dd/d20/f45 x:0 0 0 2024-04-26T04:52:46.442 INFO:tasks.workunit.client.1.smithi078.stdout:9/109: unlink f5 0 2024-04-26T04:52:46.442 INFO:tasks.workunit.client.1.smithi078.stdout:9/110: chown dc/dd/f13 9526215 1 2024-04-26T04:52:46.443 INFO:tasks.workunit.client.1.smithi078.stdout:9/111: chown dc/df/c12 3145 1 2024-04-26T04:52:46.446 INFO:tasks.workunit.client.1.smithi078.stdout:2/109: rmdir d8/d11/d14/d15 39 2024-04-26T04:52:46.447 INFO:tasks.workunit.client.1.smithi078.stdout:6/126: mkdir d2/de/d23 0 2024-04-26T04:52:46.449 INFO:tasks.workunit.client.1.smithi078.stdout:7/161: rename d9/lf to d9/da/d33/l38 0 2024-04-26T04:52:46.457 INFO:tasks.workunit.client.1.smithi078.stdout:5/106: truncate d7/f22 319480 0 2024-04-26T04:52:46.457 INFO:tasks.workunit.client.1.smithi078.stdout:3/113: mkdir d16/d26/d2b/d31 0 2024-04-26T04:52:46.457 INFO:tasks.workunit.client.1.smithi078.stdout:1/206: mkdir d7/d39/db/d37/d46 0 2024-04-26T04:52:46.458 INFO:tasks.workunit.client.1.smithi078.stdout:3/114: fsync f9 0 2024-04-26T04:52:46.458 INFO:tasks.workunit.client.1.smithi078.stdout:0/102: creat d2/f1d x:0 0 0 2024-04-26T04:52:46.458 INFO:tasks.workunit.client.1.smithi078.stdout:1/207: readlink d7/l8 0 2024-04-26T04:52:46.458 INFO:tasks.workunit.client.1.smithi078.stdout:3/115: fdatasync d16/f17 0 2024-04-26T04:52:46.459 INFO:tasks.workunit.client.1.smithi078.stdout:9/112: symlink dc/df/d1a/l1f 0 2024-04-26T04:52:46.460 INFO:tasks.workunit.client.1.smithi078.stdout:3/116: write ff [1972408,78970] 0 2024-04-26T04:52:46.461 INFO:tasks.workunit.client.1.smithi078.stdout:9/113: dread - dc/df/f19 zero size 2024-04-26T04:52:46.462 INFO:tasks.workunit.client.1.smithi078.stdout:6/127: mknod d2/d1a/d5/d11/c24 0 2024-04-26T04:52:46.463 INFO:tasks.workunit.client.1.smithi078.stdout:3/117: write f7 [626236,96914] 0 2024-04-26T04:52:46.466 INFO:tasks.workunit.client.1.smithi078.stdout:3/118: write fc [1442119,63515] 0 2024-04-26T04:52:46.469 INFO:tasks.workunit.client.1.smithi078.stdout:5/107: dwrite d7/f16 [0,4194304] 0 2024-04-26T04:52:46.470 INFO:tasks.workunit.client.1.smithi078.stdout:4/172: dwrite d0/de/f1a [0,4194304] 0 2024-04-26T04:52:46.470 INFO:tasks.workunit.client.1.smithi078.stdout:7/162: truncate d9/da/d11/d13/d1a/f1c 1344641 0 2024-04-26T04:52:46.470 INFO:tasks.workunit.client.1.smithi078.stdout:3/119: write d16/d26/f29 [297558,57248] 0 2024-04-26T04:52:46.476 INFO:tasks.workunit.client.1.smithi078.stdout:5/108: dread d7/f16 [0,4194304] 0 2024-04-26T04:52:46.482 INFO:tasks.workunit.client.1.smithi078.stdout:5/109: dread d7/d11/f1d [0,4194304] 0 2024-04-26T04:52:46.483 INFO:tasks.workunit.client.1.smithi078.stdout:5/110: write f3 [3750833,29096] 0 2024-04-26T04:52:46.523 INFO:tasks.workunit.client.1.smithi078.stdout:8/194: sync 2024-04-26T04:52:46.528 INFO:tasks.workunit.client.1.smithi078.stdout:8/195: dwrite d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:46.529 INFO:tasks.workunit.client.1.smithi078.stdout:8/196: chown d0/d21/f29 294 1 2024-04-26T04:52:46.557 INFO:tasks.workunit.client.1.smithi078.stdout:1/208: mkdir d7/d39/db/d47 0 2024-04-26T04:52:46.557 INFO:tasks.workunit.client.1.smithi078.stdout:2/110: creat d8/d11/d14/d15/f24 x:0 0 0 2024-04-26T04:52:46.558 INFO:tasks.workunit.client.1.smithi078.stdout:1/209: write d7/d39/dd/d20/f40 [751807,85176] 0 2024-04-26T04:52:46.560 INFO:tasks.workunit.client.1.smithi078.stdout:9/114: rmdir dc/dd 39 2024-04-26T04:52:46.563 INFO:tasks.workunit.client.1.smithi078.stdout:3/120: creat d16/d26/f32 x:0 0 0 2024-04-26T04:52:46.563 INFO:tasks.workunit.client.1.smithi078.stdout:4/173: symlink d0/d1d/l4a 0 2024-04-26T04:52:46.564 INFO:tasks.workunit.client.1.smithi078.stdout:7/163: rename d9/da/d11/l1f to d9/d16/l39 0 2024-04-26T04:52:46.565 INFO:tasks.workunit.client.1.smithi078.stdout:4/174: dread d0/d1d/f36 [0,4194304] 0 2024-04-26T04:52:46.569 INFO:tasks.workunit.client.1.smithi078.stdout:8/197: rmdir d0/d3 39 2024-04-26T04:52:46.570 INFO:tasks.workunit.client.1.smithi078.stdout:0/103: mkdir d2/d1e 0 2024-04-26T04:52:46.571 INFO:tasks.workunit.client.1.smithi078.stdout:0/104: fdatasync d2/d9/f16 0 2024-04-26T04:52:46.572 INFO:tasks.workunit.client.1.smithi078.stdout:2/111: stat d8/lc 0 2024-04-26T04:52:46.572 INFO:tasks.workunit.client.1.smithi078.stdout:0/105: write d2/f7 [2749908,37425] 0 2024-04-26T04:52:46.573 INFO:tasks.workunit.client.1.smithi078.stdout:9/115: rmdir dc/df/d1a 39 2024-04-26T04:52:46.573 INFO:tasks.workunit.client.1.smithi078.stdout:2/112: write d8/fb [6512669,116052] 0 2024-04-26T04:52:46.574 INFO:tasks.workunit.client.1.smithi078.stdout:6/128: symlink d2/de/d23/l25 0 2024-04-26T04:52:46.576 INFO:tasks.workunit.client.1.smithi078.stdout:1/210: dread f1 [0,4194304] 0 2024-04-26T04:52:46.577 INFO:tasks.workunit.client.1.smithi078.stdout:1/211: dread - d7/d2e/f33 zero size 2024-04-26T04:52:46.579 INFO:tasks.workunit.client.1.smithi078.stdout:4/175: write d0/d25/d20/f2a [1524345,38282] 0 2024-04-26T04:52:46.581 INFO:tasks.workunit.client.1.smithi078.stdout:0/106: fsync d2/ff 0 2024-04-26T04:52:46.582 INFO:tasks.workunit.client.1.smithi078.stdout:2/113: creat d8/f25 x:0 0 0 2024-04-26T04:52:46.583 INFO:tasks.workunit.client.1.smithi078.stdout:0/107: write d2/f12 [1175358,42748] 0 2024-04-26T04:52:46.584 INFO:tasks.workunit.client.1.smithi078.stdout:9/116: unlink dc/df/f19 0 2024-04-26T04:52:46.584 INFO:tasks.workunit.client.1.smithi078.stdout:3/121: mkdir d16/d26/d2b/d31/d33 0 2024-04-26T04:52:46.586 INFO:tasks.workunit.client.1.smithi078.stdout:0/108: chown d2/f10 705 1 2024-04-26T04:52:46.589 INFO:tasks.workunit.client.1.smithi078.stdout:0/109: truncate d2/ff 65342 0 2024-04-26T04:52:46.590 INFO:tasks.workunit.client.1.smithi078.stdout:0/110: stat d2/ff 0 2024-04-26T04:52:46.590 INFO:tasks.workunit.client.1.smithi078.stdout:8/198: dread d0/f1a [0,4194304] 0 2024-04-26T04:52:46.590 INFO:tasks.workunit.client.1.smithi078.stdout:4/176: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:52:46.596 INFO:tasks.workunit.client.1.smithi078.stdout:4/177: dwrite d0/d25/d20/f15 [4194304,4194304] 0 2024-04-26T04:52:46.608 INFO:tasks.workunit.client.1.smithi078.stdout:2/114: unlink d8/d11/d14/f23 0 2024-04-26T04:52:46.609 INFO:tasks.workunit.client.1.smithi078.stdout:2/115: chown l5 1550 1 2024-04-26T04:52:46.611 INFO:tasks.workunit.client.1.smithi078.stdout:5/111: rmdir d7/de 0 2024-04-26T04:52:46.613 INFO:tasks.workunit.client.1.smithi078.stdout:2/116: dread d8/f1f [0,4194304] 0 2024-04-26T04:52:46.615 INFO:tasks.workunit.client.1.smithi078.stdout:3/122: creat d16/d26/d2b/d31/f34 x:0 0 0 2024-04-26T04:52:46.617 INFO:tasks.workunit.client.1.smithi078.stdout:8/199: mknod d0/d19/d25/c43 0 2024-04-26T04:52:46.618 INFO:tasks.workunit.client.1.smithi078.stdout:8/200: write d0/d21/f29 [4979710,34092] 0 2024-04-26T04:52:46.619 INFO:tasks.workunit.client.1.smithi078.stdout:4/178: chown d0/f5 1 1 2024-04-26T04:52:46.619 INFO:tasks.workunit.client.1.smithi078.stdout:3/123: dread d16/d26/f2c [0,4194304] 0 2024-04-26T04:52:46.619 INFO:tasks.workunit.client.1.smithi078.stdout:3/124: fsync f7 0 2024-04-26T04:52:46.620 INFO:tasks.workunit.client.1.smithi078.stdout:6/129: rename d2/d1a/d5/d11/c24 to d2/d1a/c26 0 2024-04-26T04:52:46.621 INFO:tasks.workunit.client.1.smithi078.stdout:6/130: write d2/f1e [732892,1147] 0 2024-04-26T04:52:46.623 INFO:tasks.workunit.client.1.smithi078.stdout:9/117: creat dc/df/d1a/f20 x:0 0 0 2024-04-26T04:52:46.623 INFO:tasks.workunit.client.1.smithi078.stdout:3/125: dread d16/f21 [0,4194304] 0 2024-04-26T04:52:46.624 INFO:tasks.workunit.client.1.smithi078.stdout:5/112: unlink l2 0 2024-04-26T04:52:46.624 INFO:tasks.workunit.client.1.smithi078.stdout:9/118: chown dc/df/c12 0 1 2024-04-26T04:52:46.626 INFO:tasks.workunit.client.1.smithi078.stdout:2/117: mknod d8/c26 0 2024-04-26T04:52:46.627 INFO:tasks.workunit.client.1.smithi078.stdout:0/111: mknod d2/d1e/c1f 0 2024-04-26T04:52:46.629 INFO:tasks.workunit.client.1.smithi078.stdout:6/131: dwrite d2/d1a/f6 [4194304,4194304] 0 2024-04-26T04:52:46.631 INFO:tasks.workunit.client.1.smithi078.stdout:8/201: unlink d0/f33 0 2024-04-26T04:52:46.635 INFO:tasks.workunit.client.1.smithi078.stdout:3/126: mkdir d16/d26/d35 0 2024-04-26T04:52:46.636 INFO:tasks.workunit.client.1.smithi078.stdout:0/112: dwrite d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:52:46.639 INFO:tasks.workunit.client.1.smithi078.stdout:1/212: getdents d7/d39/db 0 2024-04-26T04:52:46.639 INFO:tasks.workunit.client.1.smithi078.stdout:9/119: fsync dc/dd/f13 0 2024-04-26T04:52:46.639 INFO:tasks.workunit.client.1.smithi078.stdout:5/113: mknod d7/d11/c25 0 2024-04-26T04:52:46.639 INFO:tasks.workunit.client.1.smithi078.stdout:2/118: read d8/f1f [1825913,8703] 0 2024-04-26T04:52:46.640 INFO:tasks.workunit.client.1.smithi078.stdout:2/119: write d8/d11/d14/f1c [3434602,55411] 0 2024-04-26T04:52:46.641 INFO:tasks.workunit.client.1.smithi078.stdout:6/132: mkdir d2/de/d27 0 2024-04-26T04:52:46.646 INFO:tasks.workunit.client.1.smithi078.stdout:4/179: link d0/d25/d20/f23 d0/d2f/f4b 0 2024-04-26T04:52:46.647 INFO:tasks.workunit.client.1.smithi078.stdout:2/120: dwrite d8/fb [8388608,4194304] 0 2024-04-26T04:52:46.648 INFO:tasks.workunit.client.1.smithi078.stdout:2/121: readlink d8/lc 0 2024-04-26T04:52:46.648 INFO:tasks.workunit.client.1.smithi078.stdout:2/122: readlink d8/l10 0 2024-04-26T04:52:46.649 INFO:tasks.workunit.client.1.smithi078.stdout:0/113: creat d2/d9/f20 x:0 0 0 2024-04-26T04:52:46.651 INFO:tasks.workunit.client.1.smithi078.stdout:0/114: write d2/ff [593941,17810] 0 2024-04-26T04:52:46.652 INFO:tasks.workunit.client.1.smithi078.stdout:5/114: dread d7/da/f23 [0,4194304] 0 2024-04-26T04:52:46.653 INFO:tasks.workunit.client.1.smithi078.stdout:0/115: chown d2/f15 3605771 1 2024-04-26T04:52:46.653 INFO:tasks.workunit.client.1.smithi078.stdout:9/120: symlink dc/dd/l21 0 2024-04-26T04:52:46.656 INFO:tasks.workunit.client.1.smithi078.stdout:1/213: dread d7/fc [0,4194304] 0 2024-04-26T04:52:46.657 INFO:tasks.workunit.client.1.smithi078.stdout:7/164: write d9/da/f15 [4883265,83019] 0 2024-04-26T04:52:46.658 INFO:tasks.workunit.client.1.smithi078.stdout:8/202: rename d0/d3 to d0/d21/d35/d44 0 2024-04-26T04:52:46.659 INFO:tasks.workunit.client.1.smithi078.stdout:4/180: mkdir d0/d25/d20/d31/d33/d4c 0 2024-04-26T04:52:46.662 INFO:tasks.workunit.client.1.smithi078.stdout:3/127: link d16/f17 d16/f36 0 2024-04-26T04:52:46.662 INFO:tasks.workunit.client.1.smithi078.stdout:3/128: stat f9 0 2024-04-26T04:52:46.663 INFO:tasks.workunit.client.1.smithi078.stdout:3/129: write fc [172982,94894] 0 2024-04-26T04:52:46.664 INFO:tasks.workunit.client.1.smithi078.stdout:3/130: chown f6 1 1 2024-04-26T04:52:46.667 INFO:tasks.workunit.client.1.smithi078.stdout:9/121: mkdir dc/df/d1a/d22 0 2024-04-26T04:52:46.668 INFO:tasks.workunit.client.1.smithi078.stdout:6/133: link d2/f10 d2/d1a/d5/d11/f28 0 2024-04-26T04:52:46.669 INFO:tasks.workunit.client.1.smithi078.stdout:3/131: dread fc [0,4194304] 0 2024-04-26T04:52:46.669 INFO:tasks.workunit.client.1.smithi078.stdout:6/134: chown d2/d1a/f6 1490466 1 2024-04-26T04:52:46.669 INFO:tasks.workunit.client.1.smithi078.stdout:3/132: fsync fd 0 2024-04-26T04:52:46.670 INFO:tasks.workunit.client.1.smithi078.stdout:3/133: chown d16/l27 519 1 2024-04-26T04:52:46.670 INFO:tasks.workunit.client.1.smithi078.stdout:3/134: readlink d16/l27 0 2024-04-26T04:52:46.670 INFO:tasks.workunit.client.1.smithi078.stdout:1/214: rename d7/d39/db/c16 to d7/d1e/c48 0 2024-04-26T04:52:46.670 INFO:tasks.workunit.client.1.smithi078.stdout:2/123: rmdir d8/d11/d14 39 2024-04-26T04:52:46.671 INFO:tasks.workunit.client.1.smithi078.stdout:3/135: chown f12 28 1 2024-04-26T04:52:46.672 INFO:tasks.workunit.client.1.smithi078.stdout:4/181: write d0/d2f/f4b [181131,34391] 0 2024-04-26T04:52:46.677 INFO:tasks.workunit.client.1.smithi078.stdout:5/115: link d7/d11/f1f d7/d18/f26 0 2024-04-26T04:52:46.677 INFO:tasks.workunit.client.1.smithi078.stdout:5/116: read - d7/d18/f1a zero size 2024-04-26T04:52:46.677 INFO:tasks.workunit.client.1.smithi078.stdout:9/122: mknod dc/df/d1a/c23 0 2024-04-26T04:52:46.678 INFO:tasks.workunit.client.1.smithi078.stdout:9/123: dread - dc/df/f1d zero size 2024-04-26T04:52:46.680 INFO:tasks.workunit.client.1.smithi078.stdout:6/135: symlink d2/de/l29 0 2024-04-26T04:52:46.680 INFO:tasks.workunit.client.1.smithi078.stdout:6/136: readlink d2/de/d23/l25 0 2024-04-26T04:52:46.682 INFO:tasks.workunit.client.1.smithi078.stdout:1/215: rename d7/d1e/d35 to d7/d39/db/d49 0 2024-04-26T04:52:46.683 INFO:tasks.workunit.client.1.smithi078.stdout:3/136: creat d16/d26/f37 x:0 0 0 2024-04-26T04:52:46.683 INFO:tasks.workunit.client.1.smithi078.stdout:1/216: write d7/d39/db/f34 [938893,69332] 0 2024-04-26T04:52:46.687 INFO:tasks.workunit.client.1.smithi078.stdout:5/117: creat d7/d18/f27 x:0 0 0 2024-04-26T04:52:46.688 INFO:tasks.workunit.client.1.smithi078.stdout:5/118: dread - d7/d18/f1a zero size 2024-04-26T04:52:46.688 INFO:tasks.workunit.client.1.smithi078.stdout:7/165: creat d9/da/d11/f3a x:0 0 0 2024-04-26T04:52:46.688 INFO:tasks.workunit.client.1.smithi078.stdout:7/166: readlink d9/d16/l1d 0 2024-04-26T04:52:46.689 INFO:tasks.workunit.client.1.smithi078.stdout:9/124: unlink dc/dd/l21 0 2024-04-26T04:52:46.691 INFO:tasks.workunit.client.1.smithi078.stdout:6/137: rename d2/de/d23 to d2/d1a/d5/d2a 0 2024-04-26T04:52:46.693 INFO:tasks.workunit.client.1.smithi078.stdout:6/138: readlink d2/d1a/d5/da/l20 0 2024-04-26T04:52:46.694 INFO:tasks.workunit.client.1.smithi078.stdout:5/119: dwrite d7/d18/f1a [0,4194304] 0 2024-04-26T04:52:46.695 INFO:tasks.workunit.client.1.smithi078.stdout:5/120: chown d7/f22 202952937 1 2024-04-26T04:52:46.695 INFO:tasks.workunit.client.1.smithi078.stdout:0/116: rename d2/f8 to d2/f21 0 2024-04-26T04:52:46.696 INFO:tasks.workunit.client.1.smithi078.stdout:0/117: fdatasync d2/d9/f1a 0 2024-04-26T04:52:46.700 INFO:tasks.workunit.client.1.smithi078.stdout:1/217: rename d7/d39/db/d37/d46 to d7/d39/db/d47/d4a 0 2024-04-26T04:52:46.704 INFO:tasks.workunit.client.1.smithi078.stdout:6/139: creat d2/de/f2b x:0 0 0 2024-04-26T04:52:46.705 INFO:tasks.workunit.client.1.smithi078.stdout:5/121: read d7/f16 [4154224,130718] 0 2024-04-26T04:52:46.706 INFO:tasks.workunit.client.1.smithi078.stdout:7/167: link d9/f14 d9/da/d11/d13/d1a/f3b 0 2024-04-26T04:52:46.706 INFO:tasks.workunit.client.1.smithi078.stdout:7/168: read - d9/da/d11/f2f zero size 2024-04-26T04:52:46.707 INFO:tasks.workunit.client.1.smithi078.stdout:0/118: creat d2/d9/d18/f22 x:0 0 0 2024-04-26T04:52:46.707 INFO:tasks.workunit.client.1.smithi078.stdout:2/124: link d8/d11/d14/f19 d8/f27 0 2024-04-26T04:52:46.707 INFO:tasks.workunit.client.1.smithi078.stdout:0/119: stat d2/d9/d18/f22 0 2024-04-26T04:52:46.710 INFO:tasks.workunit.client.1.smithi078.stdout:9/125: rename dc/f14 to dc/dd/f24 0 2024-04-26T04:52:46.711 INFO:tasks.workunit.client.1.smithi078.stdout:3/137: link d16/l24 d16/d26/d35/l38 0 2024-04-26T04:52:46.713 INFO:tasks.workunit.client.1.smithi078.stdout:5/122: fsync d7/f22 0 2024-04-26T04:52:46.718 INFO:tasks.workunit.client.1.smithi078.stdout:9/126: unlink dc/df/d1a/c23 0 2024-04-26T04:52:46.719 INFO:tasks.workunit.client.1.smithi078.stdout:1/218: mknod d7/d39/c4b 0 2024-04-26T04:52:46.719 INFO:tasks.workunit.client.1.smithi078.stdout:8/203: truncate d0/d19/f1c 1046067 0 2024-04-26T04:52:46.721 INFO:tasks.workunit.client.1.smithi078.stdout:6/140: link d2/d1a/fd d2/d1a/d5/da/f2c 0 2024-04-26T04:52:46.723 INFO:tasks.workunit.client.1.smithi078.stdout:7/169: getdents d9/da/d11/d13/d1a/d2d 0 2024-04-26T04:52:46.725 INFO:tasks.workunit.client.1.smithi078.stdout:0/120: link d2/d9/d18/fd d2/d1e/f23 0 2024-04-26T04:52:46.726 INFO:tasks.workunit.client.1.smithi078.stdout:4/182: truncate d0/d25/d20/f15 1762263 0 2024-04-26T04:52:46.728 INFO:tasks.workunit.client.1.smithi078.stdout:2/125: rmdir d8 39 2024-04-26T04:52:46.728 INFO:tasks.workunit.client.1.smithi078.stdout:8/204: write d0/f1a [1782733,18948] 0 2024-04-26T04:52:46.729 INFO:tasks.workunit.client.1.smithi078.stdout:1/219: truncate d7/d39/db/f15 5016056 0 2024-04-26T04:52:46.730 INFO:tasks.workunit.client.1.smithi078.stdout:4/183: write d0/de/f34 [831416,15140] 0 2024-04-26T04:52:46.730 INFO:tasks.workunit.client.1.smithi078.stdout:6/141: unlink d2/d1a/d5/d11/f28 0 2024-04-26T04:52:46.732 INFO:tasks.workunit.client.1.smithi078.stdout:7/170: dwrite d9/d16/f22 [0,4194304] 0 2024-04-26T04:52:46.732 INFO:tasks.workunit.client.1.smithi078.stdout:8/205: chown d0/d19/d25/d31 1519 1 2024-04-26T04:52:46.739 INFO:tasks.workunit.client.1.smithi078.stdout:7/171: fsync d9/da/d11/f3a 0 2024-04-26T04:52:46.739 INFO:tasks.workunit.client.1.smithi078.stdout:5/123: creat d7/f28 x:0 0 0 2024-04-26T04:52:46.739 INFO:tasks.workunit.client.1.smithi078.stdout:2/126: write d8/f9 [6640565,62616] 0 2024-04-26T04:52:46.739 INFO:tasks.workunit.client.1.smithi078.stdout:6/142: mknod d2/de/c2d 0 2024-04-26T04:52:46.740 INFO:tasks.workunit.client.1.smithi078.stdout:8/206: mknod d0/d21/d3c/c45 0 2024-04-26T04:52:46.742 INFO:tasks.workunit.client.1.smithi078.stdout:3/138: getdents d16/d26/d2e 0 2024-04-26T04:52:46.743 INFO:tasks.workunit.client.1.smithi078.stdout:2/127: chown d8/d11/d14/d15/d1a/c21 850699542 1 2024-04-26T04:52:46.743 INFO:tasks.workunit.client.1.smithi078.stdout:1/220: dread d7/d39/dd/d20/f40 [0,4194304] 0 2024-04-26T04:52:46.743 INFO:tasks.workunit.client.1.smithi078.stdout:0/121: dread d2/f6 [0,4194304] 0 2024-04-26T04:52:46.744 INFO:tasks.workunit.client.1.smithi078.stdout:6/143: symlink d2/d1a/d5/l2e 0 2024-04-26T04:52:46.744 INFO:tasks.workunit.client.1.smithi078.stdout:5/124: mkdir d7/d29 0 2024-04-26T04:52:46.745 INFO:tasks.workunit.client.1.smithi078.stdout:8/207: rmdir d0/d19/d25/d32 39 2024-04-26T04:52:46.746 INFO:tasks.workunit.client.1.smithi078.stdout:5/125: chown d7/da/lf 27011552 1 2024-04-26T04:52:46.748 INFO:tasks.workunit.client.1.smithi078.stdout:3/139: rename f7 to d16/f39 0 2024-04-26T04:52:46.750 INFO:tasks.workunit.client.1.smithi078.stdout:0/122: unlink d2/f19 0 2024-04-26T04:52:46.750 INFO:tasks.workunit.client.1.smithi078.stdout:3/140: rename d16/d26 to d16/d26/d2b/d3a 22 2024-04-26T04:52:46.750 INFO:tasks.workunit.client.1.smithi078.stdout:0/123: truncate f1 4267398 0 2024-04-26T04:52:46.751 INFO:tasks.workunit.client.1.smithi078.stdout:0/124: readlink - no filename 2024-04-26T04:52:46.751 INFO:tasks.workunit.client.1.smithi078.stdout:8/208: dread d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:46.751 INFO:tasks.workunit.client.1.smithi078.stdout:2/128: creat d8/d11/d18/f28 x:0 0 0 2024-04-26T04:52:46.751 INFO:tasks.workunit.client.1.smithi078.stdout:9/127: truncate dc/df/f1e 735180 0 2024-04-26T04:52:46.752 INFO:tasks.workunit.client.1.smithi078.stdout:1/221: fdatasync d7/d39/f12 0 2024-04-26T04:52:46.753 INFO:tasks.workunit.client.1.smithi078.stdout:1/222: chown d7/l8 138 1 2024-04-26T04:52:46.754 INFO:tasks.workunit.client.1.smithi078.stdout:2/129: chown d8/d11/d18/f28 3444 1 2024-04-26T04:52:46.754 INFO:tasks.workunit.client.1.smithi078.stdout:7/172: link d9/f10 d9/da/d11/f3c 0 2024-04-26T04:52:46.756 INFO:tasks.workunit.client.1.smithi078.stdout:8/209: dread d0/d21/f29 [4194304,4194304] 0 2024-04-26T04:52:46.756 INFO:tasks.workunit.client.1.smithi078.stdout:5/126: unlink d7/f22 0 2024-04-26T04:52:46.757 INFO:tasks.workunit.client.1.smithi078.stdout:5/127: stat d7/d18/d1c 0 2024-04-26T04:52:46.759 INFO:tasks.workunit.client.1.smithi078.stdout:3/141: write d16/f39 [1366410,106749] 0 2024-04-26T04:52:46.763 INFO:tasks.workunit.client.1.smithi078.stdout:2/130: symlink d8/d11/d18/l29 0 2024-04-26T04:52:46.763 INFO:tasks.workunit.client.1.smithi078.stdout:4/184: dread d0/d25/d20/f15 [0,4194304] 0 2024-04-26T04:52:46.764 INFO:tasks.workunit.client.1.smithi078.stdout:7/173: unlink d9/da/d11/f3c 0 2024-04-26T04:52:46.764 INFO:tasks.workunit.client.1.smithi078.stdout:7/174: chown d9 54 1 2024-04-26T04:52:46.764 INFO:tasks.workunit.client.1.smithi078.stdout:4/185: stat d0/d25/d20/d31/d33/f40 0 2024-04-26T04:52:46.765 INFO:tasks.workunit.client.1.smithi078.stdout:7/175: chown d9/da/d20/c32 67711 1 2024-04-26T04:52:46.766 INFO:tasks.workunit.client.1.smithi078.stdout:0/125: rmdir d2 39 2024-04-26T04:52:46.766 INFO:tasks.workunit.client.1.smithi078.stdout:7/176: write f6 [2805423,83192] 0 2024-04-26T04:52:46.768 INFO:tasks.workunit.client.1.smithi078.stdout:3/142: fsync d16/f21 0 2024-04-26T04:52:46.768 INFO:tasks.workunit.client.1.smithi078.stdout:7/177: dread - d9/da/d11/f3a zero size 2024-04-26T04:52:46.770 INFO:tasks.workunit.client.1.smithi078.stdout:6/144: dread d2/f14 [0,4194304] 0 2024-04-26T04:52:46.771 INFO:tasks.workunit.client.1.smithi078.stdout:8/210: getdents d0/d19/d25/d31 0 2024-04-26T04:52:46.771 INFO:tasks.workunit.client.1.smithi078.stdout:8/211: stat d0/d21/f24 0 2024-04-26T04:52:46.772 INFO:tasks.workunit.client.1.smithi078.stdout:5/128: mknod d7/d29/c2a 0 2024-04-26T04:52:46.773 INFO:tasks.workunit.client.1.smithi078.stdout:8/212: read d0/d21/d35/d44/f26 [250697,86532] 0 2024-04-26T04:52:46.773 INFO:tasks.workunit.client.1.smithi078.stdout:4/186: creat d0/d25/d20/d31/d33/f4d x:0 0 0 2024-04-26T04:52:46.774 INFO:tasks.workunit.client.1.smithi078.stdout:6/145: dread d2/f1e [0,4194304] 0 2024-04-26T04:52:46.777 INFO:tasks.workunit.client.1.smithi078.stdout:1/223: rename d7/d39/d1f to d7/d39/d4c 0 2024-04-26T04:52:46.777 INFO:tasks.workunit.client.1.smithi078.stdout:1/224: chown d7/d39/db 26954 1 2024-04-26T04:52:46.778 INFO:tasks.workunit.client.1.smithi078.stdout:7/178: rmdir d9/da/d11/d13/d1a 39 2024-04-26T04:52:46.784 INFO:tasks.workunit.client.1.smithi078.stdout:5/129: rename d7/d11/f1d to d7/d11/f2b 0 2024-04-26T04:52:46.785 INFO:tasks.workunit.client.1.smithi078.stdout:5/130: chown f5 7 1 2024-04-26T04:52:46.786 INFO:tasks.workunit.client.1.smithi078.stdout:6/146: write d2/f14 [819297,90963] 0 2024-04-26T04:52:46.786 INFO:tasks.workunit.client.1.smithi078.stdout:8/213: mknod d0/d21/d3c/c46 0 2024-04-26T04:52:46.786 INFO:tasks.workunit.client.1.smithi078.stdout:4/187: rmdir d0/de/d22/d26/d2e 39 2024-04-26T04:52:46.786 INFO:tasks.workunit.client.1.smithi078.stdout:7/179: dwrite d9/da/d20/f21 [0,4194304] 0 2024-04-26T04:52:46.787 INFO:tasks.workunit.client.1.smithi078.stdout:4/188: readlink d0/l11 0 2024-04-26T04:52:46.788 INFO:tasks.workunit.client.1.smithi078.stdout:8/214: write d0/f1a [845001,82589] 0 2024-04-26T04:52:46.789 INFO:tasks.workunit.client.1.smithi078.stdout:0/126: mknod d2/d9/d18/c24 0 2024-04-26T04:52:46.792 INFO:tasks.workunit.client.1.smithi078.stdout:2/131: creat d8/d11/f2a x:0 0 0 2024-04-26T04:52:46.795 INFO:tasks.workunit.client.1.smithi078.stdout:7/180: rmdir d9 39 2024-04-26T04:52:46.796 INFO:tasks.workunit.client.1.smithi078.stdout:7/181: read f5 [4323862,46540] 0 2024-04-26T04:52:46.796 INFO:tasks.workunit.client.1.smithi078.stdout:6/147: creat d2/d1a/d5/da/f2f x:0 0 0 2024-04-26T04:52:46.797 INFO:tasks.workunit.client.1.smithi078.stdout:4/189: rmdir d0/d25/d20 39 2024-04-26T04:52:46.797 INFO:tasks.workunit.client.1.smithi078.stdout:2/132: dread d8/f1f [0,4194304] 0 2024-04-26T04:52:46.797 INFO:tasks.workunit.client.1.smithi078.stdout:6/148: fsync d2/d1a/d5/da/f22 0 2024-04-26T04:52:46.799 INFO:tasks.workunit.client.1.smithi078.stdout:3/143: rmdir d16/d26/d2b/d31/d33 0 2024-04-26T04:52:46.799 INFO:tasks.workunit.client.1.smithi078.stdout:2/133: truncate d8/d11/f22 862638 0 2024-04-26T04:52:46.799 INFO:tasks.workunit.client.1.smithi078.stdout:2/134: readlink d8/lc 0 2024-04-26T04:52:46.800 INFO:tasks.workunit.client.1.smithi078.stdout:1/225: mknod d7/d39/c4d 0 2024-04-26T04:52:46.803 INFO:tasks.workunit.client.1.smithi078.stdout:5/131: write d7/f16 [4394534,100370] 0 2024-04-26T04:52:46.804 INFO:tasks.workunit.client.1.smithi078.stdout:5/132: readlink d7/ld 0 2024-04-26T04:52:46.805 INFO:tasks.workunit.client.1.smithi078.stdout:2/135: dread d8/d11/d14/d15/f1b [0,4194304] 0 2024-04-26T04:52:46.805 INFO:tasks.workunit.client.1.smithi078.stdout:6/149: mknod d2/c30 0 2024-04-26T04:52:46.809 INFO:tasks.workunit.client.1.smithi078.stdout:1/226: creat d7/d39/d4c/f4e x:0 0 0 2024-04-26T04:52:46.811 INFO:tasks.workunit.client.1.smithi078.stdout:5/133: creat d7/d11/f2c x:0 0 0 2024-04-26T04:52:46.811 INFO:tasks.workunit.client.1.smithi078.stdout:7/182: fsync d9/da/d11/d13/d1a/f1e 0 2024-04-26T04:52:46.813 INFO:tasks.workunit.client.1.smithi078.stdout:3/144: mknod d16/d26/d35/c3b 0 2024-04-26T04:52:46.813 INFO:tasks.workunit.client.1.smithi078.stdout:6/150: creat d2/d1a/d5/f31 x:0 0 0 2024-04-26T04:52:46.813 INFO:tasks.workunit.client.1.smithi078.stdout:7/183: stat d9 0 2024-04-26T04:52:46.814 INFO:tasks.workunit.client.1.smithi078.stdout:4/190: dwrite d0/d25/d20/d31/f3f [0,4194304] 0 2024-04-26T04:52:46.814 INFO:tasks.workunit.client.1.smithi078.stdout:8/215: write d0/d19/f1c [1156676,35740] 0 2024-04-26T04:52:46.815 INFO:tasks.workunit.client.1.smithi078.stdout:6/151: chown d2/d1a/d5/da/cc 0 1 2024-04-26T04:52:46.820 INFO:tasks.workunit.client.1.smithi078.stdout:0/127: link d2/d9/d18/f22 d2/f25 0 2024-04-26T04:52:46.821 INFO:tasks.workunit.client.1.smithi078.stdout:1/227: creat d7/d39/db/d47/f4f x:0 0 0 2024-04-26T04:52:46.823 INFO:tasks.workunit.client.1.smithi078.stdout:3/145: dwrite d16/f22 [0,4194304] 0 2024-04-26T04:52:46.826 INFO:tasks.workunit.client.1.smithi078.stdout:1/228: truncate d7/d39/dd/d20/f45 506754 0 2024-04-26T04:52:46.827 INFO:tasks.workunit.client.1.smithi078.stdout:8/216: rename d0/d19/d25/d31 to d0/d19/d25/d32/d47 0 2024-04-26T04:52:46.827 INFO:tasks.workunit.client.1.smithi078.stdout:5/134: chown d7/da/f10 735328143 1 2024-04-26T04:52:46.828 INFO:tasks.workunit.client.1.smithi078.stdout:3/146: rmdir d16/d26/d2e 39 2024-04-26T04:52:46.829 INFO:tasks.workunit.client.1.smithi078.stdout:2/136: dread f3 [0,4194304] 0 2024-04-26T04:52:46.834 INFO:tasks.workunit.client.1.smithi078.stdout:8/217: fdatasync d0/d21/d35/d44/f26 0 2024-04-26T04:52:46.836 INFO:tasks.workunit.client.1.smithi078.stdout:3/147: chown d16/d26/d2e 9509539 1 2024-04-26T04:52:46.836 INFO:tasks.workunit.client.1.smithi078.stdout:6/152: write d2/f1e [1294905,9923] 0 2024-04-26T04:52:46.837 INFO:tasks.workunit.client.1.smithi078.stdout:2/137: creat d8/f2b x:0 0 0 2024-04-26T04:52:46.837 INFO:tasks.workunit.client.1.smithi078.stdout:0/128: write d2/d9/d18/fd [940362,39874] 0 2024-04-26T04:52:46.838 INFO:tasks.workunit.client.1.smithi078.stdout:1/229: truncate d7/d39/d4c/f26 1447820 0 2024-04-26T04:52:46.839 INFO:tasks.workunit.client.1.smithi078.stdout:3/148: dread - d16/f1e zero size 2024-04-26T04:52:46.841 INFO:tasks.workunit.client.1.smithi078.stdout:8/218: creat d0/d19/d25/d36/f48 x:0 0 0 2024-04-26T04:52:46.841 INFO:tasks.workunit.client.1.smithi078.stdout:6/153: creat d2/d1a/d5/f32 x:0 0 0 2024-04-26T04:52:46.842 INFO:tasks.workunit.client.1.smithi078.stdout:3/149: mkdir d16/d3c 0 2024-04-26T04:52:46.843 INFO:tasks.workunit.client.1.smithi078.stdout:2/138: dread d8/d11/f22 [0,4194304] 0 2024-04-26T04:52:46.845 INFO:tasks.workunit.client.1.smithi078.stdout:4/191: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:52:46.847 INFO:tasks.workunit.client.1.smithi078.stdout:0/129: getdents d2/d9 0 2024-04-26T04:52:46.848 INFO:tasks.workunit.client.1.smithi078.stdout:1/230: rename d7/d39/db/f34 to d7/d39/dd/d20/f50 0 2024-04-26T04:52:46.848 INFO:tasks.workunit.client.1.smithi078.stdout:2/139: chown d8/d11/d18/l29 22915 1 2024-04-26T04:52:46.851 INFO:tasks.workunit.client.1.smithi078.stdout:5/135: dread d7/f8 [4194304,4194304] 0 2024-04-26T04:52:46.852 INFO:tasks.workunit.client.1.smithi078.stdout:3/150: creat d16/d26/d2b/d31/f3d x:0 0 0 2024-04-26T04:52:46.853 INFO:tasks.workunit.client.1.smithi078.stdout:4/192: symlink d0/d25/d20/d31/d33/l4e 0 2024-04-26T04:52:46.856 INFO:tasks.workunit.client.1.smithi078.stdout:3/151: read d16/f17 [35,130193] 0 2024-04-26T04:52:46.857 INFO:tasks.workunit.client.1.smithi078.stdout:2/140: rename d8/d11/d18/c20 to d8/d11/d18/c2c 0 2024-04-26T04:52:46.859 INFO:tasks.workunit.client.1.smithi078.stdout:5/136: creat d7/d18/f2d x:0 0 0 2024-04-26T04:52:46.860 INFO:tasks.workunit.client.1.smithi078.stdout:4/193: creat d0/d25/d20/f4f x:0 0 0 2024-04-26T04:52:46.863 INFO:tasks.workunit.client.1.smithi078.stdout:1/231: link d7/d39/dd/d20/f50 d7/d39/d4c/f51 0 2024-04-26T04:52:46.864 INFO:tasks.workunit.client.1.smithi078.stdout:0/130: dwrite d2/f6 [4194304,4194304] 0 2024-04-26T04:52:46.866 INFO:tasks.workunit.client.1.smithi078.stdout:8/219: dread d0/f5 [0,4194304] 0 2024-04-26T04:52:46.867 INFO:tasks.workunit.client.1.smithi078.stdout:2/141: symlink d8/d11/l2d 0 2024-04-26T04:52:46.869 INFO:tasks.workunit.client.1.smithi078.stdout:2/142: fdatasync d8/f25 0 2024-04-26T04:52:46.869 INFO:tasks.workunit.client.1.smithi078.stdout:4/194: creat d0/de/d22/d26/d2e/f50 x:0 0 0 2024-04-26T04:52:46.872 INFO:tasks.workunit.client.1.smithi078.stdout:5/137: dwrite d7/d18/f27 [0,4194304] 0 2024-04-26T04:52:46.874 INFO:tasks.workunit.client.1.smithi078.stdout:3/152: getdents d16 0 2024-04-26T04:52:46.875 INFO:tasks.workunit.client.1.smithi078.stdout:0/131: creat d2/f26 x:0 0 0 2024-04-26T04:52:46.877 INFO:tasks.workunit.client.1.smithi078.stdout:2/143: rmdir d8/d11/d14/d15 39 2024-04-26T04:52:46.877 INFO:tasks.workunit.client.1.smithi078.stdout:0/132: chown d2/d1e/f23 454793 1 2024-04-26T04:52:46.877 INFO:tasks.workunit.client.1.smithi078.stdout:6/154: dread d2/f10 [0,4194304] 0 2024-04-26T04:52:46.879 INFO:tasks.workunit.client.1.smithi078.stdout:1/232: dread d7/d39/d4c/f51 [0,4194304] 0 2024-04-26T04:52:46.879 INFO:tasks.workunit.client.1.smithi078.stdout:3/153: symlink d16/l3e 0 2024-04-26T04:52:46.880 INFO:tasks.workunit.client.1.smithi078.stdout:5/138: chown d7/c9 40 1 2024-04-26T04:52:46.881 INFO:tasks.workunit.client.1.smithi078.stdout:3/154: dread - d16/f1c zero size 2024-04-26T04:52:46.882 INFO:tasks.workunit.client.1.smithi078.stdout:2/144: truncate d8/d11/f2a 956485 0 2024-04-26T04:52:46.882 INFO:tasks.workunit.client.1.smithi078.stdout:6/155: chown d2/d1a/d5/da/f2c 1097226 1 2024-04-26T04:52:46.883 INFO:tasks.workunit.client.1.smithi078.stdout:3/155: chown ff 1711450 1 2024-04-26T04:52:46.883 INFO:tasks.workunit.client.1.smithi078.stdout:5/139: creat d7/d18/f2e x:0 0 0 2024-04-26T04:52:46.885 INFO:tasks.workunit.client.1.smithi078.stdout:3/156: truncate d16/f25 286067 0 2024-04-26T04:52:46.885 INFO:tasks.workunit.client.1.smithi078.stdout:6/156: dread - d2/d1a/d5/d11/f1c zero size 2024-04-26T04:52:46.886 INFO:tasks.workunit.client.1.smithi078.stdout:2/145: dread d8/d11/f2a [0,4194304] 0 2024-04-26T04:52:46.886 INFO:tasks.workunit.client.1.smithi078.stdout:0/133: truncate d2/f21 5201745 0 2024-04-26T04:52:46.886 INFO:tasks.workunit.client.1.smithi078.stdout:5/140: write d7/d18/f2e [813221,128368] 0 2024-04-26T04:52:46.889 INFO:tasks.workunit.client.1.smithi078.stdout:0/134: truncate d2/d9/d18/fd 1923038 0 2024-04-26T04:52:46.890 INFO:tasks.workunit.client.1.smithi078.stdout:2/146: read d8/d11/d14/d15/f1b [1142641,54723] 0 2024-04-26T04:52:46.891 INFO:tasks.workunit.client.1.smithi078.stdout:3/157: read f12 [574243,92694] 0 2024-04-26T04:52:46.893 INFO:tasks.workunit.client.1.smithi078.stdout:5/141: fdatasync d7/d18/f27 0 2024-04-26T04:52:46.893 INFO:tasks.workunit.client.1.smithi078.stdout:6/157: dread d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:46.896 INFO:tasks.workunit.client.1.smithi078.stdout:8/220: dread d0/f23 [0,4194304] 0 2024-04-26T04:52:46.896 INFO:tasks.workunit.client.1.smithi078.stdout:0/135: creat d2/f27 x:0 0 0 2024-04-26T04:52:46.897 INFO:tasks.workunit.client.1.smithi078.stdout:5/142: readlink d7/da/d1e/l20 0 2024-04-26T04:52:46.899 INFO:tasks.workunit.client.1.smithi078.stdout:2/147: creat d8/d11/d14/f2e x:0 0 0 2024-04-26T04:52:46.901 INFO:tasks.workunit.client.1.smithi078.stdout:0/136: link d2/ff d2/d9/f28 0 2024-04-26T04:52:46.901 INFO:tasks.workunit.client.1.smithi078.stdout:5/143: mkdir d7/d21/d2f 0 2024-04-26T04:52:46.904 INFO:tasks.workunit.client.1.smithi078.stdout:4/195: dread d0/fc [0,4194304] 0 2024-04-26T04:52:46.907 INFO:tasks.workunit.client.1.smithi078.stdout:8/221: dread d0/d19/f1c [0,4194304] 0 2024-04-26T04:52:46.907 INFO:tasks.workunit.client.1.smithi078.stdout:0/137: creat d2/f29 x:0 0 0 2024-04-26T04:52:46.909 INFO:tasks.workunit.client.1.smithi078.stdout:6/158: dwrite d2/f10 [0,4194304] 0 2024-04-26T04:52:46.909 INFO:tasks.workunit.client.1.smithi078.stdout:3/158: dwrite d16/f21 [0,4194304] 0 2024-04-26T04:52:46.911 INFO:tasks.workunit.client.1.smithi078.stdout:4/196: write d0/f37 [3905811,43124] 0 2024-04-26T04:52:46.913 INFO:tasks.workunit.client.1.smithi078.stdout:0/138: unlink d2/c1b 0 2024-04-26T04:52:46.914 INFO:tasks.workunit.client.1.smithi078.stdout:6/159: write d2/f10 [1917862,53803] 0 2024-04-26T04:52:46.914 INFO:tasks.workunit.client.1.smithi078.stdout:3/159: creat d16/f3f x:0 0 0 2024-04-26T04:52:46.918 INFO:tasks.workunit.client.1.smithi078.stdout:6/160: creat d2/de/f33 x:0 0 0 2024-04-26T04:52:46.920 INFO:tasks.workunit.client.1.smithi078.stdout:0/139: creat d2/d1e/f2a x:0 0 0 2024-04-26T04:52:46.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/144: dwrite d7/d11/f1f [0,4194304] 0 2024-04-26T04:52:46.922 INFO:tasks.workunit.client.1.smithi078.stdout:3/160: truncate fd 1497702 0 2024-04-26T04:52:46.922 INFO:tasks.workunit.client.1.smithi078.stdout:6/161: chown d2/d1a/d5/f32 6 1 2024-04-26T04:52:46.924 INFO:tasks.workunit.client.1.smithi078.stdout:5/145: fsync f3 0 2024-04-26T04:52:46.925 INFO:tasks.workunit.client.1.smithi078.stdout:8/222: dwrite d0/f10 [0,4194304] 0 2024-04-26T04:52:46.929 INFO:tasks.workunit.client.1.smithi078.stdout:5/146: chown d7/da/lf 105639668 1 2024-04-26T04:52:46.929 INFO:tasks.workunit.client.1.smithi078.stdout:6/162: truncate d2/d1a/d5/da/f2f 472777 0 2024-04-26T04:52:46.934 INFO:tasks.workunit.client.1.smithi078.stdout:9/128: sync 2024-04-26T04:52:46.936 INFO:tasks.workunit.client.1.smithi078.stdout:6/163: creat d2/de/d27/f34 x:0 0 0 2024-04-26T04:52:46.942 INFO:tasks.workunit.client.1.smithi078.stdout:3/161: dwrite d16/f20 [0,4194304] 0 2024-04-26T04:52:46.942 INFO:tasks.workunit.client.1.smithi078.stdout:0/140: dwrite d2/f15 [0,4194304] 0 2024-04-26T04:52:46.943 INFO:tasks.workunit.client.1.smithi078.stdout:8/223: dwrite d0/d19/d25/f2b [0,4194304] 0 2024-04-26T04:52:46.945 INFO:tasks.workunit.client.1.smithi078.stdout:9/129: truncate dc/df/d1a/f1c 1008074 0 2024-04-26T04:52:46.945 INFO:tasks.workunit.client.1.smithi078.stdout:6/164: write d2/d1a/d5/da/f1b [340799,128605] 0 2024-04-26T04:52:46.947 INFO:tasks.workunit.client.1.smithi078.stdout:9/130: truncate dc/df/f1d 857455 0 2024-04-26T04:52:46.952 INFO:tasks.workunit.client.1.smithi078.stdout:6/165: unlink d2/d1a/d5/da/f1b 0 2024-04-26T04:52:46.953 INFO:tasks.workunit.client.1.smithi078.stdout:9/131: read - dc/df/d1a/f20 zero size 2024-04-26T04:52:46.955 INFO:tasks.workunit.client.1.smithi078.stdout:9/132: readlink dc/df/d1a/l1f 0 2024-04-26T04:52:46.955 INFO:tasks.workunit.client.1.smithi078.stdout:3/162: dwrite d16/f3f [0,4194304] 0 2024-04-26T04:52:46.961 INFO:tasks.workunit.client.1.smithi078.stdout:9/133: dwrite dc/df/d1a/f1c [0,4194304] 0 2024-04-26T04:52:47.011 INFO:tasks.workunit.client.1.smithi078.stdout:8/224: link d0/c11 d0/c49 0 2024-04-26T04:52:47.014 INFO:tasks.workunit.client.1.smithi078.stdout:6/166: mkdir d2/d1a/d5/da/d35 0 2024-04-26T04:52:47.015 INFO:tasks.workunit.client.1.smithi078.stdout:6/167: chown d2/c30 22734929 1 2024-04-26T04:52:47.016 INFO:tasks.workunit.client.1.smithi078.stdout:6/168: write d2/d1a/d5/da/f22 [296922,119544] 0 2024-04-26T04:52:47.017 INFO:tasks.workunit.client.1.smithi078.stdout:3/163: creat d16/d26/d2b/f40 x:0 0 0 2024-04-26T04:52:47.017 INFO:tasks.workunit.client.1.smithi078.stdout:5/147: write d7/d11/f2b [14459,72430] 0 2024-04-26T04:52:47.018 INFO:tasks.workunit.client.1.smithi078.stdout:6/169: dread - d2/d1a/d5/f31 zero size 2024-04-26T04:52:47.019 INFO:tasks.workunit.client.1.smithi078.stdout:9/134: mknod dc/df/d17/c25 0 2024-04-26T04:52:47.019 INFO:tasks.workunit.client.1.smithi078.stdout:5/148: chown d7/d18/f2e 344778 1 2024-04-26T04:52:47.020 INFO:tasks.workunit.client.1.smithi078.stdout:0/141: write d2/f25 [57479,20493] 0 2024-04-26T04:52:47.020 INFO:tasks.workunit.client.1.smithi078.stdout:6/170: write d2/d1a/d5/da/f22 [1233900,3765] 0 2024-04-26T04:52:47.020 INFO:tasks.workunit.client.1.smithi078.stdout:5/149: rename d7/d21 to d7/d21/d30 22 2024-04-26T04:52:47.022 INFO:tasks.workunit.client.1.smithi078.stdout:3/164: unlink d16/l3e 0 2024-04-26T04:52:47.026 INFO:tasks.workunit.client.1.smithi078.stdout:0/142: symlink d2/d1e/l2b 0 2024-04-26T04:52:47.027 INFO:tasks.workunit.client.1.smithi078.stdout:0/143: chown d2/d1e/c1f 28967 1 2024-04-26T04:52:47.027 INFO:tasks.workunit.client.1.smithi078.stdout:0/144: dread - d2/d9/f20 zero size 2024-04-26T04:52:47.029 INFO:tasks.workunit.client.1.smithi078.stdout:3/165: rmdir d16/d26 39 2024-04-26T04:52:47.031 INFO:tasks.workunit.client.1.smithi078.stdout:6/171: truncate d2/d1a/fd 109482 0 2024-04-26T04:52:47.033 INFO:tasks.workunit.client.1.smithi078.stdout:0/145: mknod d2/d9/c2c 0 2024-04-26T04:52:47.034 INFO:tasks.workunit.client.1.smithi078.stdout:3/166: unlink d16/l27 0 2024-04-26T04:52:47.036 INFO:tasks.workunit.client.1.smithi078.stdout:8/225: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:52:47.037 INFO:tasks.workunit.client.1.smithi078.stdout:9/135: rmdir dc/d18 0 2024-04-26T04:52:47.040 INFO:tasks.workunit.client.1.smithi078.stdout:0/146: dwrite d2/f15 [0,4194304] 0 2024-04-26T04:52:47.042 INFO:tasks.workunit.client.1.smithi078.stdout:3/167: unlink l5 0 2024-04-26T04:52:47.043 INFO:tasks.workunit.client.1.smithi078.stdout:9/136: read dc/dd/f13 [20580,102036] 0 2024-04-26T04:52:47.046 INFO:tasks.workunit.client.1.smithi078.stdout:0/147: creat d2/f2d x:0 0 0 2024-04-26T04:52:47.048 INFO:tasks.workunit.client.1.smithi078.stdout:9/137: fdatasync dc/dd/f24 0 2024-04-26T04:52:47.048 INFO:tasks.workunit.client.1.smithi078.stdout:8/226: dwrite d0/d19/f1c [0,4194304] 0 2024-04-26T04:52:47.053 INFO:tasks.workunit.client.1.smithi078.stdout:3/168: symlink d16/d26/l41 0 2024-04-26T04:52:47.053 INFO:tasks.workunit.client.1.smithi078.stdout:6/172: truncate d2/f1e 824307 0 2024-04-26T04:52:47.054 INFO:tasks.workunit.client.1.smithi078.stdout:3/169: write d16/d26/d2b/d31/f34 [242495,56703] 0 2024-04-26T04:52:47.061 INFO:tasks.workunit.client.1.smithi078.stdout:6/173: mkdir d2/d1a/d5/da/d36 0 2024-04-26T04:52:47.061 INFO:tasks.workunit.client.1.smithi078.stdout:6/174: dread - d2/d1a/d5/f31 zero size 2024-04-26T04:52:47.063 INFO:tasks.workunit.client.1.smithi078.stdout:0/148: rename d2/d9/f16 to d2/f2e 0 2024-04-26T04:52:47.063 INFO:tasks.workunit.client.1.smithi078.stdout:0/149: chown f1 788972 1 2024-04-26T04:52:47.064 INFO:tasks.workunit.client.1.smithi078.stdout:3/170: creat d16/d26/d2e/f42 x:0 0 0 2024-04-26T04:52:47.064 INFO:tasks.workunit.client.1.smithi078.stdout:7/184: sync 2024-04-26T04:52:47.065 INFO:tasks.workunit.client.1.smithi078.stdout:9/138: creat dc/f26 x:0 0 0 2024-04-26T04:52:47.066 INFO:tasks.workunit.client.1.smithi078.stdout:8/227: link d0/c17 d0/d21/d35/c4a 0 2024-04-26T04:52:47.067 INFO:tasks.workunit.client.1.smithi078.stdout:9/139: truncate dc/df/d1a/f20 458617 0 2024-04-26T04:52:47.067 INFO:tasks.workunit.client.1.smithi078.stdout:0/150: dread d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:52:47.068 INFO:tasks.workunit.client.1.smithi078.stdout:6/175: write d2/d1a/f6 [2788720,77389] 0 2024-04-26T04:52:47.069 INFO:tasks.workunit.client.1.smithi078.stdout:7/185: getdents d9/da/d11/d13/d1a/d2d 0 2024-04-26T04:52:47.069 INFO:tasks.workunit.client.1.smithi078.stdout:9/140: dread dc/df/d1a/f20 [0,4194304] 0 2024-04-26T04:52:47.069 INFO:tasks.workunit.client.1.smithi078.stdout:6/176: chown d2/f10 35 1 2024-04-26T04:52:47.071 INFO:tasks.workunit.client.1.smithi078.stdout:8/228: rmdir d0/d19/d25/d32/d47 0 2024-04-26T04:52:47.072 INFO:tasks.workunit.client.1.smithi078.stdout:9/141: rmdir dc/dd 39 2024-04-26T04:52:47.074 INFO:tasks.workunit.client.1.smithi078.stdout:7/186: creat d9/da/d11/d13/d1a/d2d/f3d x:0 0 0 2024-04-26T04:52:47.074 INFO:tasks.workunit.client.1.smithi078.stdout:8/229: rmdir d0/d19/d25/d32 39 2024-04-26T04:52:47.076 INFO:tasks.workunit.client.1.smithi078.stdout:9/142: fdatasync dc/df/f1d 0 2024-04-26T04:52:47.077 INFO:tasks.workunit.client.1.smithi078.stdout:8/230: write d0/f23 [4969405,85596] 0 2024-04-26T04:52:47.080 INFO:tasks.workunit.client.1.smithi078.stdout:0/151: dwrite d2/f2d [0,4194304] 0 2024-04-26T04:52:47.082 INFO:tasks.workunit.client.1.smithi078.stdout:6/177: dwrite d2/de/f12 [0,4194304] 0 2024-04-26T04:52:47.088 INFO:tasks.workunit.client.1.smithi078.stdout:0/152: truncate d2/f2d 4858070 0 2024-04-26T04:52:47.089 INFO:tasks.workunit.client.1.smithi078.stdout:6/178: fsync d2/d1a/d5/f8 0 2024-04-26T04:52:47.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/231: creat d0/d19/d25/d32/f4b x:0 0 0 2024-04-26T04:52:47.094 INFO:tasks.workunit.client.1.smithi078.stdout:6/179: write d2/de/d27/f34 [874979,124764] 0 2024-04-26T04:52:47.094 INFO:tasks.workunit.client.1.smithi078.stdout:9/143: dwrite dc/df/f1d [0,4194304] 0 2024-04-26T04:52:47.097 INFO:tasks.workunit.client.1.smithi078.stdout:7/187: dwrite f6 [4194304,4194304] 0 2024-04-26T04:52:47.100 INFO:tasks.workunit.client.1.smithi078.stdout:6/180: write d2/d1a/d5/f8 [4706662,42509] 0 2024-04-26T04:52:47.100 INFO:tasks.workunit.client.1.smithi078.stdout:9/144: fdatasync f8 0 2024-04-26T04:52:47.102 INFO:tasks.workunit.client.1.smithi078.stdout:6/181: dread - d2/f16 zero size 2024-04-26T04:52:47.104 INFO:tasks.workunit.client.1.smithi078.stdout:7/188: creat d9/da/d33/f3e x:0 0 0 2024-04-26T04:52:47.104 INFO:tasks.workunit.client.1.smithi078.stdout:8/232: dread d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:47.104 INFO:tasks.workunit.client.1.smithi078.stdout:0/153: dwrite d2/f15 [0,4194304] 0 2024-04-26T04:52:47.104 INFO:tasks.workunit.client.1.smithi078.stdout:6/182: stat d2/d1a/d5/l2e 0 2024-04-26T04:52:47.109 INFO:tasks.workunit.client.1.smithi078.stdout:6/183: chown d2/de/c19 1008295208 1 2024-04-26T04:52:47.112 INFO:tasks.workunit.client.1.smithi078.stdout:6/184: rename d2/f1d to d2/de/d27/f37 0 2024-04-26T04:52:47.117 INFO:tasks.workunit.client.1.smithi078.stdout:9/145: truncate dc/df/f1d 2064755 0 2024-04-26T04:52:47.120 INFO:tasks.workunit.client.1.smithi078.stdout:6/185: dwrite d2/de/f2b [0,4194304] 0 2024-04-26T04:52:47.122 INFO:tasks.workunit.client.1.smithi078.stdout:9/146: truncate dc/df/f1e 389036 0 2024-04-26T04:52:47.123 INFO:tasks.workunit.client.1.smithi078.stdout:0/154: dread d2/f3 [0,4194304] 0 2024-04-26T04:52:47.124 INFO:tasks.workunit.client.1.smithi078.stdout:6/186: mkdir d2/d1a/d38 0 2024-04-26T04:52:47.124 INFO:tasks.workunit.client.1.smithi078.stdout:0/155: dread - d2/d9/f17 zero size 2024-04-26T04:52:47.129 INFO:tasks.workunit.client.1.smithi078.stdout:6/187: dread d2/f14 [0,4194304] 0 2024-04-26T04:52:47.129 INFO:tasks.workunit.client.1.smithi078.stdout:0/156: creat d2/f2f x:0 0 0 2024-04-26T04:52:47.130 INFO:tasks.workunit.client.1.smithi078.stdout:0/157: chown d2/d9/c11 338191990 1 2024-04-26T04:52:47.131 INFO:tasks.workunit.client.1.smithi078.stdout:9/147: rename dc/dd to dc/df/d27 0 2024-04-26T04:52:47.131 INFO:tasks.workunit.client.1.smithi078.stdout:6/188: mknod d2/de/c39 0 2024-04-26T04:52:47.133 INFO:tasks.workunit.client.1.smithi078.stdout:6/189: chown d2/d1a/d5 251807 1 2024-04-26T04:52:47.134 INFO:tasks.workunit.client.1.smithi078.stdout:9/148: dread dc/df/d1a/f20 [0,4194304] 0 2024-04-26T04:52:47.135 INFO:tasks.workunit.client.1.smithi078.stdout:6/190: read d2/d1a/fd [84843,128887] 0 2024-04-26T04:52:47.138 INFO:tasks.workunit.client.1.smithi078.stdout:0/158: dwrite d2/d9/f1a [0,4194304] 0 2024-04-26T04:52:47.139 INFO:tasks.workunit.client.1.smithi078.stdout:6/191: mkdir d2/d1a/d5/d3a 0 2024-04-26T04:52:47.140 INFO:tasks.workunit.client.1.smithi078.stdout:0/159: read d2/d9/d18/fd [642808,16612] 0 2024-04-26T04:52:47.142 INFO:tasks.workunit.client.1.smithi078.stdout:6/192: mknod d2/c3b 0 2024-04-26T04:52:47.143 INFO:tasks.workunit.client.1.smithi078.stdout:6/193: stat d2/de/l29 0 2024-04-26T04:52:47.150 INFO:tasks.workunit.client.1.smithi078.stdout:0/160: dwrite d2/f4 [4194304,4194304] 0 2024-04-26T04:52:47.152 INFO:tasks.workunit.client.1.smithi078.stdout:9/149: write f8 [580241,4088] 0 2024-04-26T04:52:47.155 INFO:tasks.workunit.client.1.smithi078.stdout:6/194: dwrite d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:47.161 INFO:tasks.workunit.client.1.smithi078.stdout:9/150: creat dc/df/f28 x:0 0 0 2024-04-26T04:52:47.162 INFO:tasks.workunit.client.1.smithi078.stdout:0/161: dwrite d2/f29 [0,4194304] 0 2024-04-26T04:52:47.163 INFO:tasks.workunit.client.1.smithi078.stdout:6/195: mknod d2/d1a/c3c 0 2024-04-26T04:52:47.164 INFO:tasks.workunit.client.1.smithi078.stdout:6/196: chown d2/d1a/d38 0 1 2024-04-26T04:52:47.164 INFO:tasks.workunit.client.1.smithi078.stdout:0/162: truncate d2/d9/f28 793249 0 2024-04-26T04:52:47.164 INFO:tasks.workunit.client.1.smithi078.stdout:6/197: chown d2/de 0 1 2024-04-26T04:52:47.165 INFO:tasks.workunit.client.1.smithi078.stdout:6/198: chown d2/d1a/c1f 343027164 1 2024-04-26T04:52:47.168 INFO:tasks.workunit.client.1.smithi078.stdout:9/151: read f1 [1644240,31565] 0 2024-04-26T04:52:47.169 INFO:tasks.workunit.client.1.smithi078.stdout:0/163: rename d2/f2d to d2/d9/f30 0 2024-04-26T04:52:47.170 INFO:tasks.workunit.client.1.smithi078.stdout:0/164: fsync d2/d9/f17 0 2024-04-26T04:52:47.172 INFO:tasks.workunit.client.1.smithi078.stdout:9/152: fdatasync dc/df/f1d 0 2024-04-26T04:52:47.174 INFO:tasks.workunit.client.1.smithi078.stdout:9/153: readlink dc/df/d1a/l1f 0 2024-04-26T04:52:47.177 INFO:tasks.workunit.client.1.smithi078.stdout:0/165: dwrite d2/d9/d18/f22 [0,4194304] 0 2024-04-26T04:52:47.179 INFO:tasks.workunit.client.1.smithi078.stdout:0/166: mkdir d2/d9/d31 0 2024-04-26T04:52:47.180 INFO:tasks.workunit.client.1.smithi078.stdout:0/167: chown d2/f12 3 1 2024-04-26T04:52:47.181 INFO:tasks.workunit.client.1.smithi078.stdout:0/168: write d2/f12 [486593,104597] 0 2024-04-26T04:52:47.185 INFO:tasks.workunit.client.1.smithi078.stdout:0/169: symlink d2/l32 0 2024-04-26T04:52:47.187 INFO:tasks.workunit.client.1.smithi078.stdout:0/170: write d2/f14 [22633,81247] 0 2024-04-26T04:52:47.188 INFO:tasks.workunit.client.1.smithi078.stdout:0/171: symlink d2/d9/l33 0 2024-04-26T04:52:47.220 INFO:tasks.workunit.client.1.smithi078.stdout:1/233: sync 2024-04-26T04:52:47.221 INFO:tasks.workunit.client.1.smithi078.stdout:9/154: truncate f8 295586 0 2024-04-26T04:52:47.223 INFO:tasks.workunit.client.1.smithi078.stdout:6/199: dread d2/d1a/d5/d11/f15 [4194304,4194304] 0 2024-04-26T04:52:47.223 INFO:tasks.workunit.client.1.smithi078.stdout:9/155: creat dc/df/d1a/f29 x:0 0 0 2024-04-26T04:52:47.224 INFO:tasks.workunit.client.1.smithi078.stdout:6/200: readlink d2/d1a/d5/l2e 0 2024-04-26T04:52:47.224 INFO:tasks.workunit.client.1.smithi078.stdout:0/172: dread d2/f25 [0,4194304] 0 2024-04-26T04:52:47.227 INFO:tasks.workunit.client.1.smithi078.stdout:6/201: unlink d2/d1a/d5/da/f2f 0 2024-04-26T04:52:47.227 INFO:tasks.workunit.client.1.smithi078.stdout:0/173: mknod d2/d1e/c34 0 2024-04-26T04:52:47.229 INFO:tasks.workunit.client.1.smithi078.stdout:6/202: stat d2/d1a/c26 0 2024-04-26T04:52:47.230 INFO:tasks.workunit.client.1.smithi078.stdout:0/174: dread - d2/d9/f20 zero size 2024-04-26T04:52:47.231 INFO:tasks.workunit.client.1.smithi078.stdout:1/234: dread d7/d39/db/f15 [0,4194304] 0 2024-04-26T04:52:47.231 INFO:tasks.workunit.client.1.smithi078.stdout:9/156: dwrite dc/df/f1d [0,4194304] 0 2024-04-26T04:52:47.233 INFO:tasks.workunit.client.1.smithi078.stdout:0/175: write d2/f6 [8103931,114181] 0 2024-04-26T04:52:47.233 INFO:tasks.workunit.client.1.smithi078.stdout:1/235: readlink d7/d39/dd/d20/l29 0 2024-04-26T04:52:47.236 INFO:tasks.workunit.client.1.smithi078.stdout:1/236: symlink d7/d2e/l52 0 2024-04-26T04:52:47.237 INFO:tasks.workunit.client.1.smithi078.stdout:9/157: mkdir dc/df/d1a/d22/d2a 0 2024-04-26T04:52:47.238 INFO:tasks.workunit.client.1.smithi078.stdout:9/158: readlink dc/df/d1a/l1f 0 2024-04-26T04:52:47.242 INFO:tasks.workunit.client.1.smithi078.stdout:9/159: symlink dc/df/d1a/d22/d2a/l2b 0 2024-04-26T04:52:47.244 INFO:tasks.workunit.client.1.smithi078.stdout:0/176: dwrite d2/f29 [0,4194304] 0 2024-04-26T04:52:47.246 INFO:tasks.workunit.client.1.smithi078.stdout:1/237: dwrite d7/ff [0,4194304] 0 2024-04-26T04:52:47.249 INFO:tasks.workunit.client.1.smithi078.stdout:0/177: getdents d2/d9/d31 0 2024-04-26T04:52:47.252 INFO:tasks.workunit.client.1.smithi078.stdout:0/178: mknod d2/d9/c35 0 2024-04-26T04:52:47.257 INFO:tasks.workunit.client.1.smithi078.stdout:0/179: link d2/f21 d2/d9/d31/f36 0 2024-04-26T04:52:47.258 INFO:tasks.workunit.client.1.smithi078.stdout:0/180: stat d2/f2f 0 2024-04-26T04:52:47.266 INFO:tasks.workunit.client.1.smithi078.stdout:0/181: dwrite d2/f2e [0,4194304] 0 2024-04-26T04:52:47.266 INFO:tasks.workunit.client.1.smithi078.stdout:0/182: write d2/f7 [1355030,26193] 0 2024-04-26T04:52:47.270 INFO:tasks.workunit.client.1.smithi078.stdout:0/183: unlink d2/f2f 0 2024-04-26T04:52:47.270 INFO:tasks.workunit.client.1.smithi078.stdout:0/184: readlink d2/d1e/l2b 0 2024-04-26T04:52:47.274 INFO:tasks.workunit.client.1.smithi078.stdout:9/160: dread f1 [0,4194304] 0 2024-04-26T04:52:47.275 INFO:tasks.workunit.client.1.smithi078.stdout:1/238: dread f1 [0,4194304] 0 2024-04-26T04:52:47.276 INFO:tasks.workunit.client.1.smithi078.stdout:0/185: dread d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:52:47.277 INFO:tasks.workunit.client.1.smithi078.stdout:1/239: unlink d7/d39/d17/l23 0 2024-04-26T04:52:47.278 INFO:tasks.workunit.client.1.smithi078.stdout:1/240: stat d7/d39/db 0 2024-04-26T04:52:47.280 INFO:tasks.workunit.client.1.smithi078.stdout:1/241: write d7/d39/dd/f2d [3133080,125455] 0 2024-04-26T04:52:47.281 INFO:tasks.workunit.client.1.smithi078.stdout:0/186: dread d2/d9/f1a [0,4194304] 0 2024-04-26T04:52:47.282 INFO:tasks.workunit.client.1.smithi078.stdout:1/242: rmdir d7/d1e 39 2024-04-26T04:52:47.282 INFO:tasks.workunit.client.1.smithi078.stdout:1/243: chown d7/l8 109164492 1 2024-04-26T04:52:47.285 INFO:tasks.workunit.client.1.smithi078.stdout:1/244: write d7/d39/dd/d20/f50 [269433,102833] 0 2024-04-26T04:52:47.287 INFO:tasks.workunit.client.1.smithi078.stdout:1/245: mkdir d7/d39/dd/d20/d53 0 2024-04-26T04:52:47.287 INFO:tasks.workunit.client.1.smithi078.stdout:0/187: dread d2/d9/f28 [0,4194304] 0 2024-04-26T04:52:47.289 INFO:tasks.workunit.client.1.smithi078.stdout:1/246: fdatasync d7/d39/db/f15 0 2024-04-26T04:52:47.291 INFO:tasks.workunit.client.1.smithi078.stdout:0/188: rename d2/f7 to d2/f37 0 2024-04-26T04:52:47.293 INFO:tasks.workunit.client.1.smithi078.stdout:1/247: creat d7/d1e/f54 x:0 0 0 2024-04-26T04:52:47.293 INFO:tasks.workunit.client.1.smithi078.stdout:0/189: symlink d2/d9/l38 0 2024-04-26T04:52:47.295 INFO:tasks.workunit.client.1.smithi078.stdout:0/190: mkdir d2/d39 0 2024-04-26T04:52:47.296 INFO:tasks.workunit.client.1.smithi078.stdout:0/191: read - d2/d9/f17 zero size 2024-04-26T04:52:47.298 INFO:tasks.workunit.client.1.smithi078.stdout:1/248: dwrite d7/d39/f12 [8388608,4194304] 0 2024-04-26T04:52:47.298 INFO:tasks.workunit.client.1.smithi078.stdout:0/192: creat d2/f3a x:0 0 0 2024-04-26T04:52:47.299 INFO:tasks.workunit.client.1.smithi078.stdout:0/193: chown d2/f15 479731 1 2024-04-26T04:52:47.302 INFO:tasks.workunit.client.1.smithi078.stdout:1/249: rename d7/d39/d4c/f51 to d7/d39/dd/f55 0 2024-04-26T04:52:47.305 INFO:tasks.workunit.client.1.smithi078.stdout:1/250: fsync f3 0 2024-04-26T04:52:47.305 INFO:tasks.workunit.client.1.smithi078.stdout:0/194: dwrite d2/d9/d18/f1c [0,4194304] 0 2024-04-26T04:52:47.309 INFO:tasks.workunit.client.1.smithi078.stdout:0/195: creat d2/d9/d18/f3b x:0 0 0 2024-04-26T04:52:47.310 INFO:tasks.workunit.client.1.smithi078.stdout:1/251: symlink d7/d39/d4c/d3d/l56 0 2024-04-26T04:52:47.313 INFO:tasks.workunit.client.1.smithi078.stdout:1/252: unlink d7/d39/db/d47/f4f 0 2024-04-26T04:52:47.319 INFO:tasks.workunit.client.1.smithi078.stdout:1/253: read d7/d39/dd/f2d [462261,48940] 0 2024-04-26T04:52:47.321 INFO:tasks.workunit.client.1.smithi078.stdout:1/254: creat d7/f57 x:0 0 0 2024-04-26T04:52:47.322 INFO:tasks.workunit.client.1.smithi078.stdout:1/255: mkdir d7/d39/db/d37/d58 0 2024-04-26T04:52:47.323 INFO:tasks.workunit.client.1.smithi078.stdout:1/256: dread - d7/d2e/f33 zero size 2024-04-26T04:52:47.325 INFO:tasks.workunit.client.1.smithi078.stdout:1/257: symlink d7/d39/db/d47/l59 0 2024-04-26T04:52:47.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/258: write f3 [740945,33309] 0 2024-04-26T04:52:47.328 INFO:tasks.workunit.client.1.smithi078.stdout:1/259: creat d7/d39/db/f5a x:0 0 0 2024-04-26T04:52:47.331 INFO:tasks.workunit.client.1.smithi078.stdout:1/260: unlink d7/d39/d17/l25 0 2024-04-26T04:52:47.332 INFO:tasks.workunit.client.1.smithi078.stdout:1/261: write d7/ff [2729646,113143] 0 2024-04-26T04:52:47.335 INFO:tasks.workunit.client.1.smithi078.stdout:2/148: sync 2024-04-26T04:52:47.335 INFO:tasks.workunit.client.1.smithi078.stdout:4/197: sync 2024-04-26T04:52:47.336 INFO:tasks.workunit.client.1.smithi078.stdout:0/196: dread d2/f37 [0,4194304] 0 2024-04-26T04:52:47.354 INFO:tasks.workunit.client.1.smithi078.stdout:2/149: mknod d8/d11/d14/d15/c2f 0 2024-04-26T04:52:47.354 INFO:tasks.workunit.client.1.smithi078.stdout:0/197: chown d2/d9/d31/f36 0 1 2024-04-26T04:52:47.355 INFO:tasks.workunit.client.1.smithi078.stdout:2/150: symlink d8/l30 0 2024-04-26T04:52:47.356 INFO:tasks.workunit.client.1.smithi078.stdout:2/151: dread - d8/f25 zero size 2024-04-26T04:52:47.356 INFO:tasks.workunit.client.1.smithi078.stdout:0/198: symlink d2/d1e/l3c 0 2024-04-26T04:52:47.358 INFO:tasks.workunit.client.1.smithi078.stdout:2/152: creat d8/d11/f31 x:0 0 0 2024-04-26T04:52:47.361 INFO:tasks.workunit.client.1.smithi078.stdout:2/153: creat d8/d11/d14/d15/f32 x:0 0 0 2024-04-26T04:52:47.361 INFO:tasks.workunit.client.1.smithi078.stdout:0/199: dwrite d2/f15 [0,4194304] 0 2024-04-26T04:52:47.362 INFO:tasks.workunit.client.1.smithi078.stdout:0/200: read d2/d9/f28 [1977,35626] 0 2024-04-26T04:52:47.362 INFO:tasks.workunit.client.1.smithi078.stdout:2/154: write d8/d11/d14/d15/f24 [400351,125176] 0 2024-04-26T04:52:47.365 INFO:tasks.workunit.client.1.smithi078.stdout:0/201: mknod d2/c3d 0 2024-04-26T04:52:47.366 INFO:tasks.workunit.client.1.smithi078.stdout:2/155: symlink d8/d11/d14/l33 0 2024-04-26T04:52:47.368 INFO:tasks.workunit.client.1.smithi078.stdout:2/156: mkdir d8/d11/d14/d15/d1a/d34 0 2024-04-26T04:52:47.369 INFO:tasks.workunit.client.1.smithi078.stdout:2/157: chown d8/le 469 1 2024-04-26T04:52:47.372 INFO:tasks.workunit.client.1.smithi078.stdout:2/158: fdatasync d8/d11/d14/f2e 0 2024-04-26T04:52:47.372 INFO:tasks.workunit.client.1.smithi078.stdout:0/202: dwrite d2/f1d [0,4194304] 0 2024-04-26T04:52:47.376 INFO:tasks.workunit.client.1.smithi078.stdout:2/159: mknod d8/d11/d14/d15/d1a/c35 0 2024-04-26T04:52:47.382 INFO:tasks.workunit.client.1.smithi078.stdout:0/203: dwrite d2/d9/f30 [4194304,4194304] 0 2024-04-26T04:52:47.383 INFO:tasks.workunit.client.1.smithi078.stdout:0/204: truncate d2/d9/f20 343021 0 2024-04-26T04:52:47.386 INFO:tasks.workunit.client.1.smithi078.stdout:0/205: truncate d2/f21 3555082 0 2024-04-26T04:52:47.387 INFO:tasks.workunit.client.1.smithi078.stdout:0/206: truncate d2/d1e/f2a 383281 0 2024-04-26T04:52:47.387 INFO:tasks.workunit.client.1.smithi078.stdout:0/207: chown d2/d9/d18/c24 6 1 2024-04-26T04:52:47.389 INFO:tasks.workunit.client.1.smithi078.stdout:0/208: creat d2/d9/f3e x:0 0 0 2024-04-26T04:52:47.392 INFO:tasks.workunit.client.1.smithi078.stdout:0/209: link d2/f26 d2/f3f 0 2024-04-26T04:52:47.392 INFO:tasks.workunit.client.1.smithi078.stdout:0/210: truncate d2/d9/f30 8683217 0 2024-04-26T04:52:47.394 INFO:tasks.workunit.client.1.smithi078.stdout:0/211: creat d2/f40 x:0 0 0 2024-04-26T04:52:47.402 INFO:tasks.workunit.client.1.smithi078.stdout:2/160: write d8/d11/f2a [1846031,24936] 0 2024-04-26T04:52:47.403 INFO:tasks.workunit.client.1.smithi078.stdout:2/161: dread - d8/d11/f31 zero size 2024-04-26T04:52:47.406 INFO:tasks.workunit.client.1.smithi078.stdout:0/212: getdents d2 0 2024-04-26T04:52:47.407 INFO:tasks.workunit.client.1.smithi078.stdout:0/213: symlink d2/d9/d18/l41 0 2024-04-26T04:52:47.409 INFO:tasks.workunit.client.1.smithi078.stdout:0/214: creat d2/d39/f42 x:0 0 0 2024-04-26T04:52:47.414 INFO:tasks.workunit.client.1.smithi078.stdout:0/215: truncate d2/f2e 2794759 0 2024-04-26T04:52:47.415 INFO:tasks.workunit.client.1.smithi078.stdout:0/216: symlink d2/d39/l43 0 2024-04-26T04:52:47.415 INFO:tasks.workunit.client.1.smithi078.stdout:0/217: fsync d2/d1e/f2a 0 2024-04-26T04:52:47.437 INFO:tasks.workunit.client.1.smithi078.stdout:0/218: dwrite d2/ff [0,4194304] 0 2024-04-26T04:52:47.439 INFO:tasks.workunit.client.1.smithi078.stdout:5/150: sync 2024-04-26T04:52:47.440 INFO:tasks.workunit.client.1.smithi078.stdout:0/219: mknod d2/d39/c44 0 2024-04-26T04:52:47.442 INFO:tasks.workunit.client.1.smithi078.stdout:5/151: rename d7/da/lf to d7/d21/l31 0 2024-04-26T04:52:47.444 INFO:tasks.workunit.client.1.smithi078.stdout:5/152: mkdir d7/d18/d32 0 2024-04-26T04:52:47.445 INFO:tasks.workunit.client.1.smithi078.stdout:5/153: fsync d7/d11/f2b 0 2024-04-26T04:52:47.445 INFO:tasks.workunit.client.1.smithi078.stdout:0/220: dread d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:52:47.445 INFO:tasks.workunit.client.1.smithi078.stdout:0/221: dread - d2/f26 zero size 2024-04-26T04:52:47.447 INFO:tasks.workunit.client.1.smithi078.stdout:5/154: mknod d7/da/d1e/c33 0 2024-04-26T04:52:47.450 INFO:tasks.workunit.client.1.smithi078.stdout:5/155: rename d7/d11/f1f to d7/d18/f34 0 2024-04-26T04:52:47.488 INFO:tasks.workunit.client.1.smithi078.stdout:3/171: sync 2024-04-26T04:52:47.488 INFO:tasks.workunit.client.1.smithi078.stdout:3/172: write d16/f22 [2180417,23179] 0 2024-04-26T04:52:47.499 INFO:tasks.workunit.client.1.smithi078.stdout:3/173: dread f9 [0,4194304] 0 2024-04-26T04:52:47.529 INFO:tasks.workunit.client.1.smithi078.stdout:5/156: creat d7/f35 x:0 0 0 2024-04-26T04:52:47.530 INFO:tasks.workunit.client.1.smithi078.stdout:3/174: truncate d16/f20 903168 0 2024-04-26T04:52:47.530 INFO:tasks.workunit.client.1.smithi078.stdout:5/157: truncate d7/d18/f26 4509066 0 2024-04-26T04:52:47.532 INFO:tasks.workunit.client.1.smithi078.stdout:3/175: creat d16/d26/d2e/f43 x:0 0 0 2024-04-26T04:52:47.533 INFO:tasks.workunit.client.1.smithi078.stdout:0/222: dwrite d2/f21 [0,4194304] 0 2024-04-26T04:52:47.535 INFO:tasks.workunit.client.1.smithi078.stdout:0/223: write d2/f12 [1190774,86163] 0 2024-04-26T04:52:47.538 INFO:tasks.workunit.client.1.smithi078.stdout:1/262: sync 2024-04-26T04:52:47.538 INFO:tasks.workunit.client.1.smithi078.stdout:6/203: sync 2024-04-26T04:52:47.538 INFO:tasks.workunit.client.1.smithi078.stdout:8/233: sync 2024-04-26T04:52:47.538 INFO:tasks.workunit.client.1.smithi078.stdout:7/189: sync 2024-04-26T04:52:47.538 INFO:tasks.workunit.client.1.smithi078.stdout:2/162: sync 2024-04-26T04:52:47.539 INFO:tasks.workunit.client.1.smithi078.stdout:4/198: sync 2024-04-26T04:52:47.539 INFO:tasks.workunit.client.1.smithi078.stdout:9/161: sync 2024-04-26T04:52:47.540 INFO:tasks.workunit.client.1.smithi078.stdout:5/158: dwrite d7/f28 [0,4194304] 0 2024-04-26T04:52:47.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/234: mknod d0/d19/c4c 0 2024-04-26T04:52:47.546 INFO:tasks.workunit.client.1.smithi078.stdout:3/176: rmdir d16/d26/d2e 39 2024-04-26T04:52:47.546 INFO:tasks.workunit.client.1.smithi078.stdout:5/159: read d7/d18/f27 [696399,40044] 0 2024-04-26T04:52:47.546 INFO:tasks.workunit.client.1.smithi078.stdout:6/204: chown d2/d1a/d5/d11/f1c 90172847 1 2024-04-26T04:52:47.546 INFO:tasks.workunit.client.1.smithi078.stdout:1/263: chown d7/d39/d4c/f4e 60 1 2024-04-26T04:52:47.546 INFO:tasks.workunit.client.1.smithi078.stdout:7/190: chown d9/da/d20/c32 1199592 1 2024-04-26T04:52:47.548 INFO:tasks.workunit.client.1.smithi078.stdout:2/163: mknod d8/c36 0 2024-04-26T04:52:47.550 INFO:tasks.workunit.client.1.smithi078.stdout:9/162: dread - dc/df/f28 zero size 2024-04-26T04:52:47.551 INFO:tasks.workunit.client.1.smithi078.stdout:4/199: write d0/d25/d20/d31/d33/d44/f3d [105722,8984] 0 2024-04-26T04:52:47.553 INFO:tasks.workunit.client.1.smithi078.stdout:6/205: chown d2/d1a/d5/l2e 2 1 2024-04-26T04:52:47.554 INFO:tasks.workunit.client.1.smithi078.stdout:0/224: dwrite d2/d1e/f2a [0,4194304] 0 2024-04-26T04:52:47.554 INFO:tasks.workunit.client.1.smithi078.stdout:8/235: truncate d0/d19/f2a 1999989 0 2024-04-26T04:52:47.554 INFO:tasks.workunit.client.1.smithi078.stdout:2/164: stat d8/d11/d14/d15/f1b 0 2024-04-26T04:52:47.558 INFO:tasks.workunit.client.1.smithi078.stdout:4/200: mknod d0/c51 0 2024-04-26T04:52:47.559 INFO:tasks.workunit.client.1.smithi078.stdout:7/191: truncate d9/da/d11/f2f 698650 0 2024-04-26T04:52:47.559 INFO:tasks.workunit.client.1.smithi078.stdout:9/163: symlink dc/df/l2c 0 2024-04-26T04:52:47.559 INFO:tasks.workunit.client.1.smithi078.stdout:0/225: truncate d2/d9/f3e 883386 0 2024-04-26T04:52:47.561 INFO:tasks.workunit.client.1.smithi078.stdout:8/236: fsync d0/d19/d25/d36/f48 0 2024-04-26T04:52:47.564 INFO:tasks.workunit.client.1.smithi078.stdout:6/206: creat d2/d1a/d5/d2a/f3d x:0 0 0 2024-04-26T04:52:47.564 INFO:tasks.workunit.client.1.smithi078.stdout:9/164: write f1 [2877277,59379] 0 2024-04-26T04:52:47.564 INFO:tasks.workunit.client.1.smithi078.stdout:7/192: chown d9/da/d11/f12 16010803 1 2024-04-26T04:52:47.564 INFO:tasks.workunit.client.1.smithi078.stdout:2/165: symlink d8/d11/d14/l37 0 2024-04-26T04:52:47.565 INFO:tasks.workunit.client.1.smithi078.stdout:3/177: rename d16/d26/l41 to d16/d26/d2e/l44 0 2024-04-26T04:52:47.565 INFO:tasks.workunit.client.1.smithi078.stdout:4/201: write d0/d25/d20/d31/d33/d44/f3d [1377295,52245] 0 2024-04-26T04:52:47.567 INFO:tasks.workunit.client.1.smithi078.stdout:0/226: read d2/d9/d18/fd [1422460,22466] 0 2024-04-26T04:52:47.569 INFO:tasks.workunit.client.1.smithi078.stdout:1/264: dwrite f3 [0,4194304] 0 2024-04-26T04:52:47.571 INFO:tasks.workunit.client.1.smithi078.stdout:8/237: creat d0/d21/d35/d44/f4d x:0 0 0 2024-04-26T04:52:47.573 INFO:tasks.workunit.client.1.smithi078.stdout:0/227: stat d2/d9/d18/ce 0 2024-04-26T04:52:47.573 INFO:tasks.workunit.client.1.smithi078.stdout:7/193: write d9/da/d11/f12 [5273520,7350] 0 2024-04-26T04:52:47.573 INFO:tasks.workunit.client.1.smithi078.stdout:9/165: mkdir dc/df/d1a/d22/d2d 0 2024-04-26T04:52:47.574 INFO:tasks.workunit.client.1.smithi078.stdout:5/160: write d7/da/f23 [548312,90437] 0 2024-04-26T04:52:47.575 INFO:tasks.workunit.client.1.smithi078.stdout:3/178: rename ff to d16/d26/d2e/f45 0 2024-04-26T04:52:47.575 INFO:tasks.workunit.client.1.smithi078.stdout:6/207: write d2/f14 [2896076,52337] 0 2024-04-26T04:52:47.577 INFO:tasks.workunit.client.1.smithi078.stdout:9/166: write dc/f26 [549816,30849] 0 2024-04-26T04:52:47.580 INFO:tasks.workunit.client.1.smithi078.stdout:7/194: dread d9/da/d11/f2f [0,4194304] 0 2024-04-26T04:52:47.580 INFO:tasks.workunit.client.1.smithi078.stdout:1/265: dread d7/d39/db/f15 [0,4194304] 0 2024-04-26T04:52:47.582 INFO:tasks.workunit.client.1.smithi078.stdout:4/202: rename d0/f5 to d0/de/d22/d26/d2e/f52 0 2024-04-26T04:52:47.586 INFO:tasks.workunit.client.1.smithi078.stdout:2/166: link d8/d11/l16 d8/d11/d14/d15/d1a/l38 0 2024-04-26T04:52:47.586 INFO:tasks.workunit.client.1.smithi078.stdout:3/179: mkdir d16/d3c/d46 0 2024-04-26T04:52:47.587 INFO:tasks.workunit.client.1.smithi078.stdout:6/208: creat d2/d1a/d5/da/d35/f3e x:0 0 0 2024-04-26T04:52:47.589 INFO:tasks.workunit.client.1.smithi078.stdout:5/161: dread d7/d11/f2b [0,4194304] 0 2024-04-26T04:52:47.589 INFO:tasks.workunit.client.1.smithi078.stdout:0/228: rename d2/f4 to d2/d9/d18/f45 0 2024-04-26T04:52:47.589 INFO:tasks.workunit.client.1.smithi078.stdout:3/180: chown fc 129048820 1 2024-04-26T04:52:47.589 INFO:tasks.workunit.client.1.smithi078.stdout:4/203: symlink d0/de/d22/l53 0 2024-04-26T04:52:47.590 INFO:tasks.workunit.client.1.smithi078.stdout:8/238: dwrite d0/d21/d35/d44/f4d [0,4194304] 0 2024-04-26T04:52:47.591 INFO:tasks.workunit.client.1.smithi078.stdout:6/209: dread - d2/d1a/d5/d2a/f3d zero size 2024-04-26T04:52:47.592 INFO:tasks.workunit.client.1.smithi078.stdout:8/239: read d0/f1a [8702254,31681] 0 2024-04-26T04:52:47.592 INFO:tasks.workunit.client.1.smithi078.stdout:4/204: stat d0/d25/d20/d31/d33/d44/f3d 0 2024-04-26T04:52:47.593 INFO:tasks.workunit.client.1.smithi078.stdout:8/240: stat d0/d19/d25/l2e 0 2024-04-26T04:52:47.594 INFO:tasks.workunit.client.1.smithi078.stdout:2/167: creat d8/d11/d14/d15/f39 x:0 0 0 2024-04-26T04:52:47.597 INFO:tasks.workunit.client.1.smithi078.stdout:5/162: creat d7/d29/f36 x:0 0 0 2024-04-26T04:52:47.597 INFO:tasks.workunit.client.1.smithi078.stdout:8/241: dread d0/d21/d35/d44/f26 [0,4194304] 0 2024-04-26T04:52:47.600 INFO:tasks.workunit.client.1.smithi078.stdout:3/181: dread d16/f39 [0,4194304] 0 2024-04-26T04:52:47.602 INFO:tasks.workunit.client.1.smithi078.stdout:1/266: link d7/d39/d17/f3c d7/d2e/f5b 0 2024-04-26T04:52:47.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/168: rmdir d8/d11/d18 39 2024-04-26T04:52:47.605 INFO:tasks.workunit.client.1.smithi078.stdout:1/267: write d7/f57 [773713,119108] 0 2024-04-26T04:52:47.605 INFO:tasks.workunit.client.1.smithi078.stdout:9/167: dwrite dc/df/d1a/f20 [0,4194304] 0 2024-04-26T04:52:47.605 INFO:tasks.workunit.client.1.smithi078.stdout:7/195: dread d9/da/d20/f21 [0,4194304] 0 2024-04-26T04:52:47.609 INFO:tasks.workunit.client.1.smithi078.stdout:8/242: creat d0/d21/d35/f4e x:0 0 0 2024-04-26T04:52:47.610 INFO:tasks.workunit.client.1.smithi078.stdout:6/210: truncate d2/f1e 1595320 0 2024-04-26T04:52:47.610 INFO:tasks.workunit.client.1.smithi078.stdout:8/243: readlink d0/d19/l3b 0 2024-04-26T04:52:47.611 INFO:tasks.workunit.client.1.smithi078.stdout:3/182: symlink d16/d26/d2b/l47 0 2024-04-26T04:52:47.612 INFO:tasks.workunit.client.1.smithi078.stdout:6/211: read - d2/d1a/d5/da/d35/f3e zero size 2024-04-26T04:52:47.612 INFO:tasks.workunit.client.1.smithi078.stdout:8/244: chown d0/d21/d35/d44/l1e 18 1 2024-04-26T04:52:47.613 INFO:tasks.workunit.client.1.smithi078.stdout:3/183: fsync d16/f21 0 2024-04-26T04:52:47.614 INFO:tasks.workunit.client.1.smithi078.stdout:2/169: creat d8/f3a x:0 0 0 2024-04-26T04:52:47.614 INFO:tasks.workunit.client.1.smithi078.stdout:9/168: creat dc/df/d17/f2e x:0 0 0 2024-04-26T04:52:47.615 INFO:tasks.workunit.client.1.smithi078.stdout:8/245: read - d0/d21/d35/f4e zero size 2024-04-26T04:52:47.617 INFO:tasks.workunit.client.1.smithi078.stdout:4/205: rename d0/de/d22/d26/d2e/f35 to d0/d25/f54 0 2024-04-26T04:52:47.618 INFO:tasks.workunit.client.1.smithi078.stdout:6/212: symlink d2/de/d27/l3f 0 2024-04-26T04:52:47.620 INFO:tasks.workunit.client.1.smithi078.stdout:0/229: unlink d2/d9/f28 0 2024-04-26T04:52:47.623 INFO:tasks.workunit.client.1.smithi078.stdout:2/170: dread d8/d11/f2a [0,4194304] 0 2024-04-26T04:52:47.623 INFO:tasks.workunit.client.1.smithi078.stdout:1/268: getdents d7/d39/db/d47/d4a 0 2024-04-26T04:52:47.625 INFO:tasks.workunit.client.1.smithi078.stdout:8/246: creat d0/d21/d35/d44/f4f x:0 0 0 2024-04-26T04:52:47.625 INFO:tasks.workunit.client.1.smithi078.stdout:4/206: dread d0/d25/d20/f15 [0,4194304] 0 2024-04-26T04:52:47.627 INFO:tasks.workunit.client.1.smithi078.stdout:6/213: creat d2/d1a/d5/f40 x:0 0 0 2024-04-26T04:52:47.628 INFO:tasks.workunit.client.1.smithi078.stdout:5/163: getdents d7/da 0 2024-04-26T04:52:47.628 INFO:tasks.workunit.client.1.smithi078.stdout:4/207: readlink d0/d1d/l4a 0 2024-04-26T04:52:47.629 INFO:tasks.workunit.client.1.smithi078.stdout:3/184: creat d16/d26/f48 x:0 0 0 2024-04-26T04:52:47.632 INFO:tasks.workunit.client.1.smithi078.stdout:9/169: creat dc/df/f2f x:0 0 0 2024-04-26T04:52:47.633 INFO:tasks.workunit.client.1.smithi078.stdout:0/230: dwrite d2/d9/f17 [0,4194304] 0 2024-04-26T04:52:47.634 INFO:tasks.workunit.client.1.smithi078.stdout:2/171: chown d8/d11/d14/d15/d1a/l38 1691277 1 2024-04-26T04:52:47.635 INFO:tasks.workunit.client.1.smithi078.stdout:5/164: fdatasync d7/d18/f27 0 2024-04-26T04:52:47.636 INFO:tasks.workunit.client.1.smithi078.stdout:7/196: dwrite d9/da/d11/f2f [0,4194304] 0 2024-04-26T04:52:47.638 INFO:tasks.workunit.client.1.smithi078.stdout:3/185: unlink d16/d26/d2e/l2f 0 2024-04-26T04:52:47.639 INFO:tasks.workunit.client.1.smithi078.stdout:9/170: creat dc/df/d17/f30 x:0 0 0 2024-04-26T04:52:47.641 INFO:tasks.workunit.client.1.smithi078.stdout:6/214: mkdir d2/d1a/d41 0 2024-04-26T04:52:47.642 INFO:tasks.workunit.client.1.smithi078.stdout:8/247: write d0/d21/f29 [1926893,117048] 0 2024-04-26T04:52:47.642 INFO:tasks.workunit.client.1.smithi078.stdout:3/186: dread - d16/d26/d2e/f42 zero size 2024-04-26T04:52:47.647 INFO:tasks.workunit.client.1.smithi078.stdout:9/171: symlink dc/df/d1a/d22/l31 0 2024-04-26T04:52:47.648 INFO:tasks.workunit.client.1.smithi078.stdout:5/165: mkdir d7/d18/d32/d37 0 2024-04-26T04:52:47.648 INFO:tasks.workunit.client.1.smithi078.stdout:0/231: dwrite d2/d9/f17 [0,4194304] 0 2024-04-26T04:52:47.649 INFO:tasks.workunit.client.1.smithi078.stdout:8/248: mknod d0/d21/d35/d44/c50 0 2024-04-26T04:52:47.649 INFO:tasks.workunit.client.1.smithi078.stdout:3/187: write d16/d26/d2e/f45 [210117,77314] 0 2024-04-26T04:52:47.650 INFO:tasks.workunit.client.1.smithi078.stdout:2/172: rename d8/f9 to d8/d11/d14/f3b 0 2024-04-26T04:52:47.651 INFO:tasks.workunit.client.1.smithi078.stdout:9/172: rename dc to dc/df/d1a/d32 22 2024-04-26T04:52:47.652 INFO:tasks.workunit.client.1.smithi078.stdout:8/249: fdatasync d0/f23 0 2024-04-26T04:52:47.657 INFO:tasks.workunit.client.1.smithi078.stdout:8/250: write d0/d21/f29 [3586339,51899] 0 2024-04-26T04:52:47.659 INFO:tasks.workunit.client.1.smithi078.stdout:2/173: unlink d8/d11/d14/f2e 0 2024-04-26T04:52:47.660 INFO:tasks.workunit.client.1.smithi078.stdout:0/232: mknod d2/d39/c46 0 2024-04-26T04:52:47.662 INFO:tasks.workunit.client.1.smithi078.stdout:7/197: write d9/da/d20/f24 [559991,56657] 0 2024-04-26T04:52:47.662 INFO:tasks.workunit.client.1.smithi078.stdout:6/215: getdents d2/d1a/d5/d11 0 2024-04-26T04:52:47.663 INFO:tasks.workunit.client.1.smithi078.stdout:5/166: dwrite f5 [0,4194304] 0 2024-04-26T04:52:47.663 INFO:tasks.workunit.client.1.smithi078.stdout:4/208: truncate d0/de/f1a 745236 0 2024-04-26T04:52:47.665 INFO:tasks.workunit.client.1.smithi078.stdout:1/269: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:47.667 INFO:tasks.workunit.client.1.smithi078.stdout:3/188: mkdir d16/d3c/d46/d49 0 2024-04-26T04:52:47.669 INFO:tasks.workunit.client.1.smithi078.stdout:4/209: readlink d0/d25/d20/d31/d33/l38 0 2024-04-26T04:52:47.671 INFO:tasks.workunit.client.1.smithi078.stdout:5/167: read d7/f8 [8298381,47359] 0 2024-04-26T04:52:47.672 INFO:tasks.workunit.client.1.smithi078.stdout:9/173: dwrite dc/df/f28 [0,4194304] 0 2024-04-26T04:52:47.673 INFO:tasks.workunit.client.1.smithi078.stdout:3/189: write d16/d26/d2b/f40 [80380,113429] 0 2024-04-26T04:52:47.673 INFO:tasks.workunit.client.1.smithi078.stdout:2/174: read d8/d11/f22 [342058,127599] 0 2024-04-26T04:52:47.673 INFO:tasks.workunit.client.1.smithi078.stdout:4/210: chown d0/f13 22259988 1 2024-04-26T04:52:47.674 INFO:tasks.workunit.client.1.smithi078.stdout:7/198: creat d9/da/d11/d13/d1a/d2d/f3f x:0 0 0 2024-04-26T04:52:47.674 INFO:tasks.workunit.client.1.smithi078.stdout:0/233: symlink d2/d1e/l47 0 2024-04-26T04:52:47.679 INFO:tasks.workunit.client.1.smithi078.stdout:8/251: getdents d0/d19/d25/d36 0 2024-04-26T04:52:47.679 INFO:tasks.workunit.client.1.smithi078.stdout:7/199: write d9/da/d20/f24 [415736,65341] 0 2024-04-26T04:52:47.679 INFO:tasks.workunit.client.1.smithi078.stdout:2/175: write d8/f25 [442580,118513] 0 2024-04-26T04:52:47.682 INFO:tasks.workunit.client.1.smithi078.stdout:6/216: dwrite d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:47.685 INFO:tasks.workunit.client.1.smithi078.stdout:6/217: dread - d2/d1a/d5/f40 zero size 2024-04-26T04:52:47.686 INFO:tasks.workunit.client.1.smithi078.stdout:4/211: unlink d0/de/d22/l53 0 2024-04-26T04:52:47.688 INFO:tasks.workunit.client.1.smithi078.stdout:6/218: truncate d2/d1a/d5/d11/f1c 641567 0 2024-04-26T04:52:47.689 INFO:tasks.workunit.client.1.smithi078.stdout:9/174: dread dc/df/d1a/f20 [0,4194304] 0 2024-04-26T04:52:47.690 INFO:tasks.workunit.client.1.smithi078.stdout:8/252: symlink d0/d21/l51 0 2024-04-26T04:52:47.690 INFO:tasks.workunit.client.1.smithi078.stdout:5/168: creat d7/f38 x:0 0 0 2024-04-26T04:52:47.695 INFO:tasks.workunit.client.1.smithi078.stdout:9/175: mknod dc/df/d1a/d22/c33 0 2024-04-26T04:52:47.696 INFO:tasks.workunit.client.1.smithi078.stdout:3/190: getdents d16/d26/d2b 0 2024-04-26T04:52:47.699 INFO:tasks.workunit.client.1.smithi078.stdout:7/200: dread d9/da/f23 [0,4194304] 0 2024-04-26T04:52:47.699 INFO:tasks.workunit.client.1.smithi078.stdout:6/219: dread d2/d1a/d5/f8 [4194304,4194304] 0 2024-04-26T04:52:47.700 INFO:tasks.workunit.client.1.smithi078.stdout:3/191: rmdir d16/d26/d2b/d31 39 2024-04-26T04:52:47.702 INFO:tasks.workunit.client.1.smithi078.stdout:6/220: fsync d2/d1a/d5/f32 0 2024-04-26T04:52:47.702 INFO:tasks.workunit.client.1.smithi078.stdout:2/176: dwrite d8/d11/d14/f3b [8388608,4194304] 0 2024-04-26T04:52:47.707 INFO:tasks.workunit.client.1.smithi078.stdout:6/221: write d2/d1a/d5/f31 [342840,96634] 0 2024-04-26T04:52:47.708 INFO:tasks.workunit.client.1.smithi078.stdout:3/192: link d16/f3f d16/d26/f4a 0 2024-04-26T04:52:47.709 INFO:tasks.workunit.client.1.smithi078.stdout:2/177: creat d8/d11/d14/d15/f3c x:0 0 0 2024-04-26T04:52:47.710 INFO:tasks.workunit.client.1.smithi078.stdout:0/234: dwrite d2/f37 [0,4194304] 0 2024-04-26T04:52:47.711 INFO:tasks.workunit.client.1.smithi078.stdout:9/176: dread dc/df/f1d [0,4194304] 0 2024-04-26T04:52:47.711 INFO:tasks.workunit.client.1.smithi078.stdout:3/193: fdatasync d16/d26/f29 0 2024-04-26T04:52:47.712 INFO:tasks.workunit.client.1.smithi078.stdout:6/222: write d2/de/d27/f37 [172992,10375] 0 2024-04-26T04:52:47.713 INFO:tasks.workunit.client.1.smithi078.stdout:8/253: rename d0/d21 to d0/d19/d52 0 2024-04-26T04:52:47.718 INFO:tasks.workunit.client.1.smithi078.stdout:2/178: chown d8/d11/d14/f1c 212 1 2024-04-26T04:52:47.718 INFO:tasks.workunit.client.1.smithi078.stdout:9/177: write dc/df/f2f [714155,10659] 0 2024-04-26T04:52:47.718 INFO:tasks.workunit.client.1.smithi078.stdout:4/212: fdatasync d0/de/f1a 0 2024-04-26T04:52:47.718 INFO:tasks.workunit.client.1.smithi078.stdout:6/223: chown d2/d1a/f6 253860 1 2024-04-26T04:52:47.718 INFO:tasks.workunit.client.1.smithi078.stdout:7/201: dwrite d9/da/d20/f31 [0,4194304] 0 2024-04-26T04:52:47.719 INFO:tasks.workunit.client.1.smithi078.stdout:0/235: rmdir d2/d1e 39 2024-04-26T04:52:47.720 INFO:tasks.workunit.client.1.smithi078.stdout:9/178: fdatasync dc/df/d27/f13 0 2024-04-26T04:52:47.723 INFO:tasks.workunit.client.1.smithi078.stdout:6/224: chown d2/d1a/c26 3770 1 2024-04-26T04:52:47.724 INFO:tasks.workunit.client.1.smithi078.stdout:3/194: rename d16/f1e to d16/d3c/d46/d49/f4b 0 2024-04-26T04:52:47.727 INFO:tasks.workunit.client.1.smithi078.stdout:7/202: chown d9/da/d11/f2f 47 1 2024-04-26T04:52:47.728 INFO:tasks.workunit.client.1.smithi078.stdout:9/179: symlink dc/df/d27/l34 0 2024-04-26T04:52:47.729 INFO:tasks.workunit.client.1.smithi078.stdout:4/213: dread d0/d25/f3a [0,4194304] 0 2024-04-26T04:52:47.731 INFO:tasks.workunit.client.1.smithi078.stdout:6/225: dread d2/d1a/d5/f31 [0,4194304] 0 2024-04-26T04:52:47.732 INFO:tasks.workunit.client.1.smithi078.stdout:3/195: mknod d16/d3c/d46/d49/c4c 0 2024-04-26T04:52:47.733 INFO:tasks.workunit.client.1.smithi078.stdout:2/179: dwrite d8/f2b [0,4194304] 0 2024-04-26T04:52:47.733 INFO:tasks.workunit.client.1.smithi078.stdout:9/180: creat dc/df/d1a/f35 x:0 0 0 2024-04-26T04:52:47.734 INFO:tasks.workunit.client.1.smithi078.stdout:0/236: fsync d2/d9/d18/f22 0 2024-04-26T04:52:47.735 INFO:tasks.workunit.client.1.smithi078.stdout:9/181: write dc/f26 [1394035,32703] 0 2024-04-26T04:52:47.737 INFO:tasks.workunit.client.1.smithi078.stdout:7/203: rename d9/d16/f22 to d9/da/d11/d13/d1a/f40 0 2024-04-26T04:52:47.738 INFO:tasks.workunit.client.1.smithi078.stdout:4/214: mkdir d0/d25/d20/d31/d33/d44/d55 0 2024-04-26T04:52:47.742 INFO:tasks.workunit.client.1.smithi078.stdout:6/226: mkdir d2/d1a/d5/d3a/d42 0 2024-04-26T04:52:47.744 INFO:tasks.workunit.client.1.smithi078.stdout:3/196: dwrite d16/d26/d2e/f42 [0,4194304] 0 2024-04-26T04:52:47.744 INFO:tasks.workunit.client.1.smithi078.stdout:7/204: mkdir d9/d16/d41 0 2024-04-26T04:52:47.746 INFO:tasks.workunit.client.1.smithi078.stdout:4/215: mkdir d0/de/d22/d27/d56 0 2024-04-26T04:52:47.748 INFO:tasks.workunit.client.1.smithi078.stdout:2/180: rename d8/f27 to d8/d11/d14/d15/d1a/d34/f3d 0 2024-04-26T04:52:47.750 INFO:tasks.workunit.client.1.smithi078.stdout:0/237: creat d2/f48 x:0 0 0 2024-04-26T04:52:47.752 INFO:tasks.workunit.client.1.smithi078.stdout:6/227: unlink d2/d1a/d5/da/cc 0 2024-04-26T04:52:47.752 INFO:tasks.workunit.client.1.smithi078.stdout:2/181: mkdir d8/d3e 0 2024-04-26T04:52:47.753 INFO:tasks.workunit.client.1.smithi078.stdout:4/216: write d0/de/f1e [230683,30443] 0 2024-04-26T04:52:47.755 INFO:tasks.workunit.client.1.smithi078.stdout:9/182: dread f8 [0,4194304] 0 2024-04-26T04:52:47.760 INFO:tasks.workunit.client.1.smithi078.stdout:7/205: dwrite d9/da/d33/f3e [0,4194304] 0 2024-04-26T04:52:47.760 INFO:tasks.workunit.client.1.smithi078.stdout:3/197: dwrite f13 [0,4194304] 0 2024-04-26T04:52:47.761 INFO:tasks.workunit.client.1.smithi078.stdout:6/228: creat d2/d1a/f43 x:0 0 0 2024-04-26T04:52:47.765 INFO:tasks.workunit.client.1.smithi078.stdout:0/238: rename d2/d1e/f23 to d2/d9/d18/f49 0 2024-04-26T04:52:47.766 INFO:tasks.workunit.client.1.smithi078.stdout:9/183: creat dc/df/f36 x:0 0 0 2024-04-26T04:52:47.767 INFO:tasks.workunit.client.1.smithi078.stdout:2/182: dwrite d8/f25 [0,4194304] 0 2024-04-26T04:52:47.767 INFO:tasks.workunit.client.1.smithi078.stdout:3/198: creat d16/d26/d2e/f4d x:0 0 0 2024-04-26T04:52:47.769 INFO:tasks.workunit.client.1.smithi078.stdout:6/229: mkdir d2/d1a/d5/da/d35/d44 0 2024-04-26T04:52:47.770 INFO:tasks.workunit.client.1.smithi078.stdout:7/206: creat d9/da/d11/d13/d1a/f42 x:0 0 0 2024-04-26T04:52:47.770 INFO:tasks.workunit.client.1.smithi078.stdout:6/230: write d2/d1a/f6 [603646,48886] 0 2024-04-26T04:52:47.771 INFO:tasks.workunit.client.1.smithi078.stdout:4/217: creat d0/de/d22/d26/f57 x:0 0 0 2024-04-26T04:52:47.773 INFO:tasks.workunit.client.1.smithi078.stdout:7/207: chown d9/da/d11/d13/d1a/f40 10 1 2024-04-26T04:52:47.774 INFO:tasks.workunit.client.1.smithi078.stdout:7/208: read d9/da/f15 [8076767,27461] 0 2024-04-26T04:52:47.774 INFO:tasks.workunit.client.1.smithi078.stdout:4/218: rename d0/d25/d20/d31/d33/d44/d55 to d0/d25/d20/d31/d33/d44/d55/d58 22 2024-04-26T04:52:47.777 INFO:tasks.workunit.client.1.smithi078.stdout:0/239: mkdir d2/d9/d31/d4a 0 2024-04-26T04:52:47.780 INFO:tasks.workunit.client.1.smithi078.stdout:7/209: symlink d9/da/d33/l43 0 2024-04-26T04:52:47.780 INFO:tasks.workunit.client.1.smithi078.stdout:4/219: creat d0/d25/d20/d31/d33/d44/f59 x:0 0 0 2024-04-26T04:52:47.781 INFO:tasks.workunit.client.1.smithi078.stdout:0/240: symlink d2/d9/d31/l4b 0 2024-04-26T04:52:47.783 INFO:tasks.workunit.client.1.smithi078.stdout:2/183: creat d8/d11/f3f x:0 0 0 2024-04-26T04:52:47.784 INFO:tasks.workunit.client.1.smithi078.stdout:2/184: write d8/d11/f31 [776975,104213] 0 2024-04-26T04:52:47.785 INFO:tasks.workunit.client.1.smithi078.stdout:4/220: mkdir d0/de/d22/d27/d56/d5a 0 2024-04-26T04:52:47.788 INFO:tasks.workunit.client.1.smithi078.stdout:2/185: creat d8/d11/d14/f40 x:0 0 0 2024-04-26T04:52:47.788 INFO:tasks.workunit.client.1.smithi078.stdout:2/186: readlink d8/l30 0 2024-04-26T04:52:47.789 INFO:tasks.workunit.client.1.smithi078.stdout:4/221: creat d0/d25/d20/d31/d33/f5b x:0 0 0 2024-04-26T04:52:47.794 INFO:tasks.workunit.client.1.smithi078.stdout:0/241: dwrite d2/d39/f42 [0,4194304] 0 2024-04-26T04:52:47.794 INFO:tasks.workunit.client.1.smithi078.stdout:6/231: getdents d2/d1a/d5/da/d35 0 2024-04-26T04:52:47.795 INFO:tasks.workunit.client.1.smithi078.stdout:6/232: dread d2/d1a/fd [0,4194304] 0 2024-04-26T04:52:47.796 INFO:tasks.workunit.client.1.smithi078.stdout:0/242: creat d2/d9/d31/d4a/f4c x:0 0 0 2024-04-26T04:52:47.799 INFO:tasks.workunit.client.1.smithi078.stdout:2/187: write d8/f1f [727958,116132] 0 2024-04-26T04:52:47.804 INFO:tasks.workunit.client.1.smithi078.stdout:2/188: symlink d8/d11/d18/l41 0 2024-04-26T04:52:47.805 INFO:tasks.workunit.client.1.smithi078.stdout:0/243: dwrite d2/d1e/f2a [0,4194304] 0 2024-04-26T04:52:47.807 INFO:tasks.workunit.client.1.smithi078.stdout:2/189: symlink d8/l42 0 2024-04-26T04:52:47.808 INFO:tasks.workunit.client.1.smithi078.stdout:2/190: fsync d8/d11/d14/d15/f32 0 2024-04-26T04:52:47.810 INFO:tasks.workunit.client.1.smithi078.stdout:0/244: rename d2/d9/d18/c24 to d2/d1e/c4d 0 2024-04-26T04:52:47.811 INFO:tasks.workunit.client.1.smithi078.stdout:2/191: creat d8/d3e/f43 x:0 0 0 2024-04-26T04:52:47.815 INFO:tasks.workunit.client.1.smithi078.stdout:4/222: dread d0/f37 [0,4194304] 0 2024-04-26T04:52:47.815 INFO:tasks.workunit.client.1.smithi078.stdout:0/245: rename d2/f27 to d2/d9/d18/f4e 0 2024-04-26T04:52:47.816 INFO:tasks.workunit.client.1.smithi078.stdout:2/192: rmdir d8/d11/d14/d15/d1a/d34 39 2024-04-26T04:52:47.817 INFO:tasks.workunit.client.1.smithi078.stdout:6/233: dread d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:47.818 INFO:tasks.workunit.client.1.smithi078.stdout:6/234: dread - d2/d1a/d5/da/d35/f3e zero size 2024-04-26T04:52:47.818 INFO:tasks.workunit.client.1.smithi078.stdout:4/223: fdatasync d0/d25/d20/f15 0 2024-04-26T04:52:47.822 INFO:tasks.workunit.client.1.smithi078.stdout:2/193: rename d8/lc to d8/d11/d14/d15/d1a/l44 0 2024-04-26T04:52:47.824 INFO:tasks.workunit.client.1.smithi078.stdout:4/224: getdents d0/de/d22/d27/d56/d5a 0 2024-04-26T04:52:47.825 INFO:tasks.workunit.client.1.smithi078.stdout:0/246: dwrite d2/f25 [0,4194304] 0 2024-04-26T04:52:47.828 INFO:tasks.workunit.client.1.smithi078.stdout:6/235: dwrite d2/d1a/d5/f31 [0,4194304] 0 2024-04-26T04:52:47.830 INFO:tasks.workunit.client.1.smithi078.stdout:0/247: rename d2/f25 to d2/d9/d31/f4f 0 2024-04-26T04:52:47.832 INFO:tasks.workunit.client.1.smithi078.stdout:6/236: read d2/d1a/d5/da/f22 [90456,90988] 0 2024-04-26T04:52:47.836 INFO:tasks.workunit.client.1.smithi078.stdout:4/225: dwrite d0/de/f34 [0,4194304] 0 2024-04-26T04:52:47.838 INFO:tasks.workunit.client.1.smithi078.stdout:0/248: unlink d2/f40 0 2024-04-26T04:52:47.839 INFO:tasks.workunit.client.1.smithi078.stdout:6/237: mkdir d2/d1a/d5/da/d35/d44/d45 0 2024-04-26T04:52:47.841 INFO:tasks.workunit.client.1.smithi078.stdout:4/226: rename d0/d1d/f45 to d0/de/d22/d27/d56/f5c 0 2024-04-26T04:52:47.842 INFO:tasks.workunit.client.1.smithi078.stdout:6/238: fsync d2/d1a/d5/f8 0 2024-04-26T04:52:47.842 INFO:tasks.workunit.client.1.smithi078.stdout:4/227: write d0/f8 [3674064,130508] 0 2024-04-26T04:52:47.844 INFO:tasks.workunit.client.1.smithi078.stdout:4/228: stat d0/c7 0 2024-04-26T04:52:47.845 INFO:tasks.workunit.client.1.smithi078.stdout:6/239: mknod d2/c46 0 2024-04-26T04:52:47.845 INFO:tasks.workunit.client.1.smithi078.stdout:4/229: fsync d0/de/f1e 0 2024-04-26T04:52:47.847 INFO:tasks.workunit.client.1.smithi078.stdout:4/230: read - d0/d25/d20/d31/d33/d44/f59 zero size 2024-04-26T04:52:47.847 INFO:tasks.workunit.client.1.smithi078.stdout:0/249: dwrite d2/d9/d18/f22 [0,4194304] 0 2024-04-26T04:52:47.848 INFO:tasks.workunit.client.1.smithi078.stdout:4/231: chown d0/d25/d20/d31/d33/d44/d55 56595131 1 2024-04-26T04:52:47.851 INFO:tasks.workunit.client.1.smithi078.stdout:0/250: creat d2/d9/d31/f50 x:0 0 0 2024-04-26T04:52:47.852 INFO:tasks.workunit.client.1.smithi078.stdout:4/232: rename d0/d2f/f3e to d0/d25/d20/d31/d33/d44/d55/f5d 0 2024-04-26T04:52:47.853 INFO:tasks.workunit.client.1.smithi078.stdout:0/251: fdatasync d2/f29 0 2024-04-26T04:52:47.854 INFO:tasks.workunit.client.1.smithi078.stdout:4/233: write d0/d25/d20/d31/d47/f49 [135570,72878] 0 2024-04-26T04:52:47.855 INFO:tasks.workunit.client.1.smithi078.stdout:0/252: fdatasync d2/d1e/f2a 0 2024-04-26T04:52:47.856 INFO:tasks.workunit.client.1.smithi078.stdout:2/194: truncate d8/d11/f31 240119 0 2024-04-26T04:52:47.857 INFO:tasks.workunit.client.1.smithi078.stdout:2/195: dread - d8/d11/d14/d15/f39 zero size 2024-04-26T04:52:47.858 INFO:tasks.workunit.client.1.smithi078.stdout:4/234: mkdir d0/d25/d20/d31/d33/d44/d55/d5e 0 2024-04-26T04:52:47.858 INFO:tasks.workunit.client.1.smithi078.stdout:2/196: write d8/d11/d14/d15/f32 [242149,4220] 0 2024-04-26T04:52:47.862 INFO:tasks.workunit.client.1.smithi078.stdout:2/197: creat d8/d11/f45 x:0 0 0 2024-04-26T04:52:47.865 INFO:tasks.workunit.client.1.smithi078.stdout:2/198: write d8/d11/f3f [664953,29505] 0 2024-04-26T04:52:47.866 INFO:tasks.workunit.client.1.smithi078.stdout:0/253: rmdir d2/d9/d31 39 2024-04-26T04:52:47.866 INFO:tasks.workunit.client.1.smithi078.stdout:4/235: dwrite d0/d25/d20/d31/d33/d44/f59 [0,4194304] 0 2024-04-26T04:52:47.873 INFO:tasks.workunit.client.1.smithi078.stdout:2/199: link d8/f2b d8/d11/f46 0 2024-04-26T04:52:47.873 INFO:tasks.workunit.client.1.smithi078.stdout:6/240: dwrite d2/f1e [0,4194304] 0 2024-04-26T04:52:47.878 INFO:tasks.workunit.client.1.smithi078.stdout:2/200: creat d8/d11/d18/f47 x:0 0 0 2024-04-26T04:52:47.879 INFO:tasks.workunit.client.1.smithi078.stdout:0/254: dread d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:52:47.880 INFO:tasks.workunit.client.1.smithi078.stdout:4/236: dwrite d0/d25/d20/d31/d33/f5b [0,4194304] 0 2024-04-26T04:52:47.884 INFO:tasks.workunit.client.1.smithi078.stdout:4/237: symlink d0/de/d22/l5f 0 2024-04-26T04:52:47.885 INFO:tasks.workunit.client.1.smithi078.stdout:4/238: write d0/f13 [1256590,11070] 0 2024-04-26T04:52:47.886 INFO:tasks.workunit.client.1.smithi078.stdout:6/241: truncate d2/d1a/d5/d11/f15 8094757 0 2024-04-26T04:52:47.888 INFO:tasks.workunit.client.1.smithi078.stdout:0/255: symlink d2/d9/d31/d4a/l51 0 2024-04-26T04:52:47.889 INFO:tasks.workunit.client.1.smithi078.stdout:2/201: dwrite d8/d3e/f43 [0,4194304] 0 2024-04-26T04:52:47.892 INFO:tasks.workunit.client.1.smithi078.stdout:6/242: creat d2/d1a/d5/da/d35/d44/d45/f47 x:0 0 0 2024-04-26T04:52:47.892 INFO:tasks.workunit.client.1.smithi078.stdout:0/256: creat d2/d39/f52 x:0 0 0 2024-04-26T04:52:47.894 INFO:tasks.workunit.client.1.smithi078.stdout:4/239: link d0/d25/d20/f2a d0/de/d22/d27/d56/f60 0 2024-04-26T04:52:47.894 INFO:tasks.workunit.client.1.smithi078.stdout:6/243: write d2/de/f33 [34276,81948] 0 2024-04-26T04:52:47.895 INFO:tasks.workunit.client.1.smithi078.stdout:2/202: creat d8/d11/f48 x:0 0 0 2024-04-26T04:52:47.895 INFO:tasks.workunit.client.1.smithi078.stdout:6/244: chown d2/f1e 171418415 1 2024-04-26T04:52:47.897 INFO:tasks.workunit.client.1.smithi078.stdout:0/257: rmdir d2/d9/d18 39 2024-04-26T04:52:47.897 INFO:tasks.workunit.client.1.smithi078.stdout:2/203: write d8/d11/f45 [310990,74129] 0 2024-04-26T04:52:47.900 INFO:tasks.workunit.client.1.smithi078.stdout:4/240: creat d0/de/d22/d27/d56/f61 x:0 0 0 2024-04-26T04:52:47.901 INFO:tasks.workunit.client.1.smithi078.stdout:6/245: creat d2/de/f48 x:0 0 0 2024-04-26T04:52:47.904 INFO:tasks.workunit.client.1.smithi078.stdout:6/246: unlink d2/l4 0 2024-04-26T04:52:47.906 INFO:tasks.workunit.client.1.smithi078.stdout:0/258: dwrite d2/d9/d31/f50 [0,4194304] 0 2024-04-26T04:52:47.908 INFO:tasks.workunit.client.1.smithi078.stdout:6/247: unlink d2/f1e 0 2024-04-26T04:52:47.910 INFO:tasks.workunit.client.1.smithi078.stdout:4/241: dwrite d0/d25/d20/d31/f3f [0,4194304] 0 2024-04-26T04:52:47.912 INFO:tasks.workunit.client.1.smithi078.stdout:0/259: symlink d2/d9/l53 0 2024-04-26T04:52:47.913 INFO:tasks.workunit.client.1.smithi078.stdout:1/270: sync 2024-04-26T04:52:47.916 INFO:tasks.workunit.client.1.smithi078.stdout:6/248: rename d2/d1a/d5/da/d35/d44/d45 to d2/d49 0 2024-04-26T04:52:47.919 INFO:tasks.workunit.client.1.smithi078.stdout:1/271: mkdir d7/d39/db/d37/d58/d5c 0 2024-04-26T04:52:47.919 INFO:tasks.workunit.client.1.smithi078.stdout:6/249: dread d2/d1a/d5/d11/f1c [0,4194304] 0 2024-04-26T04:52:47.920 INFO:tasks.workunit.client.1.smithi078.stdout:0/260: dread d2/d39/f42 [0,4194304] 0 2024-04-26T04:52:47.920 INFO:tasks.workunit.client.1.smithi078.stdout:6/250: readlink d2/d1a/d5/l2e 0 2024-04-26T04:52:47.923 INFO:tasks.workunit.client.1.smithi078.stdout:0/261: truncate d2/f14 1071687 0 2024-04-26T04:52:47.923 INFO:tasks.workunit.client.1.smithi078.stdout:4/242: dwrite d0/de/d22/d26/d2e/f52 [4194304,4194304] 0 2024-04-26T04:52:47.923 INFO:tasks.workunit.client.1.smithi078.stdout:6/251: creat d2/d1a/d5/da/f4a x:0 0 0 2024-04-26T04:52:47.927 INFO:tasks.workunit.client.1.smithi078.stdout:1/272: link d7/d39/dd/d20/l29 d7/l5d 0 2024-04-26T04:52:47.928 INFO:tasks.workunit.client.1.smithi078.stdout:0/262: dread d2/ff [0,4194304] 0 2024-04-26T04:52:47.932 INFO:tasks.workunit.client.1.smithi078.stdout:0/263: dread d2/d1e/f2a [0,4194304] 0 2024-04-26T04:52:47.934 INFO:tasks.workunit.client.1.smithi078.stdout:6/252: rename d2/d1a/d5/da/d35/d44 to d2/d1a/d5/da/d36/d4b 0 2024-04-26T04:52:47.935 INFO:tasks.workunit.client.1.smithi078.stdout:6/253: truncate d2/de/f33 762240 0 2024-04-26T04:52:47.938 INFO:tasks.workunit.client.1.smithi078.stdout:6/254: mkdir d2/de/d27/d4c 0 2024-04-26T04:52:47.940 INFO:tasks.workunit.client.1.smithi078.stdout:0/264: fsync d2/d9/d18/f13 0 2024-04-26T04:52:47.945 INFO:tasks.workunit.client.1.smithi078.stdout:4/243: truncate d0/de/d22/d26/d2e/f52 8028375 0 2024-04-26T04:52:47.945 INFO:tasks.workunit.client.1.smithi078.stdout:0/265: dwrite d2/f14 [0,4194304] 0 2024-04-26T04:52:47.947 INFO:tasks.workunit.client.1.smithi078.stdout:0/266: write d2/d9/d31/f36 [5086469,45971] 0 2024-04-26T04:52:47.948 INFO:tasks.workunit.client.1.smithi078.stdout:0/267: fdatasync d2/d9/d31/d4a/f4c 0 2024-04-26T04:52:47.948 INFO:tasks.workunit.client.1.smithi078.stdout:0/268: chown d2/d1e 14 1 2024-04-26T04:52:47.950 INFO:tasks.workunit.client.1.smithi078.stdout:6/255: write d2/d1a/d5/da/f2c [309986,47539] 0 2024-04-26T04:52:47.950 INFO:tasks.workunit.client.1.smithi078.stdout:0/269: chown d2/f26 1814782 1 2024-04-26T04:52:47.951 INFO:tasks.workunit.client.1.smithi078.stdout:0/270: stat d2/d1e 0 2024-04-26T04:52:47.953 INFO:tasks.workunit.client.1.smithi078.stdout:6/256: mkdir d2/d1a/d5/d2a/d4d 0 2024-04-26T04:52:47.956 INFO:tasks.workunit.client.1.smithi078.stdout:6/257: truncate d2/d1a/d5/d2a/f3d 808266 0 2024-04-26T04:52:47.957 INFO:tasks.workunit.client.1.smithi078.stdout:4/244: dwrite d0/de/f34 [4194304,4194304] 0 2024-04-26T04:52:47.958 INFO:tasks.workunit.client.1.smithi078.stdout:6/258: creat d2/d1a/d5/d11/f4e x:0 0 0 2024-04-26T04:52:47.961 INFO:tasks.workunit.client.1.smithi078.stdout:6/259: getdents d2/d1a/d5/d2a/d4d 0 2024-04-26T04:52:47.963 INFO:tasks.workunit.client.1.smithi078.stdout:6/260: read d2/f14 [1479024,38644] 0 2024-04-26T04:52:47.964 INFO:tasks.workunit.client.1.smithi078.stdout:4/245: dwrite d0/de/d22/d26/f57 [0,4194304] 0 2024-04-26T04:52:47.971 INFO:tasks.workunit.client.1.smithi078.stdout:6/261: read d2/de/f2b [560109,32221] 0 2024-04-26T04:52:47.974 INFO:tasks.workunit.client.1.smithi078.stdout:6/262: creat d2/d1a/d5/d11/f4f x:0 0 0 2024-04-26T04:52:47.974 INFO:tasks.workunit.client.1.smithi078.stdout:6/263: chown d2/d1a/d5/da/f4a 6284 1 2024-04-26T04:52:47.974 INFO:tasks.workunit.client.1.smithi078.stdout:6/264: write d2/d1a/d5/d11/f4e [997200,84931] 0 2024-04-26T04:52:47.975 INFO:tasks.workunit.client.1.smithi078.stdout:6/265: fdatasync d2/d1a/d5/d11/f4f 0 2024-04-26T04:52:47.984 INFO:tasks.workunit.client.1.smithi078.stdout:5/169: sync 2024-04-26T04:52:47.989 INFO:tasks.workunit.client.1.smithi078.stdout:5/170: creat d7/f39 x:0 0 0 2024-04-26T04:52:47.989 INFO:tasks.workunit.client.1.smithi078.stdout:0/271: write d2/f26 [869273,102149] 0 2024-04-26T04:52:47.992 INFO:tasks.workunit.client.1.smithi078.stdout:5/171: creat d7/d29/f3a x:0 0 0 2024-04-26T04:52:47.992 INFO:tasks.workunit.client.1.smithi078.stdout:4/246: write d0/d25/d20/f2a [1707547,82968] 0 2024-04-26T04:52:47.992 INFO:tasks.workunit.client.1.smithi078.stdout:6/266: write d2/d1a/d5/f8 [7386137,35376] 0 2024-04-26T04:52:47.994 INFO:tasks.workunit.client.1.smithi078.stdout:0/272: dread d2/d1e/f2a [0,4194304] 0 2024-04-26T04:52:47.995 INFO:tasks.workunit.client.1.smithi078.stdout:5/172: creat d7/d18/d32/f3b x:0 0 0 2024-04-26T04:52:47.997 INFO:tasks.workunit.client.1.smithi078.stdout:6/267: creat d2/d1a/f50 x:0 0 0 2024-04-26T04:52:47.998 INFO:tasks.workunit.client.1.smithi078.stdout:0/273: creat d2/d9/f54 x:0 0 0 2024-04-26T04:52:47.999 INFO:tasks.workunit.client.1.smithi078.stdout:5/173: symlink d7/d21/d2f/l3c 0 2024-04-26T04:52:47.999 INFO:tasks.workunit.client.1.smithi078.stdout:4/247: dread d0/fc [0,4194304] 0 2024-04-26T04:52:48.000 INFO:tasks.workunit.client.1.smithi078.stdout:0/274: mknod d2/d9/d31/d4a/c55 0 2024-04-26T04:52:48.002 INFO:tasks.workunit.client.1.smithi078.stdout:5/174: unlink d7/d18/f2d 0 2024-04-26T04:52:48.002 INFO:tasks.workunit.client.1.smithi078.stdout:0/275: write d2/d9/f20 [1318351,126321] 0 2024-04-26T04:52:48.003 INFO:tasks.workunit.client.1.smithi078.stdout:6/268: dread d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:48.003 INFO:tasks.workunit.client.1.smithi078.stdout:4/248: mkdir d0/d25/d20/d31/d33/d4c/d62 0 2024-04-26T04:52:48.004 INFO:tasks.workunit.client.1.smithi078.stdout:5/175: write d7/d18/f27 [4068194,94530] 0 2024-04-26T04:52:48.004 INFO:tasks.workunit.client.1.smithi078.stdout:0/276: creat d2/d1e/f56 x:0 0 0 2024-04-26T04:52:48.005 INFO:tasks.workunit.client.1.smithi078.stdout:4/249: read d0/f13 [120391,74961] 0 2024-04-26T04:52:48.008 INFO:tasks.workunit.client.1.smithi078.stdout:0/277: creat d2/f57 x:0 0 0 2024-04-26T04:52:48.008 INFO:tasks.workunit.client.1.smithi078.stdout:4/250: symlink d0/de/l63 0 2024-04-26T04:52:48.014 INFO:tasks.workunit.client.1.smithi078.stdout:4/251: rename d0/d25/f54 to d0/de/d22/d26/f64 0 2024-04-26T04:52:48.051 INFO:tasks.workunit.client.1.smithi078.stdout:6/269: truncate d2/f14 30991 0 2024-04-26T04:52:48.051 INFO:tasks.workunit.client.1.smithi078.stdout:0/278: truncate d2/f1d 236067 0 2024-04-26T04:52:48.051 INFO:tasks.workunit.client.1.smithi078.stdout:4/252: read - d0/de/d22/d26/d2e/f50 zero size 2024-04-26T04:52:48.053 INFO:tasks.workunit.client.1.smithi078.stdout:0/279: unlink d2/d9/f17 0 2024-04-26T04:52:48.053 INFO:tasks.workunit.client.1.smithi078.stdout:4/253: dread d0/d25/d20/d31/d47/f49 [0,4194304] 0 2024-04-26T04:52:48.055 INFO:tasks.workunit.client.1.smithi078.stdout:0/280: fdatasync f1 0 2024-04-26T04:52:48.056 INFO:tasks.workunit.client.1.smithi078.stdout:6/270: link d2/c3b d2/d1a/d5/d3a/d42/c51 0 2024-04-26T04:52:48.056 INFO:tasks.workunit.client.1.smithi078.stdout:0/281: stat d2/f3f 0 2024-04-26T04:52:48.060 INFO:tasks.workunit.client.1.smithi078.stdout:4/254: dwrite d0/d25/d20/d31/d33/d44/f3d [4194304,4194304] 0 2024-04-26T04:52:48.063 INFO:tasks.workunit.client.1.smithi078.stdout:4/255: dread - d0/d25/d20/d31/d33/f4d zero size 2024-04-26T04:52:48.064 INFO:tasks.workunit.client.1.smithi078.stdout:6/271: dwrite d2/de/d27/f34 [0,4194304] 0 2024-04-26T04:52:48.064 INFO:tasks.workunit.client.1.smithi078.stdout:0/282: dwrite d2/f10 [0,4194304] 0 2024-04-26T04:52:48.065 INFO:tasks.workunit.client.1.smithi078.stdout:0/283: fsync d2/d9/f30 0 2024-04-26T04:52:48.067 INFO:tasks.workunit.client.1.smithi078.stdout:0/284: truncate d2/f26 1657859 0 2024-04-26T04:52:48.068 INFO:tasks.workunit.client.1.smithi078.stdout:0/285: fdatasync d2/f3f 0 2024-04-26T04:52:48.069 INFO:tasks.workunit.client.1.smithi078.stdout:0/286: write d2/f14 [651202,12486] 0 2024-04-26T04:52:48.070 INFO:tasks.workunit.client.1.smithi078.stdout:4/256: symlink d0/de/d22/d27/d56/d5a/l65 0 2024-04-26T04:52:48.072 INFO:tasks.workunit.client.1.smithi078.stdout:0/287: mknod d2/d9/d31/c58 0 2024-04-26T04:52:48.075 INFO:tasks.workunit.client.1.smithi078.stdout:0/288: symlink d2/d9/d31/d4a/l59 0 2024-04-26T04:52:48.076 INFO:tasks.workunit.client.1.smithi078.stdout:0/289: read d2/d9/f3e [25197,94912] 0 2024-04-26T04:52:48.078 INFO:tasks.workunit.client.1.smithi078.stdout:0/290: dread - d2/d9/d31/d4a/f4c zero size 2024-04-26T04:52:48.079 INFO:tasks.workunit.client.1.smithi078.stdout:4/257: dwrite d0/d25/d20/d31/d47/f49 [0,4194304] 0 2024-04-26T04:52:48.101 INFO:tasks.workunit.client.1.smithi078.stdout:8/254: sync 2024-04-26T04:52:48.101 INFO:tasks.workunit.client.1.smithi078.stdout:9/184: sync 2024-04-26T04:52:48.103 INFO:tasks.workunit.client.1.smithi078.stdout:8/255: mknod d0/d19/c53 0 2024-04-26T04:52:48.105 INFO:tasks.workunit.client.1.smithi078.stdout:8/256: dread d0/d19/d52/f24 [0,4194304] 0 2024-04-26T04:52:48.107 INFO:tasks.workunit.client.1.smithi078.stdout:8/257: write d0/d19/d52/d35/f4e [421399,101500] 0 2024-04-26T04:52:48.108 INFO:tasks.workunit.client.1.smithi078.stdout:9/185: dwrite dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:48.109 INFO:tasks.workunit.client.1.smithi078.stdout:0/291: truncate d2/f15 1140442 0 2024-04-26T04:52:48.109 INFO:tasks.workunit.client.1.smithi078.stdout:4/258: truncate d0/d25/d20/d31/d33/f5b 515615 0 2024-04-26T04:52:48.110 INFO:tasks.workunit.client.1.smithi078.stdout:9/186: write dc/df/d1a/f1c [3095457,103945] 0 2024-04-26T04:52:48.111 INFO:tasks.workunit.client.1.smithi078.stdout:0/292: mkdir d2/d1e/d5a 0 2024-04-26T04:52:48.114 INFO:tasks.workunit.client.1.smithi078.stdout:4/259: mknod d0/d25/d20/c66 0 2024-04-26T04:52:48.115 INFO:tasks.workunit.client.1.smithi078.stdout:9/187: rmdir dc/df/d1a 39 2024-04-26T04:52:48.118 INFO:tasks.workunit.client.1.smithi078.stdout:0/293: rename d2/d1e/l3c to d2/d9/d31/d4a/l5b 0 2024-04-26T04:52:48.118 INFO:tasks.workunit.client.1.smithi078.stdout:4/260: rmdir d0/d25/d20/d31 39 2024-04-26T04:52:48.119 INFO:tasks.workunit.client.1.smithi078.stdout:9/188: unlink dc/c10 0 2024-04-26T04:52:48.121 INFO:tasks.workunit.client.1.smithi078.stdout:0/294: dread d2/ff [0,4194304] 0 2024-04-26T04:52:48.121 INFO:tasks.workunit.client.1.smithi078.stdout:9/189: fdatasync dc/df/f1e 0 2024-04-26T04:52:48.122 INFO:tasks.workunit.client.1.smithi078.stdout:4/261: symlink d0/d25/d20/d31/d33/d4c/d62/l67 0 2024-04-26T04:52:48.123 INFO:tasks.workunit.client.1.smithi078.stdout:0/295: read d2/d9/d18/f1c [2688405,125280] 0 2024-04-26T04:52:48.124 INFO:tasks.workunit.client.1.smithi078.stdout:4/262: write d0/d25/d20/f1b [3875968,78021] 0 2024-04-26T04:52:48.125 INFO:tasks.workunit.client.1.smithi078.stdout:9/190: mkdir dc/df/d37 0 2024-04-26T04:52:48.126 INFO:tasks.workunit.client.1.smithi078.stdout:0/296: write d2/d9/d18/f13 [3124787,58155] 0 2024-04-26T04:52:48.132 INFO:tasks.workunit.client.1.smithi078.stdout:9/191: unlink dc/df/d1a/d22/d2a/l2b 0 2024-04-26T04:52:48.133 INFO:tasks.workunit.client.1.smithi078.stdout:9/192: fsync dc/f26 0 2024-04-26T04:52:48.133 INFO:tasks.workunit.client.1.smithi078.stdout:9/193: chown dc/df/f1d 49882 1 2024-04-26T04:52:48.135 INFO:tasks.workunit.client.1.smithi078.stdout:0/297: rmdir d2/d1e 39 2024-04-26T04:52:48.135 INFO:tasks.workunit.client.1.smithi078.stdout:0/298: fsync d2/f26 0 2024-04-26T04:52:48.138 INFO:tasks.workunit.client.1.smithi078.stdout:0/299: mknod d2/d9/d31/c5c 0 2024-04-26T04:52:48.140 INFO:tasks.workunit.client.1.smithi078.stdout:4/263: dread d0/d25/d20/d31/d33/d44/f3d [0,4194304] 0 2024-04-26T04:52:48.140 INFO:tasks.workunit.client.1.smithi078.stdout:9/194: dwrite dc/df/d1a/f29 [0,4194304] 0 2024-04-26T04:52:48.141 INFO:tasks.workunit.client.1.smithi078.stdout:9/195: write dc/df/d27/f24 [738519,95767] 0 2024-04-26T04:52:48.142 INFO:tasks.workunit.client.1.smithi078.stdout:0/300: creat d2/d9/d31/d4a/f5d x:0 0 0 2024-04-26T04:52:48.143 INFO:tasks.workunit.client.1.smithi078.stdout:0/301: stat d2/d9/d31/c58 0 2024-04-26T04:52:48.144 INFO:tasks.workunit.client.1.smithi078.stdout:9/196: mkdir dc/df/d27/d38 0 2024-04-26T04:52:48.148 INFO:tasks.workunit.client.1.smithi078.stdout:9/197: creat dc/f39 x:0 0 0 2024-04-26T04:52:48.151 INFO:tasks.workunit.client.1.smithi078.stdout:4/264: dwrite d0/de/d22/d26/f64 [0,4194304] 0 2024-04-26T04:52:48.152 INFO:tasks.workunit.client.1.smithi078.stdout:9/198: rename dc/df/d17/f30 to dc/df/d17/f3a 0 2024-04-26T04:52:48.153 INFO:tasks.workunit.client.1.smithi078.stdout:0/302: dwrite d2/f21 [0,4194304] 0 2024-04-26T04:52:48.154 INFO:tasks.workunit.client.1.smithi078.stdout:4/265: dread - d0/de/d22/d27/d56/f5c zero size 2024-04-26T04:52:48.157 INFO:tasks.workunit.client.1.smithi078.stdout:9/199: mknod dc/df/d1a/d22/d2a/c3b 0 2024-04-26T04:52:48.158 INFO:tasks.workunit.client.1.smithi078.stdout:0/303: mknod d2/d39/c5e 0 2024-04-26T04:52:48.160 INFO:tasks.workunit.client.1.smithi078.stdout:4/266: write d0/de/d22/d27/d56/f5c [473416,8679] 0 2024-04-26T04:52:48.161 INFO:tasks.workunit.client.1.smithi078.stdout:4/267: readlink d0/l11 0 2024-04-26T04:52:48.161 INFO:tasks.workunit.client.1.smithi078.stdout:9/200: mkdir dc/df/d1a/d22/d2a/d3c 0 2024-04-26T04:52:48.167 INFO:tasks.workunit.client.1.smithi078.stdout:0/304: dwrite d2/f57 [0,4194304] 0 2024-04-26T04:52:48.169 INFO:tasks.workunit.client.1.smithi078.stdout:4/268: dread d0/d25/d20/d31/d33/d44/f59 [0,4194304] 0 2024-04-26T04:52:48.170 INFO:tasks.workunit.client.1.smithi078.stdout:0/305: truncate d2/ff 1140099 0 2024-04-26T04:52:48.170 INFO:tasks.workunit.client.1.smithi078.stdout:9/201: dwrite f1 [4194304,4194304] 0 2024-04-26T04:52:48.175 INFO:tasks.workunit.client.1.smithi078.stdout:2/204: sync 2024-04-26T04:52:48.178 INFO:tasks.workunit.client.1.smithi078.stdout:9/202: creat dc/f3d x:0 0 0 2024-04-26T04:52:48.178 INFO:tasks.workunit.client.1.smithi078.stdout:2/205: rmdir d8/d11/d14/d15 39 2024-04-26T04:52:48.181 INFO:tasks.workunit.client.1.smithi078.stdout:0/306: link d2/d9/l33 d2/l5f 0 2024-04-26T04:52:48.185 INFO:tasks.workunit.client.1.smithi078.stdout:2/206: creat d8/d11/d14/d15/d1a/f49 x:0 0 0 2024-04-26T04:52:48.191 INFO:tasks.workunit.client.1.smithi078.stdout:0/307: dwrite d2/f6 [0,4194304] 0 2024-04-26T04:52:48.191 INFO:tasks.workunit.client.1.smithi078.stdout:9/203: truncate dc/df/d1a/f20 1516742 0 2024-04-26T04:52:48.194 INFO:tasks.workunit.client.1.smithi078.stdout:9/204: write dc/df/d27/f13 [1014438,43610] 0 2024-04-26T04:52:48.195 INFO:tasks.workunit.client.1.smithi078.stdout:2/207: dwrite d8/d11/d14/d15/f39 [0,4194304] 0 2024-04-26T04:52:48.196 INFO:tasks.workunit.client.1.smithi078.stdout:0/308: mkdir d2/d9/d60 0 2024-04-26T04:52:48.196 INFO:tasks.workunit.client.1.smithi078.stdout:9/205: truncate dc/df/d1a/f1c 4735728 0 2024-04-26T04:52:48.198 INFO:tasks.workunit.client.1.smithi078.stdout:9/206: write dc/df/f28 [1466484,25236] 0 2024-04-26T04:52:48.199 INFO:tasks.workunit.client.1.smithi078.stdout:9/207: stat dc/f26 0 2024-04-26T04:52:48.199 INFO:tasks.workunit.client.1.smithi078.stdout:0/309: chown d2/l32 1403695959 1 2024-04-26T04:52:48.199 INFO:tasks.workunit.client.1.smithi078.stdout:9/208: fsync f1 0 2024-04-26T04:52:48.201 INFO:tasks.workunit.client.1.smithi078.stdout:9/209: write dc/df/f36 [340220,86911] 0 2024-04-26T04:52:48.205 INFO:tasks.workunit.client.1.smithi078.stdout:0/310: creat d2/d1e/f61 x:0 0 0 2024-04-26T04:52:48.206 INFO:tasks.workunit.client.1.smithi078.stdout:0/311: stat d2/d1e/l47 0 2024-04-26T04:52:48.208 INFO:tasks.workunit.client.1.smithi078.stdout:9/210: rename dc/df/d1a/d22/d2a/d3c to dc/df/d1a/d3e 0 2024-04-26T04:52:48.210 INFO:tasks.workunit.client.1.smithi078.stdout:0/312: dread d2/d9/d31/f50 [0,4194304] 0 2024-04-26T04:52:48.212 INFO:tasks.workunit.client.1.smithi078.stdout:9/211: link dc/df/d1a/d22/l31 dc/df/d27/l3f 0 2024-04-26T04:52:48.215 INFO:tasks.workunit.client.1.smithi078.stdout:9/212: mknod dc/df/d27/c40 0 2024-04-26T04:52:48.216 INFO:tasks.workunit.client.1.smithi078.stdout:0/313: dwrite d2/d9/f54 [0,4194304] 0 2024-04-26T04:52:48.219 INFO:tasks.workunit.client.1.smithi078.stdout:0/314: creat d2/d9/d18/f62 x:0 0 0 2024-04-26T04:52:48.234 INFO:tasks.workunit.client.1.smithi078.stdout:0/315: dwrite d2/f1d [0,4194304] 0 2024-04-26T04:52:48.235 INFO:tasks.workunit.client.1.smithi078.stdout:0/316: dread - d2/f48 zero size 2024-04-26T04:52:48.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/317: symlink d2/d9/d31/l63 0 2024-04-26T04:52:48.240 INFO:tasks.workunit.client.1.smithi078.stdout:0/318: link d2/d9/d31/d4a/f4c d2/d9/d31/d4a/f64 0 2024-04-26T04:52:48.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:47 smithi077 ceph-mon[26311]: pgmap v13: 65 pgs: 65 active+clean; 329 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 16 MiB/s wr, 269 op/s 2024-04-26T04:52:48.242 INFO:tasks.workunit.client.1.smithi078.stdout:0/319: mknod d2/d9/d31/c65 0 2024-04-26T04:52:48.243 INFO:tasks.workunit.client.1.smithi078.stdout:0/320: stat f1 0 2024-04-26T04:52:48.246 INFO:tasks.workunit.client.1.smithi078.stdout:0/321: link d2/d9/d31/d4a/f4c d2/d9/d31/f66 0 2024-04-26T04:52:48.248 INFO:tasks.workunit.client.1.smithi078.stdout:0/322: dread d2/d9/d31/f50 [0,4194304] 0 2024-04-26T04:52:48.250 INFO:tasks.workunit.client.1.smithi078.stdout:0/323: readlink d2/d9/l38 0 2024-04-26T04:52:48.252 INFO:tasks.workunit.client.1.smithi078.stdout:0/324: creat d2/d1e/f67 x:0 0 0 2024-04-26T04:52:48.256 INFO:tasks.workunit.client.1.smithi078.stdout:0/325: dread d2/d9/d31/f50 [0,4194304] 0 2024-04-26T04:52:48.259 INFO:tasks.workunit.client.1.smithi078.stdout:0/326: creat d2/d9/d60/f68 x:0 0 0 2024-04-26T04:52:48.260 INFO:tasks.workunit.client.1.smithi078.stdout:0/327: fdatasync d2/f3a 0 2024-04-26T04:52:48.261 INFO:tasks.workunit.client.1.smithi078.stdout:0/328: creat d2/d1e/f69 x:0 0 0 2024-04-26T04:52:48.262 INFO:tasks.workunit.client.1.smithi078.stdout:0/329: mkdir d2/d9/d60/d6a 0 2024-04-26T04:52:48.264 INFO:tasks.workunit.client.1.smithi078.stdout:0/330: mkdir d2/d9/d6b 0 2024-04-26T04:52:48.265 INFO:tasks.workunit.client.1.smithi078.stdout:0/331: truncate d2/f3a 557760 0 2024-04-26T04:52:48.265 INFO:tasks.workunit.client.1.smithi078.stdout:0/332: stat d2/f10 0 2024-04-26T04:52:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:47 smithi078 ceph-mon[33746]: pgmap v13: 65 pgs: 65 active+clean; 329 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 16 MiB/s wr, 269 op/s 2024-04-26T04:52:48.268 INFO:tasks.workunit.client.1.smithi078.stdout:0/333: rename d2/d9/d18/f3b to d2/f6c 0 2024-04-26T04:52:48.268 INFO:tasks.workunit.client.1.smithi078.stdout:0/334: write d2/f26 [582281,65573] 0 2024-04-26T04:52:48.269 INFO:tasks.workunit.client.1.smithi078.stdout:0/335: write d2/f1d [904495,73999] 0 2024-04-26T04:52:48.271 INFO:tasks.workunit.client.1.smithi078.stdout:0/336: creat d2/d9/d60/d6a/f6d x:0 0 0 2024-04-26T04:52:48.276 INFO:tasks.workunit.client.1.smithi078.stdout:0/337: dwrite d2/d1e/f67 [0,4194304] 0 2024-04-26T04:52:48.279 INFO:tasks.workunit.client.1.smithi078.stdout:0/338: fdatasync d2/d9/d18/f49 0 2024-04-26T04:52:48.281 INFO:tasks.workunit.client.1.smithi078.stdout:0/339: write d2/d9/d31/f50 [3125983,83229] 0 2024-04-26T04:52:48.285 INFO:tasks.workunit.client.1.smithi078.stdout:0/340: getdents d2/d9/d60/d6a 0 2024-04-26T04:52:48.288 INFO:tasks.workunit.client.1.smithi078.stdout:0/341: mknod d2/d1e/d5a/c6e 0 2024-04-26T04:52:48.288 INFO:tasks.workunit.client.1.smithi078.stdout:3/199: sync 2024-04-26T04:52:48.288 INFO:tasks.workunit.client.1.smithi078.stdout:6/272: sync 2024-04-26T04:52:48.290 INFO:tasks.workunit.client.1.smithi078.stdout:6/273: mkdir d2/d52 0 2024-04-26T04:52:48.291 INFO:tasks.workunit.client.1.smithi078.stdout:6/274: creat d2/f53 x:0 0 0 2024-04-26T04:52:48.293 INFO:tasks.workunit.client.1.smithi078.stdout:3/200: dwrite d16/f22 [0,4194304] 0 2024-04-26T04:52:48.294 INFO:tasks.workunit.client.1.smithi078.stdout:6/275: mknod d2/d1a/d5/da/c54 0 2024-04-26T04:52:48.294 INFO:tasks.workunit.client.1.smithi078.stdout:3/201: truncate f9 4577293 0 2024-04-26T04:52:48.303 INFO:tasks.workunit.client.1.smithi078.stdout:3/202: truncate f13 2350102 0 2024-04-26T04:52:48.304 INFO:tasks.workunit.client.1.smithi078.stdout:3/203: write d16/d26/d2e/f45 [611308,118696] 0 2024-04-26T04:52:48.305 INFO:tasks.workunit.client.1.smithi078.stdout:6/276: truncate d2/d1a/d5/f8 5558333 0 2024-04-26T04:52:48.307 INFO:tasks.workunit.client.1.smithi078.stdout:3/204: write d16/d26/f29 [754260,25699] 0 2024-04-26T04:52:48.307 INFO:tasks.workunit.client.1.smithi078.stdout:0/342: dread d2/d9/f1a [0,4194304] 0 2024-04-26T04:52:48.308 INFO:tasks.workunit.client.1.smithi078.stdout:6/277: rmdir d2/d1a/d5/da 39 2024-04-26T04:52:48.309 INFO:tasks.workunit.client.1.smithi078.stdout:6/278: creat d2/de/d27/f55 x:0 0 0 2024-04-26T04:52:48.311 INFO:tasks.workunit.client.1.smithi078.stdout:3/205: dread d16/d26/d2b/d31/f34 [0,4194304] 0 2024-04-26T04:52:48.314 INFO:tasks.workunit.client.1.smithi078.stdout:0/343: link d2/d9/d31/l4b d2/d9/d31/l6f 0 2024-04-26T04:52:48.315 INFO:tasks.workunit.client.1.smithi078.stdout:6/279: fsync d2/de/d27/f55 0 2024-04-26T04:52:48.315 INFO:tasks.workunit.client.1.smithi078.stdout:3/206: creat d16/d3c/d46/d49/f4e x:0 0 0 2024-04-26T04:52:48.316 INFO:tasks.workunit.client.1.smithi078.stdout:6/280: read - d2/d1a/f50 zero size 2024-04-26T04:52:48.318 INFO:tasks.workunit.client.1.smithi078.stdout:6/281: creat d2/d1a/d5/d2a/f56 x:0 0 0 2024-04-26T04:52:48.319 INFO:tasks.workunit.client.1.smithi078.stdout:0/344: creat d2/f70 x:0 0 0 2024-04-26T04:52:48.321 INFO:tasks.workunit.client.1.smithi078.stdout:6/282: creat d2/d49/f57 x:0 0 0 2024-04-26T04:52:48.322 INFO:tasks.workunit.client.1.smithi078.stdout:3/207: dwrite d16/d26/f2a [0,4194304] 0 2024-04-26T04:52:48.323 INFO:tasks.workunit.client.1.smithi078.stdout:6/283: write d2/d1a/f50 [645242,21979] 0 2024-04-26T04:52:48.323 INFO:tasks.workunit.client.1.smithi078.stdout:0/345: link d2/d1e/f56 d2/d1e/d5a/f71 0 2024-04-26T04:52:48.324 INFO:tasks.workunit.client.1.smithi078.stdout:3/208: creat d16/d26/d35/f4f x:0 0 0 2024-04-26T04:52:48.328 INFO:tasks.workunit.client.1.smithi078.stdout:1/273: sync 2024-04-26T04:52:48.328 INFO:tasks.workunit.client.1.smithi078.stdout:7/210: sync 2024-04-26T04:52:48.329 INFO:tasks.workunit.client.1.smithi078.stdout:6/284: dread d2/d1a/f50 [0,4194304] 0 2024-04-26T04:52:48.329 INFO:tasks.workunit.client.1.smithi078.stdout:3/209: creat d16/f50 x:0 0 0 2024-04-26T04:52:48.330 INFO:tasks.workunit.client.1.smithi078.stdout:1/274: dread - d7/d1e/f54 zero size 2024-04-26T04:52:48.333 INFO:tasks.workunit.client.1.smithi078.stdout:6/285: mkdir d2/de/d27/d58 0 2024-04-26T04:52:48.333 INFO:tasks.workunit.client.1.smithi078.stdout:3/210: creat d16/d26/d2e/f51 x:0 0 0 2024-04-26T04:52:48.334 INFO:tasks.workunit.client.1.smithi078.stdout:0/346: dwrite d2/f6c [0,4194304] 0 2024-04-26T04:52:48.334 INFO:tasks.workunit.client.1.smithi078.stdout:1/275: creat d7/d39/db/d37/d58/f5e x:0 0 0 2024-04-26T04:52:48.335 INFO:tasks.workunit.client.1.smithi078.stdout:3/211: fdatasync d16/d26/f37 0 2024-04-26T04:52:48.336 INFO:tasks.workunit.client.1.smithi078.stdout:0/347: chown d2/d1e/l2b 21654 1 2024-04-26T04:52:48.336 INFO:tasks.workunit.client.1.smithi078.stdout:7/211: truncate d9/da/d11/d13/d1a/f29 6944539 0 2024-04-26T04:52:48.338 INFO:tasks.workunit.client.1.smithi078.stdout:1/276: rmdir d7/d39/d4c 39 2024-04-26T04:52:48.339 INFO:tasks.workunit.client.1.smithi078.stdout:0/348: write d2/f21 [2848127,85702] 0 2024-04-26T04:52:48.339 INFO:tasks.workunit.client.1.smithi078.stdout:6/286: rename d2/d1a/d5/f32 to d2/d1a/d5/d3a/d42/f59 0 2024-04-26T04:52:48.341 INFO:tasks.workunit.client.1.smithi078.stdout:6/287: dread - d2/f16 zero size 2024-04-26T04:52:48.341 INFO:tasks.workunit.client.1.smithi078.stdout:0/349: chown d2/d9/d31/d4a 503203 1 2024-04-26T04:52:48.343 INFO:tasks.workunit.client.1.smithi078.stdout:7/212: truncate d9/da/d11/d13/d1a/d2d/f3d 138145 0 2024-04-26T04:52:48.343 INFO:tasks.workunit.client.1.smithi078.stdout:0/350: stat d2/c3d 0 2024-04-26T04:52:48.343 INFO:tasks.workunit.client.1.smithi078.stdout:0/351: chown d2/f10 409845 1 2024-04-26T04:52:48.344 INFO:tasks.workunit.client.1.smithi078.stdout:1/277: getdents d7/d39/dd/d20/d53 0 2024-04-26T04:52:48.344 INFO:tasks.workunit.client.1.smithi078.stdout:6/288: creat d2/d1a/d5/d3a/d42/f5a x:0 0 0 2024-04-26T04:52:48.346 INFO:tasks.workunit.client.1.smithi078.stdout:0/352: symlink d2/d9/d6b/l72 0 2024-04-26T04:52:48.347 INFO:tasks.workunit.client.1.smithi078.stdout:0/353: stat d2/d9/c35 0 2024-04-26T04:52:48.348 INFO:tasks.workunit.client.1.smithi078.stdout:6/289: write d2/de/d27/f37 [2943726,95936] 0 2024-04-26T04:52:48.348 INFO:tasks.workunit.client.1.smithi078.stdout:6/290: chown d2 4 1 2024-04-26T04:52:48.350 INFO:tasks.workunit.client.1.smithi078.stdout:0/354: mkdir d2/d9/d31/d73 0 2024-04-26T04:52:48.351 INFO:tasks.workunit.client.1.smithi078.stdout:6/291: read - d2/d1a/d5/d3a/d42/f59 zero size 2024-04-26T04:52:48.351 INFO:tasks.workunit.client.1.smithi078.stdout:1/278: dwrite d7/ff [0,4194304] 0 2024-04-26T04:52:48.353 INFO:tasks.workunit.client.1.smithi078.stdout:7/213: read f5 [4279678,1820] 0 2024-04-26T04:52:48.353 INFO:tasks.workunit.client.1.smithi078.stdout:3/212: truncate f6 2611972 0 2024-04-26T04:52:48.354 INFO:tasks.workunit.client.1.smithi078.stdout:0/355: mkdir d2/d1e/d5a/d74 0 2024-04-26T04:52:48.357 INFO:tasks.workunit.client.1.smithi078.stdout:1/279: rename d7/f1c to d7/d2e/f5f 0 2024-04-26T04:52:48.358 INFO:tasks.workunit.client.1.smithi078.stdout:7/214: rmdir d9/da/d11 39 2024-04-26T04:52:48.360 INFO:tasks.workunit.client.1.smithi078.stdout:6/292: mkdir d2/de/d27/d58/d5b 0 2024-04-26T04:52:48.363 INFO:tasks.workunit.client.1.smithi078.stdout:1/280: rename d7 to d7/d60 22 2024-04-26T04:52:48.365 INFO:tasks.workunit.client.1.smithi078.stdout:3/213: dwrite d16/d3c/d46/d49/f4b [0,4194304] 0 2024-04-26T04:52:48.366 INFO:tasks.workunit.client.1.smithi078.stdout:7/215: write d9/da/d11/f2f [55391,88968] 0 2024-04-26T04:52:48.368 INFO:tasks.workunit.client.1.smithi078.stdout:3/214: fdatasync d16/f39 0 2024-04-26T04:52:48.368 INFO:tasks.workunit.client.1.smithi078.stdout:7/216: readlink d9/d16/l1d 0 2024-04-26T04:52:48.369 INFO:tasks.workunit.client.1.smithi078.stdout:0/356: creat d2/d9/f75 x:0 0 0 2024-04-26T04:52:48.370 INFO:tasks.workunit.client.1.smithi078.stdout:1/281: rename d7/d39/dd/f2d to d7/d39/db/d49/f61 0 2024-04-26T04:52:48.371 INFO:tasks.workunit.client.1.smithi078.stdout:6/293: getdents d2 0 2024-04-26T04:52:48.372 INFO:tasks.workunit.client.1.smithi078.stdout:3/215: unlink f12 0 2024-04-26T04:52:48.372 INFO:tasks.workunit.client.1.smithi078.stdout:7/217: dread d9/da/d11/d13/d1a/d2d/f3d [0,4194304] 0 2024-04-26T04:52:48.373 INFO:tasks.workunit.client.1.smithi078.stdout:6/294: chown d2/de/d27/d58 134091673 1 2024-04-26T04:52:48.374 INFO:tasks.workunit.client.1.smithi078.stdout:0/357: unlink d2/d9/d31/d4a/f4c 0 2024-04-26T04:52:48.374 INFO:tasks.workunit.client.1.smithi078.stdout:3/216: fsync d16/d26/d2e/f42 0 2024-04-26T04:52:48.377 INFO:tasks.workunit.client.1.smithi078.stdout:7/218: unlink d9/da/d11/f2a 0 2024-04-26T04:52:48.381 INFO:tasks.workunit.client.1.smithi078.stdout:0/358: dwrite d2/f29 [0,4194304] 0 2024-04-26T04:52:48.381 INFO:tasks.workunit.client.1.smithi078.stdout:7/219: creat d9/d16/f44 x:0 0 0 2024-04-26T04:52:48.382 INFO:tasks.workunit.client.1.smithi078.stdout:1/282: link d7/d39/dd/d20/f50 d7/d39/f62 0 2024-04-26T04:52:48.385 INFO:tasks.workunit.client.1.smithi078.stdout:0/359: chown d2/d9/d31/l6f 25224 1 2024-04-26T04:52:48.387 INFO:tasks.workunit.client.1.smithi078.stdout:1/283: creat d7/d2e/f63 x:0 0 0 2024-04-26T04:52:48.388 INFO:tasks.workunit.client.1.smithi078.stdout:0/360: chown d2/d9/d6b/l72 463225763 1 2024-04-26T04:52:48.390 INFO:tasks.workunit.client.1.smithi078.stdout:6/295: write d2/d1a/d5/d11/f1c [1190255,12222] 0 2024-04-26T04:52:48.390 INFO:tasks.workunit.client.1.smithi078.stdout:7/220: dwrite d9/d16/f44 [0,4194304] 0 2024-04-26T04:52:48.391 INFO:tasks.workunit.client.1.smithi078.stdout:0/361: write d2/d39/f52 [876004,38224] 0 2024-04-26T04:52:48.392 INFO:tasks.workunit.client.1.smithi078.stdout:7/221: write d9/da/d20/f31 [4316312,11500] 0 2024-04-26T04:52:48.393 INFO:tasks.workunit.client.1.smithi078.stdout:7/222: stat f7 0 2024-04-26T04:52:48.394 INFO:tasks.workunit.client.1.smithi078.stdout:7/223: chown d9/da/d11/f12 124 1 2024-04-26T04:52:48.395 INFO:tasks.workunit.client.1.smithi078.stdout:6/296: mkdir d2/d1a/d5c 0 2024-04-26T04:52:48.395 INFO:tasks.workunit.client.1.smithi078.stdout:0/362: creat d2/d9/d6b/f76 x:0 0 0 2024-04-26T04:52:48.398 INFO:tasks.workunit.client.1.smithi078.stdout:3/217: write fd [1291653,39901] 0 2024-04-26T04:52:48.398 INFO:tasks.workunit.client.1.smithi078.stdout:6/297: fsync d2/d1a/d5/d11/f1c 0 2024-04-26T04:52:48.400 INFO:tasks.workunit.client.1.smithi078.stdout:3/218: dread - d16/d26/d35/f4f zero size 2024-04-26T04:52:48.400 INFO:tasks.workunit.client.1.smithi078.stdout:3/219: write d16/d26/d2e/f45 [2716611,15623] 0 2024-04-26T04:52:48.401 INFO:tasks.workunit.client.1.smithi078.stdout:9/213: sync 2024-04-26T04:52:48.401 INFO:tasks.workunit.client.1.smithi078.stdout:8/258: sync 2024-04-26T04:52:48.403 INFO:tasks.workunit.client.1.smithi078.stdout:7/224: creat d9/da/d11/f45 x:0 0 0 2024-04-26T04:52:48.404 INFO:tasks.workunit.client.1.smithi078.stdout:2/208: sync 2024-04-26T04:52:48.404 INFO:tasks.workunit.client.1.smithi078.stdout:5/176: sync 2024-04-26T04:52:48.405 INFO:tasks.workunit.client.1.smithi078.stdout:4/269: sync 2024-04-26T04:52:48.405 INFO:tasks.workunit.client.1.smithi078.stdout:7/225: fsync d9/d16/f26 0 2024-04-26T04:52:48.405 INFO:tasks.workunit.client.1.smithi078.stdout:3/220: creat d16/d26/d2b/d31/f52 x:0 0 0 2024-04-26T04:52:48.406 INFO:tasks.workunit.client.1.smithi078.stdout:9/214: unlink dc/df/d1a/f35 0 2024-04-26T04:52:48.407 INFO:tasks.workunit.client.1.smithi078.stdout:1/284: dwrite d7/d39/dd/f55 [0,4194304] 0 2024-04-26T04:52:48.410 INFO:tasks.workunit.client.1.smithi078.stdout:8/259: truncate d0/f23 641371 0 2024-04-26T04:52:48.412 INFO:tasks.workunit.client.1.smithi078.stdout:7/226: write d9/da/d20/f21 [1676280,129434] 0 2024-04-26T04:52:48.414 INFO:tasks.workunit.client.1.smithi078.stdout:2/209: rename d8/d11/f46 to d8/d11/d14/d15/f4a 0 2024-04-26T04:52:48.417 INFO:tasks.workunit.client.1.smithi078.stdout:3/221: dwrite d16/f36 [0,4194304] 0 2024-04-26T04:52:48.418 INFO:tasks.workunit.client.1.smithi078.stdout:5/177: getdents d7/d18/d1c 0 2024-04-26T04:52:48.418 INFO:tasks.workunit.client.1.smithi078.stdout:3/222: write d16/d26/f48 [516711,50016] 0 2024-04-26T04:52:48.420 INFO:tasks.workunit.client.1.smithi078.stdout:3/223: chown d16/d3c/d46/d49 3238 1 2024-04-26T04:52:48.422 INFO:tasks.workunit.client.1.smithi078.stdout:8/260: symlink d0/d19/d25/d36/l54 0 2024-04-26T04:52:48.423 INFO:tasks.workunit.client.1.smithi078.stdout:1/285: write d7/fc [1460592,18227] 0 2024-04-26T04:52:48.424 INFO:tasks.workunit.client.1.smithi078.stdout:2/210: dwrite d8/d11/d14/f1c [0,4194304] 0 2024-04-26T04:52:48.424 INFO:tasks.workunit.client.1.smithi078.stdout:7/227: symlink d9/da/d20/l46 0 2024-04-26T04:52:48.425 INFO:tasks.workunit.client.1.smithi078.stdout:2/211: fsync d8/d11/d14/d15/f32 0 2024-04-26T04:52:48.427 INFO:tasks.workunit.client.1.smithi078.stdout:4/270: rename d0/d2f/f4b to d0/d25/d20/d31/d33/d44/f68 0 2024-04-26T04:52:48.429 INFO:tasks.workunit.client.1.smithi078.stdout:6/298: link d2/d1a/d5/da/l20 d2/d1a/l5d 0 2024-04-26T04:52:48.431 INFO:tasks.workunit.client.1.smithi078.stdout:5/178: unlink d7/d29/c2a 0 2024-04-26T04:52:48.432 INFO:tasks.workunit.client.1.smithi078.stdout:7/228: dwrite d9/da/d11/f2f [0,4194304] 0 2024-04-26T04:52:48.433 INFO:tasks.workunit.client.1.smithi078.stdout:7/229: chown d9/da/d20/f21 76 1 2024-04-26T04:52:48.434 INFO:tasks.workunit.client.1.smithi078.stdout:9/215: rmdir dc/df/d27/d38 0 2024-04-26T04:52:48.436 INFO:tasks.workunit.client.1.smithi078.stdout:9/216: dread f8 [0,4194304] 0 2024-04-26T04:52:48.437 INFO:tasks.workunit.client.1.smithi078.stdout:1/286: readlink d7/d39/dd/d20/l29 0 2024-04-26T04:52:48.437 INFO:tasks.workunit.client.1.smithi078.stdout:9/217: chown dc/df/d27/c40 229546 1 2024-04-26T04:52:48.438 INFO:tasks.workunit.client.1.smithi078.stdout:9/218: write dc/df/f28 [4039295,50928] 0 2024-04-26T04:52:48.441 INFO:tasks.workunit.client.1.smithi078.stdout:4/271: mkdir d0/d25/d20/d31/d33/d44/d69 0 2024-04-26T04:52:48.441 INFO:tasks.workunit.client.1.smithi078.stdout:4/272: stat d0/de/d22 0 2024-04-26T04:52:48.444 INFO:tasks.workunit.client.1.smithi078.stdout:7/230: rename d9/da/d11/d13/d1a/f3b to d9/d16/d41/f47 0 2024-04-26T04:52:48.445 INFO:tasks.workunit.client.1.smithi078.stdout:1/287: symlink d7/d39/dd/d20/l64 0 2024-04-26T04:52:48.446 INFO:tasks.workunit.client.1.smithi078.stdout:9/219: mknod dc/df/d27/c41 0 2024-04-26T04:52:48.447 INFO:tasks.workunit.client.1.smithi078.stdout:2/212: mknod d8/d11/d14/d15/d1a/c4b 0 2024-04-26T04:52:48.448 INFO:tasks.workunit.client.1.smithi078.stdout:6/299: creat d2/d1a/d41/f5e x:0 0 0 2024-04-26T04:52:48.449 INFO:tasks.workunit.client.1.smithi078.stdout:5/179: creat d7/d18/d32/d37/f3d x:0 0 0 2024-04-26T04:52:48.449 INFO:tasks.workunit.client.1.smithi078.stdout:4/273: creat d0/de/d22/d27/d56/f6a x:0 0 0 2024-04-26T04:52:48.450 INFO:tasks.workunit.client.1.smithi078.stdout:2/213: chown d8/d11/d14/d15/d1a 2 1 2024-04-26T04:52:48.451 INFO:tasks.workunit.client.1.smithi078.stdout:6/300: fsync d2/de/f2b 0 2024-04-26T04:52:48.452 INFO:tasks.workunit.client.1.smithi078.stdout:4/274: dread - d0/d25/d20/f4f zero size 2024-04-26T04:52:48.452 INFO:tasks.workunit.client.1.smithi078.stdout:1/288: dread d7/d39/dd/d20/f40 [0,4194304] 0 2024-04-26T04:52:48.454 INFO:tasks.workunit.client.1.smithi078.stdout:2/214: fsync d8/d11/d14/d15/f1b 0 2024-04-26T04:52:48.454 INFO:tasks.workunit.client.1.smithi078.stdout:9/220: dread dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:48.507 INFO:tasks.workunit.client.1.smithi078.stdout:1/289: creat d7/d39/db/d37/f65 x:0 0 0 2024-04-26T04:52:48.507 INFO:tasks.workunit.client.1.smithi078.stdout:4/275: rename d0/de/f1c to d0/d25/d20/d31/f6b 0 2024-04-26T04:52:48.508 INFO:tasks.workunit.client.1.smithi078.stdout:6/301: mkdir d2/de/d27/d4c/d5f 0 2024-04-26T04:52:48.508 INFO:tasks.workunit.client.1.smithi078.stdout:8/261: dread d0/f23 [0,4194304] 0 2024-04-26T04:52:48.509 INFO:tasks.workunit.client.1.smithi078.stdout:3/224: truncate f9 1212857 0 2024-04-26T04:52:48.509 INFO:tasks.workunit.client.1.smithi078.stdout:9/221: chown dc/df/f36 46440180 1 2024-04-26T04:52:48.509 INFO:tasks.workunit.client.1.smithi078.stdout:8/262: stat d0/d19/f1c 0 2024-04-26T04:52:48.510 INFO:tasks.workunit.client.1.smithi078.stdout:7/231: write d9/f10 [1002197,53271] 0 2024-04-26T04:52:48.511 INFO:tasks.workunit.client.1.smithi078.stdout:1/290: creat d7/d39/d17/f66 x:0 0 0 2024-04-26T04:52:48.511 INFO:tasks.workunit.client.1.smithi078.stdout:3/225: rmdir d16/d3c/d46/d49 39 2024-04-26T04:52:48.513 INFO:tasks.workunit.client.1.smithi078.stdout:4/276: write d0/d25/d20/d31/d33/d44/f59 [4392173,59068] 0 2024-04-26T04:52:48.515 INFO:tasks.workunit.client.1.smithi078.stdout:4/277: chown d0/d25/d20/d31/c43 723105 1 2024-04-26T04:52:48.515 INFO:tasks.workunit.client.1.smithi078.stdout:7/232: truncate d9/da/d11/d13/d1a/d2d/f3f 948419 0 2024-04-26T04:52:48.515 INFO:tasks.workunit.client.1.smithi078.stdout:8/263: mknod d0/d19/d52/d35/c55 0 2024-04-26T04:52:48.515 INFO:tasks.workunit.client.1.smithi078.stdout:7/233: chown d9/da/d20/f21 19 1 2024-04-26T04:52:48.520 INFO:tasks.workunit.client.1.smithi078.stdout:6/302: link d2/d1a/f6 d2/d1a/d5/d2a/d4d/f60 0 2024-04-26T04:52:48.522 INFO:tasks.workunit.client.1.smithi078.stdout:2/215: write d8/d11/d14/d15/d1a/d34/f3d [1582859,77604] 0 2024-04-26T04:52:48.522 INFO:tasks.workunit.client.1.smithi078.stdout:3/226: symlink d16/d26/d2b/l53 0 2024-04-26T04:52:48.523 INFO:tasks.workunit.client.1.smithi078.stdout:7/234: dwrite d9/da/f23 [0,4194304] 0 2024-04-26T04:52:48.523 INFO:tasks.workunit.client.1.smithi078.stdout:8/264: truncate d0/d19/d52/d35/d44/f26 925921 0 2024-04-26T04:52:48.525 INFO:tasks.workunit.client.1.smithi078.stdout:7/235: stat d9/da/d11/d13/d1a/d2d/f3f 0 2024-04-26T04:52:48.526 INFO:tasks.workunit.client.1.smithi078.stdout:9/222: getdents dc/df/d1a/d22 0 2024-04-26T04:52:48.527 INFO:tasks.workunit.client.1.smithi078.stdout:3/227: write d16/f30 [532495,110416] 0 2024-04-26T04:52:48.528 INFO:tasks.workunit.client.1.smithi078.stdout:4/278: creat d0/d25/f6c x:0 0 0 2024-04-26T04:52:48.529 INFO:tasks.workunit.client.1.smithi078.stdout:1/291: getdents d7 0 2024-04-26T04:52:48.529 INFO:tasks.workunit.client.1.smithi078.stdout:2/216: symlink d8/d11/d18/l4c 0 2024-04-26T04:52:48.530 INFO:tasks.workunit.client.1.smithi078.stdout:3/228: stat d16/d26/d2e/f4d 0 2024-04-26T04:52:48.531 INFO:tasks.workunit.client.1.smithi078.stdout:6/303: creat d2/d1a/d5/d2a/d4d/f61 x:0 0 0 2024-04-26T04:52:48.534 INFO:tasks.workunit.client.1.smithi078.stdout:4/279: mknod d0/de/d22/c6d 0 2024-04-26T04:52:48.534 INFO:tasks.workunit.client.1.smithi078.stdout:3/229: dread - d16/d26/d35/f4f zero size 2024-04-26T04:52:48.534 INFO:tasks.workunit.client.1.smithi078.stdout:9/223: symlink dc/l42 0 2024-04-26T04:52:48.536 INFO:tasks.workunit.client.1.smithi078.stdout:8/265: dwrite d0/d19/d52/f29 [0,4194304] 0 2024-04-26T04:52:48.537 INFO:tasks.workunit.client.1.smithi078.stdout:6/304: creat d2/d1a/d5/d3a/f62 x:0 0 0 2024-04-26T04:52:48.539 INFO:tasks.workunit.client.1.smithi078.stdout:0/363: sync 2024-04-26T04:52:48.540 INFO:tasks.workunit.client.1.smithi078.stdout:6/305: read - d2/f16 zero size 2024-04-26T04:52:48.540 INFO:tasks.workunit.client.1.smithi078.stdout:4/280: fsync d0/d25/d20/d31/d33/f40 0 2024-04-26T04:52:48.545 INFO:tasks.workunit.client.1.smithi078.stdout:6/306: fdatasync d2/de/d27/f55 0 2024-04-26T04:52:48.545 INFO:tasks.workunit.client.1.smithi078.stdout:8/266: mknod d0/d19/d52/d3c/c56 0 2024-04-26T04:52:48.547 INFO:tasks.workunit.client.1.smithi078.stdout:2/217: dwrite d8/f3a [0,4194304] 0 2024-04-26T04:52:48.548 INFO:tasks.workunit.client.1.smithi078.stdout:0/364: fsync d2/f2e 0 2024-04-26T04:52:48.551 INFO:tasks.workunit.client.1.smithi078.stdout:8/267: fsync d0/f18 0 2024-04-26T04:52:48.551 INFO:tasks.workunit.client.1.smithi078.stdout:6/307: chown d2/d1a/d5/d2a/l25 1443 1 2024-04-26T04:52:48.552 INFO:tasks.workunit.client.1.smithi078.stdout:2/218: write d8/d3e/f43 [3082561,42983] 0 2024-04-26T04:52:48.552 INFO:tasks.workunit.client.1.smithi078.stdout:3/230: dwrite d16/f21 [0,4194304] 0 2024-04-26T04:52:48.553 INFO:tasks.workunit.client.1.smithi078.stdout:9/224: link dc/df/d17/c25 dc/df/d1a/d22/d2d/c43 0 2024-04-26T04:52:48.554 INFO:tasks.workunit.client.1.smithi078.stdout:4/281: creat d0/d1d/d46/f6e x:0 0 0 2024-04-26T04:52:48.555 INFO:tasks.workunit.client.1.smithi078.stdout:7/236: dread d9/f14 [0,4194304] 0 2024-04-26T04:52:48.558 INFO:tasks.workunit.client.1.smithi078.stdout:4/282: readlink d0/de/d22/d27/d56/d5a/l65 0 2024-04-26T04:52:48.558 INFO:tasks.workunit.client.1.smithi078.stdout:3/231: rmdir d16 39 2024-04-26T04:52:48.560 INFO:tasks.workunit.client.1.smithi078.stdout:1/292: dread d7/d39/d4c/f26 [0,4194304] 0 2024-04-26T04:52:48.562 INFO:tasks.workunit.client.1.smithi078.stdout:1/293: fsync d7/f13 0 2024-04-26T04:52:48.562 INFO:tasks.workunit.client.1.smithi078.stdout:6/308: truncate d2/d1a/d5/d3a/d42/f59 314938 0 2024-04-26T04:52:48.564 INFO:tasks.workunit.client.1.smithi078.stdout:9/225: chown dc/df/d1a/d22/d2d/c43 78 1 2024-04-26T04:52:48.565 INFO:tasks.workunit.client.1.smithi078.stdout:7/237: readlink d9/da/d33/l38 0 2024-04-26T04:52:48.565 INFO:tasks.workunit.client.1.smithi078.stdout:8/268: write d0/f1a [675750,102401] 0 2024-04-26T04:52:48.567 INFO:tasks.workunit.client.1.smithi078.stdout:1/294: dread - d7/d39/db/d37/d58/f5e zero size 2024-04-26T04:52:48.567 INFO:tasks.workunit.client.1.smithi078.stdout:4/283: creat d0/d25/d20/d31/f6f x:0 0 0 2024-04-26T04:52:48.568 INFO:tasks.workunit.client.1.smithi078.stdout:9/226: creat dc/df/d17/f44 x:0 0 0 2024-04-26T04:52:48.568 INFO:tasks.workunit.client.1.smithi078.stdout:2/219: dread d8/d11/d14/d15/d1a/d34/f3d [0,4194304] 0 2024-04-26T04:52:48.569 INFO:tasks.workunit.client.1.smithi078.stdout:6/309: dread d2/d1a/d5/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:52:48.570 INFO:tasks.workunit.client.1.smithi078.stdout:1/295: dread - d7/d1e/f54 zero size 2024-04-26T04:52:48.572 INFO:tasks.workunit.client.1.smithi078.stdout:9/227: creat dc/f45 x:0 0 0 2024-04-26T04:52:48.574 INFO:tasks.workunit.client.1.smithi078.stdout:3/232: dwrite d16/d26/f37 [0,4194304] 0 2024-04-26T04:52:48.574 INFO:tasks.workunit.client.1.smithi078.stdout:2/220: symlink d8/d11/d18/l4d 0 2024-04-26T04:52:48.575 INFO:tasks.workunit.client.1.smithi078.stdout:5/180: sync 2024-04-26T04:52:48.576 INFO:tasks.workunit.client.1.smithi078.stdout:4/284: truncate d0/de/d22/d27/d56/f5c 1526650 0 2024-04-26T04:52:48.577 INFO:tasks.workunit.client.1.smithi078.stdout:1/296: write d7/d39/dd/d20/f45 [1377348,837] 0 2024-04-26T04:52:48.579 INFO:tasks.workunit.client.1.smithi078.stdout:0/365: truncate d2/f3f 1027024 0 2024-04-26T04:52:48.580 INFO:tasks.workunit.client.1.smithi078.stdout:2/221: write d8/d3e/f43 [1847989,31626] 0 2024-04-26T04:52:48.581 INFO:tasks.workunit.client.1.smithi078.stdout:3/233: write d16/f50 [659096,115892] 0 2024-04-26T04:52:48.582 INFO:tasks.workunit.client.1.smithi078.stdout:1/297: stat d7/d39/d17/f66 0 2024-04-26T04:52:48.582 INFO:tasks.workunit.client.1.smithi078.stdout:0/366: write d2/f70 [334837,54920] 0 2024-04-26T04:52:48.584 INFO:tasks.workunit.client.1.smithi078.stdout:9/228: symlink dc/df/d17/l46 0 2024-04-26T04:52:48.587 INFO:tasks.workunit.client.1.smithi078.stdout:4/285: rename d0/d25/d20/d31/d33 to d0/d25/d20/d31/d33/d4c/d70 22 2024-04-26T04:52:48.587 INFO:tasks.workunit.client.1.smithi078.stdout:6/310: write d2/d1a/d5/da/f22 [1319635,34536] 0 2024-04-26T04:52:48.587 INFO:tasks.workunit.client.1.smithi078.stdout:3/234: write d16/f25 [1330359,112590] 0 2024-04-26T04:52:48.587 INFO:tasks.workunit.client.1.smithi078.stdout:5/181: dread d7/f16 [4194304,4194304] 0 2024-04-26T04:52:48.587 INFO:tasks.workunit.client.1.smithi078.stdout:4/286: chown d0/de/l63 0 1 2024-04-26T04:52:48.591 INFO:tasks.workunit.client.1.smithi078.stdout:7/238: dwrite d9/da/d11/d13/d1a/f40 [0,4194304] 0 2024-04-26T04:52:48.592 INFO:tasks.workunit.client.1.smithi078.stdout:9/229: mkdir dc/df/d1a/d22/d47 0 2024-04-26T04:52:48.597 INFO:tasks.workunit.client.1.smithi078.stdout:6/311: symlink d2/d1a/d41/l63 0 2024-04-26T04:52:48.600 INFO:tasks.workunit.client.1.smithi078.stdout:1/298: getdents d7/d1e 0 2024-04-26T04:52:48.601 INFO:tasks.workunit.client.1.smithi078.stdout:4/287: truncate d0/d1d/d46/f6e 181235 0 2024-04-26T04:52:48.603 INFO:tasks.workunit.client.1.smithi078.stdout:6/312: chown d2/de/c19 26904 1 2024-04-26T04:52:48.604 INFO:tasks.workunit.client.1.smithi078.stdout:7/239: mknod d9/da/d11/d13/c48 0 2024-04-26T04:52:48.606 INFO:tasks.workunit.client.1.smithi078.stdout:3/235: dwrite d16/f22 [0,4194304] 0 2024-04-26T04:52:48.606 INFO:tasks.workunit.client.1.smithi078.stdout:1/299: creat d7/d2e/f67 x:0 0 0 2024-04-26T04:52:48.609 INFO:tasks.workunit.client.1.smithi078.stdout:9/230: dread dc/df/d27/f24 [0,4194304] 0 2024-04-26T04:52:48.610 INFO:tasks.workunit.client.1.smithi078.stdout:4/288: write d0/d25/d20/f2a [2191716,16453] 0 2024-04-26T04:52:48.614 INFO:tasks.workunit.client.1.smithi078.stdout:7/240: rename l8 to d9/l49 0 2024-04-26T04:52:48.615 INFO:tasks.workunit.client.1.smithi078.stdout:2/222: dwrite d8/d11/f31 [0,4194304] 0 2024-04-26T04:52:48.617 INFO:tasks.workunit.client.1.smithi078.stdout:8/269: dread d0/d19/d25/f2b [0,4194304] 0 2024-04-26T04:52:48.619 INFO:tasks.workunit.client.1.smithi078.stdout:5/182: dwrite d7/d18/f2e [0,4194304] 0 2024-04-26T04:52:48.621 INFO:tasks.workunit.client.1.smithi078.stdout:0/367: dwrite d2/d9/d18/f49 [0,4194304] 0 2024-04-26T04:52:48.623 INFO:tasks.workunit.client.1.smithi078.stdout:6/313: truncate d2/d1a/d5/da/d35/f3e 34589 0 2024-04-26T04:52:48.624 INFO:tasks.workunit.client.1.smithi078.stdout:2/223: write d8/d11/d14/d15/d1a/f49 [707634,78575] 0 2024-04-26T04:52:48.625 INFO:tasks.workunit.client.1.smithi078.stdout:9/231: rename dc/f45 to dc/df/d17/f48 0 2024-04-26T04:52:48.625 INFO:tasks.workunit.client.1.smithi078.stdout:5/183: chown d7/f28 11 1 2024-04-26T04:52:48.626 INFO:tasks.workunit.client.1.smithi078.stdout:4/289: mkdir d0/de/d22/d71 0 2024-04-26T04:52:48.627 INFO:tasks.workunit.client.1.smithi078.stdout:1/300: mkdir d7/d39/db/d37/d58/d5c/d68 0 2024-04-26T04:52:48.631 INFO:tasks.workunit.client.1.smithi078.stdout:6/314: write d2/d1a/d5/da/f4a [655049,129715] 0 2024-04-26T04:52:48.631 INFO:tasks.workunit.client.1.smithi078.stdout:2/224: rename d8/fb to d8/f4e 0 2024-04-26T04:52:48.633 INFO:tasks.workunit.client.1.smithi078.stdout:3/236: dwrite d16/f25 [0,4194304] 0 2024-04-26T04:52:48.633 INFO:tasks.workunit.client.1.smithi078.stdout:5/184: write d7/d18/f26 [5102058,130737] 0 2024-04-26T04:52:48.634 INFO:tasks.workunit.client.1.smithi078.stdout:7/241: dread f6 [4194304,4194304] 0 2024-04-26T04:52:48.637 INFO:tasks.workunit.client.1.smithi078.stdout:0/368: mknod d2/d1e/d5a/c77 0 2024-04-26T04:52:48.637 INFO:tasks.workunit.client.1.smithi078.stdout:2/225: read - d8/d11/d18/f47 zero size 2024-04-26T04:52:48.641 INFO:tasks.workunit.client.1.smithi078.stdout:1/301: symlink d7/d39/db/d47/l69 0 2024-04-26T04:52:48.645 INFO:tasks.workunit.client.1.smithi078.stdout:2/226: unlink d8/d11/d18/f28 0 2024-04-26T04:52:48.646 INFO:tasks.workunit.client.1.smithi078.stdout:3/237: mkdir d16/d26/d2b/d54 0 2024-04-26T04:52:48.649 INFO:tasks.workunit.client.1.smithi078.stdout:2/227: dread - d8/d11/d18/f47 zero size 2024-04-26T04:52:48.649 INFO:tasks.workunit.client.1.smithi078.stdout:6/315: dwrite d2/d49/f57 [0,4194304] 0 2024-04-26T04:52:48.653 INFO:tasks.workunit.client.1.smithi078.stdout:0/369: dwrite d2/d9/d18/fd [0,4194304] 0 2024-04-26T04:52:48.653 INFO:tasks.workunit.client.1.smithi078.stdout:9/232: dwrite dc/df/d27/f24 [0,4194304] 0 2024-04-26T04:52:48.656 INFO:tasks.workunit.client.1.smithi078.stdout:9/233: stat dc/df/d1a/d22 0 2024-04-26T04:52:48.661 INFO:tasks.workunit.client.1.smithi078.stdout:6/316: dwrite d2/d1a/d5/d3a/f62 [0,4194304] 0 2024-04-26T04:52:48.668 INFO:tasks.workunit.client.1.smithi078.stdout:0/370: dread d2/d9/d31/f4f [0,4194304] 0 2024-04-26T04:52:48.723 INFO:tasks.workunit.client.1.smithi078.stdout:8/270: getdents d0/d19/d52 0 2024-04-26T04:52:48.724 INFO:tasks.workunit.client.1.smithi078.stdout:8/271: read d0/d19/d52/f29 [1057996,6097] 0 2024-04-26T04:52:48.725 INFO:tasks.workunit.client.1.smithi078.stdout:5/185: rename d7/d21/l31 to d7/d29/l3e 0 2024-04-26T04:52:48.726 INFO:tasks.workunit.client.1.smithi078.stdout:5/186: write f3 [4145196,91860] 0 2024-04-26T04:52:48.727 INFO:tasks.workunit.client.1.smithi078.stdout:1/302: truncate d7/d39/d4c/f26 2022923 0 2024-04-26T04:52:48.727 INFO:tasks.workunit.client.1.smithi078.stdout:5/187: chown d7/f38 2554745 1 2024-04-26T04:52:48.732 INFO:tasks.workunit.client.1.smithi078.stdout:9/234: chown dc/df/d27/l3f 545221 1 2024-04-26T04:52:48.735 INFO:tasks.workunit.client.1.smithi078.stdout:6/317: creat d2/de/d27/f64 x:0 0 0 2024-04-26T04:52:48.736 INFO:tasks.workunit.client.1.smithi078.stdout:8/272: readlink d0/l3a 0 2024-04-26T04:52:48.736 INFO:tasks.workunit.client.1.smithi078.stdout:6/318: fdatasync d2/d1a/fd 0 2024-04-26T04:52:48.737 INFO:tasks.workunit.client.1.smithi078.stdout:7/242: rename d9/d16/l1d to d9/da/d11/d13/d1a/d2d/l4a 0 2024-04-26T04:52:48.737 INFO:tasks.workunit.client.1.smithi078.stdout:4/290: dread d0/d25/d20/f1b [0,4194304] 0 2024-04-26T04:52:48.738 INFO:tasks.workunit.client.1.smithi078.stdout:1/303: mknod d7/d39/dd/d20/c6a 0 2024-04-26T04:52:48.739 INFO:tasks.workunit.client.1.smithi078.stdout:5/188: symlink d7/d11/l3f 0 2024-04-26T04:52:48.744 INFO:tasks.workunit.client.1.smithi078.stdout:9/235: getdents dc/df/d1a/d3e 0 2024-04-26T04:52:48.745 INFO:tasks.workunit.client.1.smithi078.stdout:9/236: stat dc/df/d17 0 2024-04-26T04:52:48.747 INFO:tasks.workunit.client.1.smithi078.stdout:0/371: rename d2/d9/d6b to d2/d9/d78 0 2024-04-26T04:52:48.747 INFO:tasks.workunit.client.1.smithi078.stdout:6/319: mkdir d2/de/d27/d58/d5b/d65 0 2024-04-26T04:52:48.748 INFO:tasks.workunit.client.1.smithi078.stdout:4/291: dread d0/d25/d20/d31/d33/f5b [0,4194304] 0 2024-04-26T04:52:48.748 INFO:tasks.workunit.client.1.smithi078.stdout:5/189: link d7/f35 d7/d18/f40 0 2024-04-26T04:52:48.748 INFO:tasks.workunit.client.1.smithi078.stdout:7/243: dwrite f7 [4194304,4194304] 0 2024-04-26T04:52:48.749 INFO:tasks.workunit.client.1.smithi078.stdout:2/228: rename d8/d11/d18/l29 to d8/d11/l4f 0 2024-04-26T04:52:48.751 INFO:tasks.workunit.client.1.smithi078.stdout:4/292: dread d0/d25/d20/d31/d33/f5b [0,4194304] 0 2024-04-26T04:52:48.752 INFO:tasks.workunit.client.1.smithi078.stdout:7/244: mkdir d9/da/d33/d4b 0 2024-04-26T04:52:48.754 INFO:tasks.workunit.client.1.smithi078.stdout:6/320: dwrite d2/de/f12 [0,4194304] 0 2024-04-26T04:52:48.754 INFO:tasks.workunit.client.1.smithi078.stdout:8/273: rename d0/f5 to d0/d19/d52/d35/d3f/f57 0 2024-04-26T04:52:48.755 INFO:tasks.workunit.client.1.smithi078.stdout:8/274: chown d0/d19/d25/d32/c34 118360 1 2024-04-26T04:52:48.759 INFO:tasks.workunit.client.1.smithi078.stdout:6/321: creat d2/d1a/d5/da/d35/f66 x:0 0 0 2024-04-26T04:52:48.760 INFO:tasks.workunit.client.1.smithi078.stdout:4/293: mknod d0/d25/c72 0 2024-04-26T04:52:48.761 INFO:tasks.workunit.client.1.smithi078.stdout:7/245: mkdir d9/da/d33/d4b/d4c 0 2024-04-26T04:52:48.761 INFO:tasks.workunit.client.1.smithi078.stdout:4/294: chown d0/d25/d20/d31/d33/d44 3186676 1 2024-04-26T04:52:48.762 INFO:tasks.workunit.client.1.smithi078.stdout:7/246: read d9/d16/f44 [163320,43272] 0 2024-04-26T04:52:48.763 INFO:tasks.workunit.client.1.smithi078.stdout:1/304: rename d7/c38 to d7/d39/dd/d20/d53/c6b 0 2024-04-26T04:52:48.764 INFO:tasks.workunit.client.1.smithi078.stdout:4/295: read d0/d25/d20/d31/d33/d44/d55/f5d [168531,14093] 0 2024-04-26T04:52:48.764 INFO:tasks.workunit.client.1.smithi078.stdout:6/322: creat d2/de/f67 x:0 0 0 2024-04-26T04:52:48.767 INFO:tasks.workunit.client.1.smithi078.stdout:2/229: write d8/d11/d14/d15/f1b [4430811,41744] 0 2024-04-26T04:52:48.768 INFO:tasks.workunit.client.1.smithi078.stdout:9/237: rename dc/df/l2c to dc/df/d17/l49 0 2024-04-26T04:52:48.768 INFO:tasks.workunit.client.1.smithi078.stdout:9/238: fsync dc/df/d27/f24 0 2024-04-26T04:52:48.768 INFO:tasks.workunit.client.1.smithi078.stdout:1/305: creat d7/d39/db/d49/f6c x:0 0 0 2024-04-26T04:52:48.769 INFO:tasks.workunit.client.1.smithi078.stdout:4/296: fdatasync d0/d25/f3a 0 2024-04-26T04:52:48.770 INFO:tasks.workunit.client.1.smithi078.stdout:1/306: chown d7/d1e/l42 0 1 2024-04-26T04:52:48.770 INFO:tasks.workunit.client.1.smithi078.stdout:4/297: dread - d0/de/d22/d27/d56/f6a zero size 2024-04-26T04:52:48.771 INFO:tasks.workunit.client.1.smithi078.stdout:4/298: dread - d0/d25/d20/f2b zero size 2024-04-26T04:52:48.772 INFO:tasks.workunit.client.1.smithi078.stdout:6/323: creat d2/d1a/d38/f68 x:0 0 0 2024-04-26T04:52:48.773 INFO:tasks.workunit.client.1.smithi078.stdout:9/239: creat dc/df/d1a/d22/d2a/f4a x:0 0 0 2024-04-26T04:52:48.775 INFO:tasks.workunit.client.1.smithi078.stdout:6/324: chown d2/d1a/d5/d2a/l25 993804 1 2024-04-26T04:52:48.777 INFO:tasks.workunit.client.1.smithi078.stdout:9/240: dread dc/df/f36 [0,4194304] 0 2024-04-26T04:52:48.779 INFO:tasks.workunit.client.1.smithi078.stdout:7/247: truncate d9/f10 148530 0 2024-04-26T04:52:48.780 INFO:tasks.workunit.client.1.smithi078.stdout:4/299: dread d0/d25/d20/f15 [0,4194304] 0 2024-04-26T04:52:48.781 INFO:tasks.workunit.client.1.smithi078.stdout:9/241: getdents dc/df/d1a/d3e 0 2024-04-26T04:52:48.781 INFO:tasks.workunit.client.1.smithi078.stdout:1/307: truncate d7/f11 9166207 0 2024-04-26T04:52:48.784 INFO:tasks.workunit.client.1.smithi078.stdout:7/248: chown d9/l49 1042007 1 2024-04-26T04:52:48.785 INFO:tasks.workunit.client.1.smithi078.stdout:9/242: stat dc/df/d1a/d22/d47 0 2024-04-26T04:52:48.789 INFO:tasks.workunit.client.1.smithi078.stdout:1/308: mknod d7/d39/d4c/d3d/c6d 0 2024-04-26T04:52:48.790 INFO:tasks.workunit.client.1.smithi078.stdout:7/249: dread - d9/da/d11/f3a zero size 2024-04-26T04:52:48.791 INFO:tasks.workunit.client.1.smithi078.stdout:2/230: dwrite d8/d11/f22 [0,4194304] 0 2024-04-26T04:52:48.791 INFO:tasks.workunit.client.1.smithi078.stdout:6/325: dwrite d2/d1a/d5/d2a/d4d/f61 [0,4194304] 0 2024-04-26T04:52:48.793 INFO:tasks.workunit.client.1.smithi078.stdout:4/300: link d0/de/d22/d27/d56/f5c d0/d25/d20/d31/f73 0 2024-04-26T04:52:48.794 INFO:tasks.workunit.client.1.smithi078.stdout:6/326: write d2/d1a/d5/d11/f4f [44762,9493] 0 2024-04-26T04:52:48.797 INFO:tasks.workunit.client.1.smithi078.stdout:9/243: write f8 [678486,127870] 0 2024-04-26T04:52:48.799 INFO:tasks.workunit.client.1.smithi078.stdout:6/327: mkdir d2/d1a/d5/d3a/d69 0 2024-04-26T04:52:48.801 INFO:tasks.workunit.client.1.smithi078.stdout:7/250: mknod d9/da/d11/c4d 0 2024-04-26T04:52:48.803 INFO:tasks.workunit.client.1.smithi078.stdout:4/301: dwrite d0/de/f1e [0,4194304] 0 2024-04-26T04:52:48.807 INFO:tasks.workunit.client.1.smithi078.stdout:9/244: rmdir dc/df/d1a/d22/d2d 39 2024-04-26T04:52:48.807 INFO:tasks.workunit.client.1.smithi078.stdout:7/251: unlink d9/da/d20/f24 0 2024-04-26T04:52:48.809 INFO:tasks.workunit.client.1.smithi078.stdout:2/231: getdents d8/d11 0 2024-04-26T04:52:48.810 INFO:tasks.workunit.client.1.smithi078.stdout:3/238: sync 2024-04-26T04:52:48.811 INFO:tasks.workunit.client.1.smithi078.stdout:6/328: dwrite d2/de/f48 [0,4194304] 0 2024-04-26T04:52:48.812 INFO:tasks.workunit.client.1.smithi078.stdout:2/232: readlink d8/d11/d14/l33 0 2024-04-26T04:52:48.814 INFO:tasks.workunit.client.1.smithi078.stdout:7/252: creat d9/da/d11/d13/d1a/f4e x:0 0 0 2024-04-26T04:52:48.815 INFO:tasks.workunit.client.1.smithi078.stdout:4/302: getdents d0/de 0 2024-04-26T04:52:48.817 INFO:tasks.workunit.client.1.smithi078.stdout:7/253: write d9/da/d11/f12 [2329583,57178] 0 2024-04-26T04:52:48.818 INFO:tasks.workunit.client.1.smithi078.stdout:4/303: dread - d0/d25/d20/d31/f6f zero size 2024-04-26T04:52:48.821 INFO:tasks.workunit.client.1.smithi078.stdout:9/245: dwrite dc/df/d17/f48 [0,4194304] 0 2024-04-26T04:52:48.823 INFO:tasks.workunit.client.1.smithi078.stdout:9/246: write dc/df/d1a/f29 [27978,19365] 0 2024-04-26T04:52:48.826 INFO:tasks.workunit.client.1.smithi078.stdout:3/239: link d16/d26/f32 d16/d26/d2b/d31/f55 0 2024-04-26T04:52:48.827 INFO:tasks.workunit.client.1.smithi078.stdout:6/329: truncate d2/d1a/d5/d11/f15 3042241 0 2024-04-26T04:52:48.829 INFO:tasks.workunit.client.1.smithi078.stdout:7/254: creat d9/da/d11/d13/d1a/d2d/f4f x:0 0 0 2024-04-26T04:52:48.829 INFO:tasks.workunit.client.1.smithi078.stdout:9/247: write dc/df/d27/f13 [1796816,105428] 0 2024-04-26T04:52:48.829 INFO:tasks.workunit.client.1.smithi078.stdout:2/233: mknod d8/d11/d14/c50 0 2024-04-26T04:52:48.830 INFO:tasks.workunit.client.1.smithi078.stdout:1/309: write d7/d39/f12 [8993824,119479] 0 2024-04-26T04:52:48.830 INFO:tasks.workunit.client.1.smithi078.stdout:6/330: read - d2/f53 zero size 2024-04-26T04:52:48.830 INFO:tasks.workunit.client.1.smithi078.stdout:3/240: rmdir d16/d26/d2e 39 2024-04-26T04:52:48.832 INFO:tasks.workunit.client.1.smithi078.stdout:4/304: dwrite d0/d25/d20/d31/f3f [0,4194304] 0 2024-04-26T04:52:48.835 INFO:tasks.workunit.client.1.smithi078.stdout:1/310: symlink d7/d2e/l6e 0 2024-04-26T04:52:48.836 INFO:tasks.workunit.client.1.smithi078.stdout:2/234: creat d8/d11/d14/d15/d1a/d34/f51 x:0 0 0 2024-04-26T04:52:48.841 INFO:tasks.workunit.client.1.smithi078.stdout:9/248: rename dc/df/d1a to dc/df/d4b 0 2024-04-26T04:52:48.841 INFO:tasks.workunit.client.1.smithi078.stdout:4/305: dread d0/de/d22/d27/d56/f5c [0,4194304] 0 2024-04-26T04:52:48.842 INFO:tasks.workunit.client.1.smithi078.stdout:3/241: mkdir d16/d26/d2b/d54/d56 0 2024-04-26T04:52:48.843 INFO:tasks.workunit.client.1.smithi078.stdout:2/235: mkdir d8/d3e/d52 0 2024-04-26T04:52:48.844 INFO:tasks.workunit.client.1.smithi078.stdout:6/331: dread d2/f14 [0,4194304] 0 2024-04-26T04:52:48.845 INFO:tasks.workunit.client.1.smithi078.stdout:1/311: dwrite d7/d39/db/d37/f65 [0,4194304] 0 2024-04-26T04:52:48.847 INFO:tasks.workunit.client.1.smithi078.stdout:1/312: write d7/d2e/f63 [817919,21640] 0 2024-04-26T04:52:48.849 INFO:tasks.workunit.client.1.smithi078.stdout:2/236: mkdir d8/d53 0 2024-04-26T04:52:48.853 INFO:tasks.workunit.client.1.smithi078.stdout:4/306: symlink d0/de/d22/d26/l74 0 2024-04-26T04:52:48.853 INFO:tasks.workunit.client.1.smithi078.stdout:7/255: dwrite d9/da/f15 [4194304,4194304] 0 2024-04-26T04:52:48.858 INFO:tasks.workunit.client.1.smithi078.stdout:1/313: creat d7/d39/db/d37/f6f x:0 0 0 2024-04-26T04:52:48.859 INFO:tasks.workunit.client.1.smithi078.stdout:4/307: dread d0/d25/d20/d31/f3f [0,4194304] 0 2024-04-26T04:52:48.860 INFO:tasks.workunit.client.1.smithi078.stdout:3/242: creat d16/f57 x:0 0 0 2024-04-26T04:52:48.862 INFO:tasks.workunit.client.1.smithi078.stdout:4/308: write d0/de/d22/d26/d2e/f52 [5631962,114919] 0 2024-04-26T04:52:48.864 INFO:tasks.workunit.client.1.smithi078.stdout:2/237: creat d8/d11/f54 x:0 0 0 2024-04-26T04:52:48.864 INFO:tasks.workunit.client.1.smithi078.stdout:7/256: dwrite d9/da/d20/f31 [4194304,4194304] 0 2024-04-26T04:52:48.865 INFO:tasks.workunit.client.1.smithi078.stdout:0/372: sync 2024-04-26T04:52:48.865 INFO:tasks.workunit.client.1.smithi078.stdout:4/309: write d0/de/f1e [171226,82256] 0 2024-04-26T04:52:48.866 INFO:tasks.workunit.client.1.smithi078.stdout:7/257: write d9/da/d11/d13/d1a/d2d/f4f [808130,52477] 0 2024-04-26T04:52:48.867 INFO:tasks.workunit.client.1.smithi078.stdout:4/310: chown d0/de/d22/d27 386416 1 2024-04-26T04:52:48.867 INFO:tasks.workunit.client.1.smithi078.stdout:9/249: truncate dc/df/d4b/f1c 942104 0 2024-04-26T04:52:48.867 INFO:tasks.workunit.client.1.smithi078.stdout:9/250: chown dc/df 118676094 1 2024-04-26T04:52:48.868 INFO:tasks.workunit.client.1.smithi078.stdout:1/314: creat d7/d39/dd/d20/d53/f70 x:0 0 0 2024-04-26T04:52:48.870 INFO:tasks.workunit.client.1.smithi078.stdout:6/332: link d2/d1a/d5/da/c54 d2/d52/c6a 0 2024-04-26T04:52:48.873 INFO:tasks.workunit.client.1.smithi078.stdout:0/373: unlink d2/f29 0 2024-04-26T04:52:48.874 INFO:tasks.workunit.client.1.smithi078.stdout:6/333: truncate d2/d1a/f50 1327824 0 2024-04-26T04:52:48.877 INFO:tasks.workunit.client.1.smithi078.stdout:9/251: dread dc/f26 [0,4194304] 0 2024-04-26T04:52:48.877 INFO:tasks.workunit.client.1.smithi078.stdout:1/315: mknod d7/d39/db/d49/c71 0 2024-04-26T04:52:48.878 INFO:tasks.workunit.client.1.smithi078.stdout:1/316: stat d7/d39/db 0 2024-04-26T04:52:48.880 INFO:tasks.workunit.client.1.smithi078.stdout:4/311: dwrite d0/de/f34 [8388608,4194304] 0 2024-04-26T04:52:48.881 INFO:tasks.workunit.client.1.smithi078.stdout:2/238: chown d8/d11/d14/d15/d1a/d34 629371 1 2024-04-26T04:52:48.883 INFO:tasks.workunit.client.1.smithi078.stdout:5/190: sync 2024-04-26T04:52:48.884 INFO:tasks.workunit.client.1.smithi078.stdout:8/275: sync 2024-04-26T04:52:48.885 INFO:tasks.workunit.client.1.smithi078.stdout:3/243: link d16/d3c/d46/d49/f4e d16/d26/d2b/d54/f58 0 2024-04-26T04:52:48.887 INFO:tasks.workunit.client.1.smithi078.stdout:6/334: rmdir d2/d1a/d5/d2a/d4d 39 2024-04-26T04:52:48.889 INFO:tasks.workunit.client.1.smithi078.stdout:1/317: creat d7/d1e/f72 x:0 0 0 2024-04-26T04:52:48.889 INFO:tasks.workunit.client.1.smithi078.stdout:0/374: creat d2/d1e/d5a/d74/f79 x:0 0 0 2024-04-26T04:52:48.890 INFO:tasks.workunit.client.1.smithi078.stdout:5/191: creat d7/d18/d32/d37/f41 x:0 0 0 2024-04-26T04:52:48.891 INFO:tasks.workunit.client.1.smithi078.stdout:8/276: mkdir d0/d19/d25/d36/d58 0 2024-04-26T04:52:48.892 INFO:tasks.workunit.client.1.smithi078.stdout:2/239: dwrite d8/d11/d14/f40 [0,4194304] 0 2024-04-26T04:52:48.892 INFO:tasks.workunit.client.1.smithi078.stdout:3/244: creat d16/d26/d2b/d31/f59 x:0 0 0 2024-04-26T04:52:48.893 INFO:tasks.workunit.client.1.smithi078.stdout:4/312: mkdir d0/d25/d20/d31/d33/d44/d55/d5e/d75 0 2024-04-26T04:52:48.895 INFO:tasks.workunit.client.1.smithi078.stdout:6/335: unlink d2/d1a/d5/f40 0 2024-04-26T04:52:48.900 INFO:tasks.workunit.client.1.smithi078.stdout:8/277: creat d0/d19/d52/d35/d44/f59 x:0 0 0 2024-04-26T04:52:48.900 INFO:tasks.workunit.client.1.smithi078.stdout:7/258: dread d9/da/d33/f3e [0,4194304] 0 2024-04-26T04:52:48.900 INFO:tasks.workunit.client.1.smithi078.stdout:2/240: creat d8/f55 x:0 0 0 2024-04-26T04:52:48.901 INFO:tasks.workunit.client.1.smithi078.stdout:4/313: dread d0/d25/d20/d31/f73 [0,4194304] 0 2024-04-26T04:52:48.901 INFO:tasks.workunit.client.1.smithi078.stdout:3/245: mkdir d16/d26/d2b/d54/d5a 0 2024-04-26T04:52:48.901 INFO:tasks.workunit.client.1.smithi078.stdout:2/241: truncate d8/d11/d14/d15/f24 766819 0 2024-04-26T04:52:48.903 INFO:tasks.workunit.client.1.smithi078.stdout:2/242: truncate d8/d11/d14/d15/d1a/d34/f3d 2608468 0 2024-04-26T04:52:48.919 INFO:tasks.workunit.client.1.smithi078.stdout:3/246: rmdir d16/d3c/d46 39 2024-04-26T04:52:48.921 INFO:tasks.workunit.client.1.smithi078.stdout:7/259: mkdir d9/da/d11/d13/d50 0 2024-04-26T04:52:48.921 INFO:tasks.workunit.client.1.smithi078.stdout:0/375: creat d2/d9/d31/d4a/f7a x:0 0 0 2024-04-26T04:52:48.922 INFO:tasks.workunit.client.1.smithi078.stdout:3/247: chown d16/d3c/d46 55 1 2024-04-26T04:52:48.923 INFO:tasks.workunit.client.1.smithi078.stdout:2/243: symlink d8/d11/d14/l56 0 2024-04-26T04:52:48.923 INFO:tasks.workunit.client.1.smithi078.stdout:5/192: getdents d7/da/d1e 0 2024-04-26T04:52:48.924 INFO:tasks.workunit.client.1.smithi078.stdout:6/336: rename d2/f10 to d2/d1a/d5/f6b 0 2024-04-26T04:52:48.926 INFO:tasks.workunit.client.1.smithi078.stdout:0/376: creat d2/d39/f7b x:0 0 0 2024-04-26T04:52:48.927 INFO:tasks.workunit.client.1.smithi078.stdout:1/318: dwrite d7/d2e/f5b [0,4194304] 0 2024-04-26T04:52:48.930 INFO:tasks.workunit.client.1.smithi078.stdout:6/337: write d2/d1a/d41/f5e [998671,6828] 0 2024-04-26T04:52:48.934 INFO:tasks.workunit.client.1.smithi078.stdout:0/377: write d2/d9/d18/f49 [93745,105015] 0 2024-04-26T04:52:48.934 INFO:tasks.workunit.client.1.smithi078.stdout:4/314: rename d0/d25/d20/d31/d33/l38 to d0/de/d22/d27/l76 0 2024-04-26T04:52:48.937 INFO:tasks.workunit.client.1.smithi078.stdout:2/244: mknod d8/d11/d18/c57 0 2024-04-26T04:52:48.939 INFO:tasks.workunit.client.1.smithi078.stdout:3/248: dwrite d16/d26/f37 [0,4194304] 0 2024-04-26T04:52:48.941 INFO:tasks.workunit.client.1.smithi078.stdout:5/193: dwrite f5 [0,4194304] 0 2024-04-26T04:52:48.941 INFO:tasks.workunit.client.1.smithi078.stdout:9/252: dread dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:52:48.943 INFO:tasks.workunit.client.1.smithi078.stdout:2/245: write d8/f3a [3118168,64274] 0 2024-04-26T04:52:48.943 INFO:tasks.workunit.client.1.smithi078.stdout:6/338: write d2/d1a/d5/d2a/d4d/f61 [4060392,89095] 0 2024-04-26T04:52:48.950 INFO:tasks.workunit.client.1.smithi078.stdout:1/319: dwrite d7/d1e/f54 [0,4194304] 0 2024-04-26T04:52:48.951 INFO:tasks.workunit.client.1.smithi078.stdout:1/320: chown d7/d39/dd/f55 257455 1 2024-04-26T04:52:48.955 INFO:tasks.workunit.client.1.smithi078.stdout:5/194: dwrite d7/d29/f3a [0,4194304] 0 2024-04-26T04:52:48.955 INFO:tasks.workunit.client.1.smithi078.stdout:3/249: dread d16/d26/f29 [0,4194304] 0 2024-04-26T04:52:49.019 INFO:tasks.workunit.client.1.smithi078.stdout:8/278: sync 2024-04-26T04:52:49.019 INFO:tasks.workunit.client.1.smithi078.stdout:8/279: readlink d0/d19/d25/l2e 0 2024-04-26T04:52:49.020 INFO:tasks.workunit.client.1.smithi078.stdout:7/260: rename f5 to d9/d16/d41/f51 0 2024-04-26T04:52:49.020 INFO:tasks.workunit.client.1.smithi078.stdout:8/280: write d0/d19/d52/d35/d44/f4f [241336,36294] 0 2024-04-26T04:52:49.021 INFO:tasks.workunit.client.1.smithi078.stdout:4/315: creat d0/d1d/d46/f77 x:0 0 0 2024-04-26T04:52:49.039 INFO:tasks.workunit.client.1.smithi078.stdout:2/246: creat d8/d3e/f58 x:0 0 0 2024-04-26T04:52:49.040 INFO:tasks.workunit.client.1.smithi078.stdout:8/281: dwrite d0/d19/d25/d32/f4b [0,4194304] 0 2024-04-26T04:52:49.040 INFO:tasks.workunit.client.1.smithi078.stdout:5/195: creat d7/d21/f42 x:0 0 0 2024-04-26T04:52:49.041 INFO:tasks.workunit.client.1.smithi078.stdout:2/247: stat d8/d11/d18/f47 0 2024-04-26T04:52:49.041 INFO:tasks.workunit.client.1.smithi078.stdout:0/378: link d2/d1e/f69 d2/d39/f7c 0 2024-04-26T04:52:49.042 INFO:tasks.workunit.client.1.smithi078.stdout:4/316: dwrite d0/de/d22/d26/f57 [0,4194304] 0 2024-04-26T04:52:49.042 INFO:tasks.workunit.client.1.smithi078.stdout:9/253: fdatasync dc/df/d4b/f20 0 2024-04-26T04:52:49.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/196: truncate d7/d11/f2c 624020 0 2024-04-26T04:52:49.048 INFO:tasks.workunit.client.1.smithi078.stdout:0/379: truncate d2/d1e/f61 289824 0 2024-04-26T04:52:49.048 INFO:tasks.workunit.client.1.smithi078.stdout:2/248: chown d8/d11/d14/d15/d1a/c21 1430062070 1 2024-04-26T04:52:49.050 INFO:tasks.workunit.client.1.smithi078.stdout:0/380: chown d2/d1e/f61 3 1 2024-04-26T04:52:49.051 INFO:tasks.workunit.client.1.smithi078.stdout:1/321: dwrite d7/d39/db/d49/f61 [0,4194304] 0 2024-04-26T04:52:49.052 INFO:tasks.workunit.client.1.smithi078.stdout:5/197: mknod d7/d21/c43 0 2024-04-26T04:52:49.052 INFO:tasks.workunit.client.1.smithi078.stdout:8/282: dread d0/d19/d25/d32/f4b [0,4194304] 0 2024-04-26T04:52:49.053 INFO:tasks.workunit.client.1.smithi078.stdout:3/250: creat d16/d3c/f5b x:0 0 0 2024-04-26T04:52:49.054 INFO:tasks.workunit.client.1.smithi078.stdout:2/249: write d8/f4e [12107466,118760] 0 2024-04-26T04:52:49.057 INFO:tasks.workunit.client.1.smithi078.stdout:6/339: getdents d2/d1a/d5/d2a/d4d 0 2024-04-26T04:52:49.058 INFO:tasks.workunit.client.1.smithi078.stdout:5/198: write d7/d18/d32/d37/f41 [491669,7842] 0 2024-04-26T04:52:49.063 INFO:tasks.workunit.client.1.smithi078.stdout:6/340: fdatasync d2/f53 0 2024-04-26T04:52:49.063 INFO:tasks.workunit.client.1.smithi078.stdout:1/322: dwrite d7/d2e/f33 [0,4194304] 0 2024-04-26T04:52:49.065 INFO:tasks.workunit.client.1.smithi078.stdout:2/250: dread - d8/d11/d14/d15/d1a/d34/f51 zero size 2024-04-26T04:52:49.067 INFO:tasks.workunit.client.1.smithi078.stdout:2/251: dread - d8/d11/d14/d15/d1a/d34/f51 zero size 2024-04-26T04:52:49.068 INFO:tasks.workunit.client.1.smithi078.stdout:2/252: stat d8/d11/d14/d15/d1a/c21 0 2024-04-26T04:52:49.089 INFO:tasks.workunit.client.1.smithi078.stdout:4/317: link d0/de/d22/d27/d56/f60 d0/de/f78 0 2024-04-26T04:52:49.090 INFO:tasks.workunit.client.1.smithi078.stdout:0/381: mkdir d2/d9/d18/d7d 0 2024-04-26T04:52:49.094 INFO:tasks.workunit.client.1.smithi078.stdout:5/199: rename d7/da/f23 to d7/d18/f44 0 2024-04-26T04:52:49.104 INFO:tasks.workunit.client.1.smithi078.stdout:5/200: symlink d7/da/l45 0 2024-04-26T04:52:49.106 INFO:tasks.workunit.client.1.smithi078.stdout:6/341: symlink d2/d1a/d5c/l6c 0 2024-04-26T04:52:49.106 INFO:tasks.workunit.client.1.smithi078.stdout:5/201: write d7/f38 [764580,64756] 0 2024-04-26T04:52:49.106 INFO:tasks.workunit.client.1.smithi078.stdout:9/254: getdents dc/df/d4b/d22/d2d 0 2024-04-26T04:52:49.112 INFO:tasks.workunit.client.1.smithi078.stdout:5/202: mkdir d7/d29/d46 0 2024-04-26T04:52:49.113 INFO:tasks.workunit.client.1.smithi078.stdout:9/255: creat dc/df/d17/f4c x:0 0 0 2024-04-26T04:52:49.114 INFO:tasks.workunit.client.1.smithi078.stdout:5/203: write d7/d18/d32/d37/f3d [599450,74290] 0 2024-04-26T04:52:49.116 INFO:tasks.workunit.client.1.smithi078.stdout:9/256: fdatasync dc/df/d17/f2e 0 2024-04-26T04:52:49.116 INFO:tasks.workunit.client.1.smithi078.stdout:5/204: truncate d7/d18/f44 1526921 0 2024-04-26T04:52:49.120 INFO:tasks.workunit.client.1.smithi078.stdout:5/205: read f3 [2837887,23397] 0 2024-04-26T04:52:49.121 INFO:tasks.workunit.client.1.smithi078.stdout:9/257: unlink dc/df/d4b/f29 0 2024-04-26T04:52:49.123 INFO:tasks.workunit.client.1.smithi078.stdout:6/342: dread d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:49.124 INFO:tasks.workunit.client.1.smithi078.stdout:6/343: dread - d2/d1a/f43 zero size 2024-04-26T04:52:49.124 INFO:tasks.workunit.client.1.smithi078.stdout:5/206: creat d7/d11/f47 x:0 0 0 2024-04-26T04:52:49.128 INFO:tasks.workunit.client.1.smithi078.stdout:6/344: creat d2/d1a/d5c/f6d x:0 0 0 2024-04-26T04:52:49.130 INFO:tasks.workunit.client.1.smithi078.stdout:5/207: mkdir d7/d48 0 2024-04-26T04:52:49.132 INFO:tasks.workunit.client.1.smithi078.stdout:7/261: sync 2024-04-26T04:52:49.132 INFO:tasks.workunit.client.1.smithi078.stdout:6/345: symlink d2/de/l6e 0 2024-04-26T04:52:49.135 INFO:tasks.workunit.client.1.smithi078.stdout:5/208: dread d7/d11/f2b [0,4194304] 0 2024-04-26T04:52:49.136 INFO:tasks.workunit.client.1.smithi078.stdout:7/262: mkdir d9/da/d11/d13/d1a/d52 0 2024-04-26T04:52:49.137 INFO:tasks.workunit.client.1.smithi078.stdout:7/263: stat f6 0 2024-04-26T04:52:49.138 INFO:tasks.workunit.client.1.smithi078.stdout:5/209: symlink d7/d29/d46/l49 0 2024-04-26T04:52:49.141 INFO:tasks.workunit.client.1.smithi078.stdout:6/346: dwrite d2/d1a/d41/f5e [0,4194304] 0 2024-04-26T04:52:49.143 INFO:tasks.workunit.client.1.smithi078.stdout:5/210: creat d7/d18/d32/f4a x:0 0 0 2024-04-26T04:52:49.143 INFO:tasks.workunit.client.1.smithi078.stdout:6/347: fsync d2/d1a/d5/d2a/d4d/f61 0 2024-04-26T04:52:49.145 INFO:tasks.workunit.client.1.smithi078.stdout:6/348: chown d2/d1a/d5/d2a/d4d 3801 1 2024-04-26T04:52:49.149 INFO:tasks.workunit.client.1.smithi078.stdout:3/251: dwrite d16/d26/d2b/d54/f58 [0,4194304] 0 2024-04-26T04:52:49.150 INFO:tasks.workunit.client.1.smithi078.stdout:5/211: truncate d7/da/f12 3659498 0 2024-04-26T04:52:49.153 INFO:tasks.workunit.client.1.smithi078.stdout:3/252: creat d16/d26/d2b/d54/f5c x:0 0 0 2024-04-26T04:52:49.154 INFO:tasks.workunit.client.1.smithi078.stdout:8/283: write d0/d19/d52/d35/d3f/f57 [2821862,48081] 0 2024-04-26T04:52:49.158 INFO:tasks.workunit.client.1.smithi078.stdout:5/212: mknod d7/c4b 0 2024-04-26T04:52:49.159 INFO:tasks.workunit.client.1.smithi078.stdout:6/349: rmdir d2/de/d27/d58/d5b/d65 0 2024-04-26T04:52:49.160 INFO:tasks.workunit.client.1.smithi078.stdout:6/350: chown d2/d1a/d5/da/d36/d4b 3908 1 2024-04-26T04:52:49.160 INFO:tasks.workunit.client.1.smithi078.stdout:8/284: readlink d0/d19/d52/l1b 0 2024-04-26T04:52:49.161 INFO:tasks.workunit.client.1.smithi078.stdout:6/351: dread - d2/d1a/f43 zero size 2024-04-26T04:52:49.161 INFO:tasks.workunit.client.1.smithi078.stdout:3/253: chown d16/d26/d2e/l44 1938 1 2024-04-26T04:52:49.163 INFO:tasks.workunit.client.1.smithi078.stdout:5/213: creat d7/da/d1e/f4c x:0 0 0 2024-04-26T04:52:49.163 INFO:tasks.workunit.client.1.smithi078.stdout:8/285: read d0/d19/d52/d35/f4e [238093,63372] 0 2024-04-26T04:52:49.163 INFO:tasks.workunit.client.1.smithi078.stdout:3/254: dread - d16/d26/f32 zero size 2024-04-26T04:52:49.164 INFO:tasks.workunit.client.1.smithi078.stdout:0/382: truncate d2/d9/d31/f36 4986420 0 2024-04-26T04:52:49.164 INFO:tasks.workunit.client.1.smithi078.stdout:8/286: readlink d0/d19/l22 0 2024-04-26T04:52:49.164 INFO:tasks.workunit.client.1.smithi078.stdout:6/352: symlink d2/d1a/d5/d3a/d42/l6f 0 2024-04-26T04:52:49.164 INFO:tasks.workunit.client.1.smithi078.stdout:8/287: readlink d0/d19/d25/l2e 0 2024-04-26T04:52:49.165 INFO:tasks.workunit.client.1.smithi078.stdout:8/288: chown d0/d19/d25/d36/l54 2947 1 2024-04-26T04:52:49.166 INFO:tasks.workunit.client.1.smithi078.stdout:1/323: sync 2024-04-26T04:52:49.167 INFO:tasks.workunit.client.1.smithi078.stdout:5/214: unlink d7/ld 0 2024-04-26T04:52:49.168 INFO:tasks.workunit.client.1.smithi078.stdout:2/253: truncate d8/d11/f2a 820929 0 2024-04-26T04:52:49.169 INFO:tasks.workunit.client.1.smithi078.stdout:7/264: dread d9/da/d11/f12 [0,4194304] 0 2024-04-26T04:52:49.170 INFO:tasks.workunit.client.1.smithi078.stdout:7/265: write d9/da/d11/f3a [209041,8928] 0 2024-04-26T04:52:49.172 INFO:tasks.workunit.client.1.smithi078.stdout:1/324: rmdir d7/d39/d17 39 2024-04-26T04:52:49.173 INFO:tasks.workunit.client.1.smithi078.stdout:3/255: truncate d16/f3f 1400826 0 2024-04-26T04:52:49.174 INFO:tasks.workunit.client.1.smithi078.stdout:2/254: mkdir d8/d11/d14/d15/d59 0 2024-04-26T04:52:49.175 INFO:tasks.workunit.client.1.smithi078.stdout:7/266: creat d9/d16/f53 x:0 0 0 2024-04-26T04:52:49.176 INFO:tasks.workunit.client.1.smithi078.stdout:1/325: creat d7/d39/db/d47/f73 x:0 0 0 2024-04-26T04:52:49.176 INFO:tasks.workunit.client.1.smithi078.stdout:7/267: dread - d9/da/d11/d13/f25 zero size 2024-04-26T04:52:49.178 INFO:tasks.workunit.client.1.smithi078.stdout:3/256: creat d16/d26/d2b/d54/d5a/f5d x:0 0 0 2024-04-26T04:52:49.179 INFO:tasks.workunit.client.1.smithi078.stdout:3/257: fdatasync d16/d26/d35/f4f 0 2024-04-26T04:52:49.180 INFO:tasks.workunit.client.1.smithi078.stdout:2/255: mknod d8/d11/d14/d15/d1a/c5a 0 2024-04-26T04:52:49.180 INFO:tasks.workunit.client.1.smithi078.stdout:3/258: stat d16/d26/d2e/f42 0 2024-04-26T04:52:49.180 INFO:tasks.workunit.client.1.smithi078.stdout:7/268: symlink d9/l54 0 2024-04-26T04:52:49.181 INFO:tasks.workunit.client.1.smithi078.stdout:1/326: symlink d7/d39/db/d47/l74 0 2024-04-26T04:52:49.182 INFO:tasks.workunit.client.1.smithi078.stdout:9/258: write dc/df/d4b/f1c [1710830,76349] 0 2024-04-26T04:52:49.182 INFO:tasks.workunit.client.1.smithi078.stdout:5/215: dread d7/da/f10 [0,4194304] 0 2024-04-26T04:52:49.182 INFO:tasks.workunit.client.1.smithi078.stdout:3/259: stat d16/d3c/d46/d49/f4e 0 2024-04-26T04:52:49.185 INFO:tasks.workunit.client.1.smithi078.stdout:5/216: truncate d7/f39 154423 0 2024-04-26T04:52:49.187 INFO:tasks.workunit.client.1.smithi078.stdout:3/260: mknod d16/d26/c5e 0 2024-04-26T04:52:49.187 INFO:tasks.workunit.client.1.smithi078.stdout:2/256: dread d8/f3a [0,4194304] 0 2024-04-26T04:52:49.188 INFO:tasks.workunit.client.1.smithi078.stdout:9/259: getdents dc/df/d4b/d3e 0 2024-04-26T04:52:49.188 INFO:tasks.workunit.client.1.smithi078.stdout:3/261: dread - d16/f57 zero size 2024-04-26T04:52:49.189 INFO:tasks.workunit.client.1.smithi078.stdout:9/260: fdatasync dc/df/d17/f44 0 2024-04-26T04:52:49.189 INFO:tasks.workunit.client.1.smithi078.stdout:7/269: dwrite d9/da/d11/d13/d1a/d2d/f3f [0,4194304] 0 2024-04-26T04:52:49.190 INFO:tasks.workunit.client.1.smithi078.stdout:6/353: unlink d2/d1a/c3c 0 2024-04-26T04:52:49.191 INFO:tasks.workunit.client.1.smithi078.stdout:5/217: mkdir d7/da/d1e/d4d 0 2024-04-26T04:52:49.193 INFO:tasks.workunit.client.1.smithi078.stdout:2/257: write d8/f3a [467251,52254] 0 2024-04-26T04:52:49.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/261: creat dc/df/d4b/d22/d2d/f4d x:0 0 0 2024-04-26T04:52:49.197 INFO:tasks.workunit.client.1.smithi078.stdout:3/262: dread d16/d26/d2b/d54/f58 [0,4194304] 0 2024-04-26T04:52:49.198 INFO:tasks.workunit.client.1.smithi078.stdout:6/354: creat d2/d1a/d5/da/d35/f70 x:0 0 0 2024-04-26T04:52:49.199 INFO:tasks.workunit.client.1.smithi078.stdout:9/262: write dc/df/d27/f13 [1118925,27465] 0 2024-04-26T04:52:49.199 INFO:tasks.workunit.client.1.smithi078.stdout:7/270: rename d9/da/d20/f31 to d9/da/d33/d4b/d4c/f55 0 2024-04-26T04:52:49.202 INFO:tasks.workunit.client.1.smithi078.stdout:4/318: dwrite d0/de/d22/d27/d56/f5c [0,4194304] 0 2024-04-26T04:52:49.210 INFO:tasks.workunit.client.1.smithi078.stdout:3/263: rename d16/d3c/d46 to d16/d26/d2e/d5f 0 2024-04-26T04:52:49.213 INFO:tasks.workunit.client.1.smithi078.stdout:2/258: mkdir d8/d11/d14/d5b 0 2024-04-26T04:52:49.215 INFO:tasks.workunit.client.1.smithi078.stdout:6/355: link d2/d1a/d38/f68 d2/f71 0 2024-04-26T04:52:49.216 INFO:tasks.workunit.client.1.smithi078.stdout:3/264: dwrite d16/d26/d2e/f51 [0,4194304] 0 2024-04-26T04:52:49.218 INFO:tasks.workunit.client.1.smithi078.stdout:4/319: creat d0/de/d22/d71/f79 x:0 0 0 2024-04-26T04:52:49.220 INFO:tasks.workunit.client.1.smithi078.stdout:9/263: creat dc/f4e x:0 0 0 2024-04-26T04:52:49.223 INFO:tasks.workunit.client.1.smithi078.stdout:2/259: unlink d8/d11/d14/d15/f39 0 2024-04-26T04:52:49.225 INFO:tasks.workunit.client.1.smithi078.stdout:8/289: dwrite d0/d19/d52/f24 [0,4194304] 0 2024-04-26T04:52:49.226 INFO:tasks.workunit.client.1.smithi078.stdout:8/290: stat d0/cf 0 2024-04-26T04:52:49.227 INFO:tasks.workunit.client.1.smithi078.stdout:8/291: dread - d0/d19/d52/d35/d44/f59 zero size 2024-04-26T04:52:49.229 INFO:tasks.workunit.client.1.smithi078.stdout:9/264: unlink dc/df/d27/f24 0 2024-04-26T04:52:49.231 INFO:tasks.workunit.client.1.smithi078.stdout:7/271: getdents d9/d16/d41 0 2024-04-26T04:52:49.234 INFO:tasks.workunit.client.1.smithi078.stdout:1/327: truncate d7/d39/f12 3976695 0 2024-04-26T04:52:49.235 INFO:tasks.workunit.client.1.smithi078.stdout:1/328: write d7/d39/dd/f55 [3405599,34561] 0 2024-04-26T04:52:49.236 INFO:tasks.workunit.client.1.smithi078.stdout:1/329: write d7/d2e/f5b [1263714,126905] 0 2024-04-26T04:52:49.237 INFO:tasks.workunit.client.1.smithi078.stdout:2/260: getdents d8/d53 0 2024-04-26T04:52:49.239 INFO:tasks.workunit.client.1.smithi078.stdout:8/292: fsync d0/d19/f2a 0 2024-04-26T04:52:49.241 INFO:tasks.workunit.client.1.smithi078.stdout:1/330: mknod d7/d39/dd/d20/d53/c75 0 2024-04-26T04:52:49.244 INFO:tasks.workunit.client.1.smithi078.stdout:7/272: dwrite d9/d16/d41/f51 [8388608,4194304] 0 2024-04-26T04:52:49.246 INFO:tasks.workunit.client.1.smithi078.stdout:8/293: unlink d0/d19/l3b 0 2024-04-26T04:52:49.248 INFO:tasks.workunit.client.1.smithi078.stdout:1/331: creat d7/d39/dd/d20/d53/f76 x:0 0 0 2024-04-26T04:52:49.250 INFO:tasks.workunit.client.1.smithi078.stdout:2/261: creat d8/d11/d14/d15/d1a/f5c x:0 0 0 2024-04-26T04:52:49.250 INFO:tasks.workunit.client.1.smithi078.stdout:5/218: dwrite d7/f35 [0,4194304] 0 2024-04-26T04:52:49.251 INFO:tasks.workunit.client.1.smithi078.stdout:1/332: dread - d7/d39/dd/d20/d53/f76 zero size 2024-04-26T04:52:49.255 INFO:tasks.workunit.client.1.smithi078.stdout:2/262: mkdir d8/d11/d18/d5d 0 2024-04-26T04:52:49.258 INFO:tasks.workunit.client.1.smithi078.stdout:5/219: creat d7/d21/f4e x:0 0 0 2024-04-26T04:52:49.259 INFO:tasks.workunit.client.1.smithi078.stdout:5/220: fsync d7/f38 0 2024-04-26T04:52:49.263 INFO:tasks.workunit.client.1.smithi078.stdout:1/333: dwrite f3 [4194304,4194304] 0 2024-04-26T04:52:49.263 INFO:tasks.workunit.client.1.smithi078.stdout:2/263: mkdir d8/d53/d5e 0 2024-04-26T04:52:49.264 INFO:tasks.workunit.client.1.smithi078.stdout:2/264: fdatasync d8/d11/d18/f47 0 2024-04-26T04:52:49.264 INFO:tasks.workunit.client.1.smithi078.stdout:2/265: stat d8/c26 0 2024-04-26T04:52:49.265 INFO:tasks.workunit.client.1.smithi078.stdout:2/266: chown d8/d11/d14/d15/f4a 279 1 2024-04-26T04:52:49.266 INFO:tasks.workunit.client.1.smithi078.stdout:3/265: rename d16/d26/f4a to d16/d26/d2b/d54/d5a/f60 0 2024-04-26T04:52:49.267 INFO:tasks.workunit.client.1.smithi078.stdout:9/265: rmdir dc/df/d27 39 2024-04-26T04:52:49.273 INFO:tasks.workunit.client.1.smithi078.stdout:1/334: mknod d7/d39/d4c/d3d/c77 0 2024-04-26T04:52:49.276 INFO:tasks.workunit.client.1.smithi078.stdout:4/320: rename d0/d25/f6c to d0/d25/d20/d31/d33/d44/d69/f7a 0 2024-04-26T04:52:49.277 INFO:tasks.workunit.client.1.smithi078.stdout:1/335: mknod d7/d39/dd/d20/d53/c78 0 2024-04-26T04:52:49.278 INFO:tasks.workunit.client.1.smithi078.stdout:1/336: fsync d7/d39/db/d47/f73 0 2024-04-26T04:52:49.281 INFO:tasks.workunit.client.1.smithi078.stdout:4/321: mkdir d0/de/d22/d27/d7b 0 2024-04-26T04:52:49.281 INFO:tasks.workunit.client.1.smithi078.stdout:2/267: creat d8/d11/d14/f5f x:0 0 0 2024-04-26T04:52:49.282 INFO:tasks.workunit.client.1.smithi078.stdout:3/266: creat d16/d3c/f61 x:0 0 0 2024-04-26T04:52:49.282 INFO:tasks.workunit.client.1.smithi078.stdout:2/268: write d8/d11/d14/d15/f32 [744269,51054] 0 2024-04-26T04:52:49.284 INFO:tasks.workunit.client.1.smithi078.stdout:4/322: mknod d0/d1d/d46/c7c 0 2024-04-26T04:52:49.284 INFO:tasks.workunit.client.1.smithi078.stdout:2/269: stat d8/d11/d14/c50 0 2024-04-26T04:52:49.284 INFO:tasks.workunit.client.1.smithi078.stdout:3/267: write d16/d26/d35/f4f [886175,76458] 0 2024-04-26T04:52:49.285 INFO:tasks.workunit.client.1.smithi078.stdout:1/337: unlink d7/d2e/c32 0 2024-04-26T04:52:49.285 INFO:tasks.workunit.client.1.smithi078.stdout:2/270: fsync d8/f55 0 2024-04-26T04:52:49.286 INFO:tasks.workunit.client.1.smithi078.stdout:2/271: fsync d8/d3e/f58 0 2024-04-26T04:52:49.288 INFO:tasks.workunit.client.1.smithi078.stdout:1/338: symlink d7/d39/dd/d20/d53/l79 0 2024-04-26T04:52:49.289 INFO:tasks.workunit.client.1.smithi078.stdout:3/268: rename d16/d26/d2e/f42 to d16/d26/f62 0 2024-04-26T04:52:49.291 INFO:tasks.workunit.client.1.smithi078.stdout:2/272: mknod d8/d11/d18/c60 0 2024-04-26T04:52:49.292 INFO:tasks.workunit.client.1.smithi078.stdout:7/273: dwrite d9/da/d11/d13/d1a/d2d/f3d [0,4194304] 0 2024-04-26T04:52:49.296 INFO:tasks.workunit.client.1.smithi078.stdout:7/274: write d9/da/d11/d13/d1a/f1c [1269653,115970] 0 2024-04-26T04:52:49.297 INFO:tasks.workunit.client.1.smithi078.stdout:4/323: dwrite d0/d25/d20/d31/d33/d44/d55/f5d [0,4194304] 0 2024-04-26T04:52:49.301 INFO:tasks.workunit.client.1.smithi078.stdout:7/275: creat d9/da/d33/f56 x:0 0 0 2024-04-26T04:52:49.302 INFO:tasks.workunit.client.1.smithi078.stdout:2/273: dwrite d8/d3e/f43 [0,4194304] 0 2024-04-26T04:52:49.304 INFO:tasks.workunit.client.1.smithi078.stdout:3/269: dwrite d16/d26/d2e/d5f/d49/f4e [0,4194304] 0 2024-04-26T04:52:49.305 INFO:tasks.workunit.client.1.smithi078.stdout:0/383: sync 2024-04-26T04:52:49.308 INFO:tasks.workunit.client.1.smithi078.stdout:0/384: symlink d2/d9/d60/d6a/l7e 0 2024-04-26T04:52:49.310 INFO:tasks.workunit.client.1.smithi078.stdout:2/274: rmdir d8/d11/d14/d15/d59 0 2024-04-26T04:52:49.314 INFO:tasks.workunit.client.1.smithi078.stdout:7/276: dwrite d9/da/f15 [4194304,4194304] 0 2024-04-26T04:52:49.316 INFO:tasks.workunit.client.1.smithi078.stdout:0/385: truncate d2/d9/d60/d6a/f6d 385934 0 2024-04-26T04:52:49.319 INFO:tasks.workunit.client.1.smithi078.stdout:1/339: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:49.320 INFO:tasks.workunit.client.1.smithi078.stdout:7/277: fdatasync d9/da/f23 0 2024-04-26T04:52:49.321 INFO:tasks.workunit.client.1.smithi078.stdout:1/340: mknod d7/d39/dd/d20/c7a 0 2024-04-26T04:52:49.324 INFO:tasks.workunit.client.1.smithi078.stdout:7/278: mkdir d9/d57 0 2024-04-26T04:52:49.332 INFO:tasks.workunit.client.1.smithi078.stdout:5/221: truncate d7/d29/f3a 2895747 0 2024-04-26T04:52:49.334 INFO:tasks.workunit.client.1.smithi078.stdout:9/266: truncate f8 61124 0 2024-04-26T04:52:49.334 INFO:tasks.workunit.client.1.smithi078.stdout:1/341: dwrite d7/d39/d17/f66 [0,4194304] 0 2024-04-26T04:52:49.335 INFO:tasks.workunit.client.1.smithi078.stdout:7/279: dwrite d9/da/d11/d13/d1a/d2d/f3f [4194304,4194304] 0 2024-04-26T04:52:49.342 INFO:tasks.workunit.client.1.smithi078.stdout:9/267: write dc/df/f1d [2912097,126557] 0 2024-04-26T04:52:49.347 INFO:tasks.workunit.client.1.smithi078.stdout:5/222: link d7/d18/d32/f4a d7/d29/d46/f4f 0 2024-04-26T04:52:49.350 INFO:tasks.workunit.client.1.smithi078.stdout:5/223: mknod d7/d29/d46/c50 0 2024-04-26T04:52:49.353 INFO:tasks.workunit.client.1.smithi078.stdout:2/275: rename d8/d11/d14/d15 to d8/d11/d18/d5d/d61 0 2024-04-26T04:52:49.353 INFO:tasks.workunit.client.1.smithi078.stdout:5/224: creat d7/d18/d32/d37/f51 x:0 0 0 2024-04-26T04:52:49.385 INFO:tasks.workunit.client.1.smithi078.stdout:9/268: rename dc/df/f36 to dc/df/d27/f4f 0 2024-04-26T04:52:49.386 INFO:tasks.workunit.client.1.smithi078.stdout:9/269: dread - dc/f3d zero size 2024-04-26T04:52:49.386 INFO:tasks.workunit.client.1.smithi078.stdout:1/342: stat d7/d39/f12 0 2024-04-26T04:52:49.388 INFO:tasks.workunit.client.1.smithi078.stdout:2/276: creat d8/d11/d18/d5d/d61/f62 x:0 0 0 2024-04-26T04:52:49.389 INFO:tasks.workunit.client.1.smithi078.stdout:9/270: symlink dc/df/d17/l50 0 2024-04-26T04:52:49.392 INFO:tasks.workunit.client.1.smithi078.stdout:1/343: getdents d7/d39/d4c 0 2024-04-26T04:52:49.393 INFO:tasks.workunit.client.1.smithi078.stdout:5/225: dwrite d7/d29/f36 [0,4194304] 0 2024-04-26T04:52:49.394 INFO:tasks.workunit.client.1.smithi078.stdout:5/226: fdatasync d7/d18/f27 0 2024-04-26T04:52:49.397 INFO:tasks.workunit.client.1.smithi078.stdout:9/271: getdents dc/df/d4b/d22/d2d 0 2024-04-26T04:52:49.401 INFO:tasks.workunit.client.1.smithi078.stdout:9/272: dread dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:49.401 INFO:tasks.workunit.client.1.smithi078.stdout:1/344: dwrite f3 [0,4194304] 0 2024-04-26T04:52:49.403 INFO:tasks.workunit.client.1.smithi078.stdout:2/277: dwrite d8/fa [0,4194304] 0 2024-04-26T04:52:49.406 INFO:tasks.workunit.client.1.smithi078.stdout:9/273: unlink dc/df/d4b/f20 0 2024-04-26T04:52:49.406 INFO:tasks.workunit.client.1.smithi078.stdout:1/345: creat d7/d39/dd/d20/f7b x:0 0 0 2024-04-26T04:52:49.409 INFO:tasks.workunit.client.1.smithi078.stdout:1/346: symlink d7/d39/db/l7c 0 2024-04-26T04:52:49.409 INFO:tasks.workunit.client.1.smithi078.stdout:9/274: fdatasync dc/df/d17/f3a 0 2024-04-26T04:52:49.419 INFO:tasks.workunit.client.1.smithi078.stdout:6/356: sync 2024-04-26T04:52:49.419 INFO:tasks.workunit.client.1.smithi078.stdout:8/294: sync 2024-04-26T04:52:49.424 INFO:tasks.workunit.client.1.smithi078.stdout:6/357: dwrite d2/d49/f47 [0,4194304] 0 2024-04-26T04:52:49.491 INFO:tasks.workunit.client.1.smithi078.stdout:7/280: sync 2024-04-26T04:52:49.491 INFO:tasks.workunit.client.1.smithi078.stdout:3/270: sync 2024-04-26T04:52:49.491 INFO:tasks.workunit.client.1.smithi078.stdout:4/324: sync 2024-04-26T04:52:49.492 INFO:tasks.workunit.client.1.smithi078.stdout:0/386: sync 2024-04-26T04:52:49.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/387: truncate d2/d1e/d5a/d74/f79 814256 0 2024-04-26T04:52:49.498 INFO:tasks.workunit.client.1.smithi078.stdout:3/271: dwrite d16/d26/d2b/d54/d5a/f5d [0,4194304] 0 2024-04-26T04:52:49.505 INFO:tasks.workunit.client.1.smithi078.stdout:4/325: dread d0/d25/d20/d31/d33/f40 [0,4194304] 0 2024-04-26T04:52:49.505 INFO:tasks.workunit.client.1.smithi078.stdout:4/326: read - d0/d25/d20/f2b zero size 2024-04-26T04:52:49.506 INFO:tasks.workunit.client.1.smithi078.stdout:4/327: truncate d0/d25/d20/d31/d33/d44/f68 504388 0 2024-04-26T04:52:49.510 INFO:tasks.workunit.client.1.smithi078.stdout:8/295: sync 2024-04-26T04:52:49.511 INFO:tasks.workunit.client.1.smithi078.stdout:5/227: sync 2024-04-26T04:52:49.539 INFO:tasks.workunit.client.1.smithi078.stdout:7/281: truncate d9/f10 712 0 2024-04-26T04:52:49.539 INFO:tasks.workunit.client.1.smithi078.stdout:0/388: rmdir d2/d1e 39 2024-04-26T04:52:49.541 INFO:tasks.workunit.client.1.smithi078.stdout:3/272: mknod d16/d26/d2e/d5f/d49/c63 0 2024-04-26T04:52:49.544 INFO:tasks.workunit.client.1.smithi078.stdout:2/278: truncate d8/d11/f45 57024 0 2024-04-26T04:52:49.545 INFO:tasks.workunit.client.1.smithi078.stdout:8/296: creat d0/d19/d52/f5a x:0 0 0 2024-04-26T04:52:49.546 INFO:tasks.workunit.client.1.smithi078.stdout:5/228: symlink d7/d21/d2f/l52 0 2024-04-26T04:52:49.547 INFO:tasks.workunit.client.1.smithi078.stdout:9/275: dwrite dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:49.548 INFO:tasks.workunit.client.1.smithi078.stdout:7/282: creat d9/f58 x:0 0 0 2024-04-26T04:52:49.549 INFO:tasks.workunit.client.1.smithi078.stdout:8/297: chown d0/d19/d52/d35 102 1 2024-04-26T04:52:49.551 INFO:tasks.workunit.client.1.smithi078.stdout:3/273: stat f9 0 2024-04-26T04:52:49.554 INFO:tasks.workunit.client.1.smithi078.stdout:7/283: stat d9/da/d11/f45 0 2024-04-26T04:52:49.555 INFO:tasks.workunit.client.1.smithi078.stdout:0/389: truncate d2/d9/f30 3218608 0 2024-04-26T04:52:49.556 INFO:tasks.workunit.client.1.smithi078.stdout:0/390: fsync d2/f6 0 2024-04-26T04:52:49.557 INFO:tasks.workunit.client.1.smithi078.stdout:9/276: write dc/df/d4b/f1c [2345885,52734] 0 2024-04-26T04:52:49.559 INFO:tasks.workunit.client.1.smithi078.stdout:2/279: write d8/d11/f22 [1880147,102009] 0 2024-04-26T04:52:49.563 INFO:tasks.workunit.client.1.smithi078.stdout:3/274: write d16/d26/d2e/d5f/d49/f4b [3766939,58989] 0 2024-04-26T04:52:49.563 INFO:tasks.workunit.client.1.smithi078.stdout:1/347: dwrite d7/f11 [0,4194304] 0 2024-04-26T04:52:49.564 INFO:tasks.workunit.client.1.smithi078.stdout:4/328: dwrite d0/f37 [4194304,4194304] 0 2024-04-26T04:52:49.565 INFO:tasks.workunit.client.1.smithi078.stdout:8/298: mkdir d0/d19/d25/d36/d58/d5b 0 2024-04-26T04:52:49.572 INFO:tasks.workunit.client.1.smithi078.stdout:0/391: mkdir d2/d9/d60/d7f 0 2024-04-26T04:52:49.576 INFO:tasks.workunit.client.1.smithi078.stdout:5/229: dwrite d7/d29/f36 [0,4194304] 0 2024-04-26T04:52:49.576 INFO:tasks.workunit.client.1.smithi078.stdout:7/284: creat d9/da/d11/d13/d50/f59 x:0 0 0 2024-04-26T04:52:49.580 INFO:tasks.workunit.client.1.smithi078.stdout:8/299: readlink d0/d19/d52/d3c/l40 0 2024-04-26T04:52:49.581 INFO:tasks.workunit.client.1.smithi078.stdout:1/348: truncate d7/d39/db/d37/d58/f5e 37995 0 2024-04-26T04:52:49.586 INFO:tasks.workunit.client.1.smithi078.stdout:9/277: dread dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:49.586 INFO:tasks.workunit.client.1.smithi078.stdout:6/358: dread d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:49.586 INFO:tasks.workunit.client.1.smithi078.stdout:2/280: link d8/le d8/d11/d14/d5b/l63 0 2024-04-26T04:52:49.587 INFO:tasks.workunit.client.1.smithi078.stdout:7/285: rename d9/d16 to d9/da/d11/d13/d1a/d52/d5a 0 2024-04-26T04:52:49.588 INFO:tasks.workunit.client.1.smithi078.stdout:8/300: chown d0/d19/d52/d35/c55 0 1 2024-04-26T04:52:49.589 INFO:tasks.workunit.client.1.smithi078.stdout:4/329: dwrite d0/de/d22/d26/f57 [0,4194304] 0 2024-04-26T04:52:49.591 INFO:tasks.workunit.client.1.smithi078.stdout:9/278: stat dc/df/d17/f48 0 2024-04-26T04:52:49.591 INFO:tasks.workunit.client.1.smithi078.stdout:2/281: stat d8/l10 0 2024-04-26T04:52:49.592 INFO:tasks.workunit.client.1.smithi078.stdout:0/392: creat d2/d9/d31/d4a/f80 x:0 0 0 2024-04-26T04:52:49.598 INFO:tasks.workunit.client.1.smithi078.stdout:5/230: dwrite d7/d18/f40 [0,4194304] 0 2024-04-26T04:52:49.598 INFO:tasks.workunit.client.1.smithi078.stdout:8/301: write d0/d19/d25/f2b [3190877,52575] 0 2024-04-26T04:52:49.600 INFO:tasks.workunit.client.1.smithi078.stdout:3/275: dread d16/d26/d2e/f45 [0,4194304] 0 2024-04-26T04:52:49.600 INFO:tasks.workunit.client.1.smithi078.stdout:4/330: write d0/de/d22/d27/d56/f61 [738862,56497] 0 2024-04-26T04:52:49.601 INFO:tasks.workunit.client.1.smithi078.stdout:0/393: fdatasync d2/d39/f42 0 2024-04-26T04:52:49.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/282: rmdir d8/d11/d14/d5b 39 2024-04-26T04:52:49.605 INFO:tasks.workunit.client.1.smithi078.stdout:7/286: truncate d9/f10 342725 0 2024-04-26T04:52:49.607 INFO:tasks.workunit.client.1.smithi078.stdout:1/349: dwrite d7/d39/d17/f66 [4194304,4194304] 0 2024-04-26T04:52:49.608 INFO:tasks.workunit.client.1.smithi078.stdout:8/302: mkdir d0/d19/d52/d5c 0 2024-04-26T04:52:49.609 INFO:tasks.workunit.client.1.smithi078.stdout:0/394: symlink d2/d9/d31/l81 0 2024-04-26T04:52:49.611 INFO:tasks.workunit.client.1.smithi078.stdout:5/231: truncate d7/d18/d32/f3b 682226 0 2024-04-26T04:52:49.611 INFO:tasks.workunit.client.1.smithi078.stdout:3/276: truncate f9 366147 0 2024-04-26T04:52:49.613 INFO:tasks.workunit.client.1.smithi078.stdout:6/359: dwrite d2/d49/f57 [0,4194304] 0 2024-04-26T04:52:49.614 INFO:tasks.workunit.client.1.smithi078.stdout:6/360: chown d2/d1a/d38 1649 1 2024-04-26T04:52:49.614 INFO:tasks.workunit.client.1.smithi078.stdout:7/287: mkdir d9/da/d33/d4b/d4c/d5b 0 2024-04-26T04:52:49.615 INFO:tasks.workunit.client.1.smithi078.stdout:2/283: truncate d8/d11/d18/d5d/d61/f3c 979265 0 2024-04-26T04:52:49.615 INFO:tasks.workunit.client.1.smithi078.stdout:9/279: rename dc/df/d17/l49 to dc/df/d4b/d22/l51 0 2024-04-26T04:52:49.617 INFO:tasks.workunit.client.1.smithi078.stdout:1/350: fdatasync d7/d39/d4c/f26 0 2024-04-26T04:52:49.619 INFO:tasks.workunit.client.1.smithi078.stdout:8/303: creat d0/d19/d52/d35/d44/f5d x:0 0 0 2024-04-26T04:52:49.620 INFO:tasks.workunit.client.1.smithi078.stdout:5/232: creat d7/d29/f53 x:0 0 0 2024-04-26T04:52:49.622 INFO:tasks.workunit.client.1.smithi078.stdout:5/233: stat d7/d18/d32 0 2024-04-26T04:52:49.622 INFO:tasks.workunit.client.1.smithi078.stdout:1/351: chown d7/d39/d4c/d3d/l2b 36545930 1 2024-04-26T04:52:49.624 INFO:tasks.workunit.client.1.smithi078.stdout:4/331: dwrite d0/de/d22/d27/d56/f60 [0,4194304] 0 2024-04-26T04:52:49.624 INFO:tasks.workunit.client.1.smithi078.stdout:1/352: readlink d7/d39/d17/l27 0 2024-04-26T04:52:49.625 INFO:tasks.workunit.client.1.smithi078.stdout:3/277: symlink d16/l64 0 2024-04-26T04:52:49.628 INFO:tasks.workunit.client.1.smithi078.stdout:2/284: rename d8/d11/d14/l33 to d8/d11/d18/d5d/d61/l64 0 2024-04-26T04:52:49.629 INFO:tasks.workunit.client.1.smithi078.stdout:1/353: fdatasync d7/d39/db/d49/f6c 0 2024-04-26T04:52:49.629 INFO:tasks.workunit.client.1.smithi078.stdout:4/332: chown d0/d25/d20/d31/d33 13 1 2024-04-26T04:52:49.634 INFO:tasks.workunit.client.1.smithi078.stdout:3/278: fdatasync d16/d26/d35/f4f 0 2024-04-26T04:52:49.634 INFO:tasks.workunit.client.1.smithi078.stdout:7/288: creat d9/da/d33/d4b/d4c/d5b/f5c x:0 0 0 2024-04-26T04:52:49.635 INFO:tasks.workunit.client.1.smithi078.stdout:5/234: creat d7/d29/d46/f54 x:0 0 0 2024-04-26T04:52:49.636 INFO:tasks.workunit.client.1.smithi078.stdout:9/280: creat dc/df/d4b/d22/d47/f52 x:0 0 0 2024-04-26T04:52:49.636 INFO:tasks.workunit.client.1.smithi078.stdout:2/285: rmdir d8/d11/d18/d5d/d61/d1a/d34 39 2024-04-26T04:52:49.637 INFO:tasks.workunit.client.1.smithi078.stdout:8/304: dwrite d0/d19/f1c [0,4194304] 0 2024-04-26T04:52:49.638 INFO:tasks.workunit.client.1.smithi078.stdout:4/333: creat d0/de/d22/d27/d56/d5a/f7d x:0 0 0 2024-04-26T04:52:49.638 INFO:tasks.workunit.client.1.smithi078.stdout:3/279: chown d16/d26/d2e/f4d 483 1 2024-04-26T04:52:49.639 INFO:tasks.workunit.client.1.smithi078.stdout:5/235: dread - d7/d21/f42 zero size 2024-04-26T04:52:49.642 INFO:tasks.workunit.client.1.smithi078.stdout:0/395: truncate d2/d9/d18/f49 3202795 0 2024-04-26T04:52:49.642 INFO:tasks.workunit.client.1.smithi078.stdout:3/280: fdatasync d16/d26/f48 0 2024-04-26T04:52:49.644 INFO:tasks.workunit.client.1.smithi078.stdout:0/396: chown d2 1051 1 2024-04-26T04:52:49.644 INFO:tasks.workunit.client.1.smithi078.stdout:7/289: read f4 [3635623,129456] 0 2024-04-26T04:52:49.646 INFO:tasks.workunit.client.1.smithi078.stdout:8/305: write d0/d19/f1c [2867843,116007] 0 2024-04-26T04:52:49.646 INFO:tasks.workunit.client.1.smithi078.stdout:5/236: chown d7/d18/d32/d37/f51 25887405 1 2024-04-26T04:52:49.646 INFO:tasks.workunit.client.1.smithi078.stdout:1/354: dwrite d7/d1e/f36 [0,4194304] 0 2024-04-26T04:52:49.646 INFO:tasks.workunit.client.1.smithi078.stdout:9/281: rename dc/df/d17/f3a to dc/df/d27/f53 0 2024-04-26T04:52:49.649 INFO:tasks.workunit.client.1.smithi078.stdout:6/361: dwrite d2/d1a/d41/f5e [4194304,4194304] 0 2024-04-26T04:52:49.650 INFO:tasks.workunit.client.1.smithi078.stdout:3/281: unlink d16/d26/d2e/f45 0 2024-04-26T04:52:49.652 INFO:tasks.workunit.client.1.smithi078.stdout:7/290: unlink d9/l54 0 2024-04-26T04:52:49.653 INFO:tasks.workunit.client.1.smithi078.stdout:5/237: mknod d7/d21/c55 0 2024-04-26T04:52:49.653 INFO:tasks.workunit.client.1.smithi078.stdout:4/334: creat d0/de/f7e x:0 0 0 2024-04-26T04:52:49.657 INFO:tasks.workunit.client.1.smithi078.stdout:8/306: creat d0/d19/d52/d35/d3f/f5e x:0 0 0 2024-04-26T04:52:49.660 INFO:tasks.workunit.client.1.smithi078.stdout:3/282: unlink l10 0 2024-04-26T04:52:49.662 INFO:tasks.workunit.client.1.smithi078.stdout:0/397: creat d2/d9/d31/d4a/f82 x:0 0 0 2024-04-26T04:52:49.663 INFO:tasks.workunit.client.1.smithi078.stdout:4/335: creat d0/d25/d20/d31/d33/d44/d55/f7f x:0 0 0 2024-04-26T04:52:49.666 INFO:tasks.workunit.client.1.smithi078.stdout:6/362: dread d2/d49/f47 [0,4194304] 0 2024-04-26T04:52:49.667 INFO:tasks.workunit.client.1.smithi078.stdout:5/238: getdents d7/d18/d1c 0 2024-04-26T04:52:49.667 INFO:tasks.workunit.client.1.smithi078.stdout:8/307: fdatasync d0/d19/d52/d35/d44/f4f 0 2024-04-26T04:52:49.668 INFO:tasks.workunit.client.1.smithi078.stdout:2/286: dwrite d8/f2b [0,4194304] 0 2024-04-26T04:52:49.668 INFO:tasks.workunit.client.1.smithi078.stdout:9/282: link dc/df/f28 dc/f54 0 2024-04-26T04:52:49.668 INFO:tasks.workunit.client.1.smithi078.stdout:1/355: dwrite d7/d39/db/d49/f61 [0,4194304] 0 2024-04-26T04:52:49.670 INFO:tasks.workunit.client.1.smithi078.stdout:3/283: fsync d16/d26/f62 0 2024-04-26T04:52:49.670 INFO:tasks.workunit.client.1.smithi078.stdout:4/336: mknod d0/c80 0 2024-04-26T04:52:49.671 INFO:tasks.workunit.client.1.smithi078.stdout:6/363: fsync d2/d1a/d5/da/f4a 0 2024-04-26T04:52:49.671 INFO:tasks.workunit.client.1.smithi078.stdout:5/239: chown d7/c9 75 1 2024-04-26T04:52:49.671 INFO:tasks.workunit.client.1.smithi078.stdout:0/398: unlink d2/d9/l53 0 2024-04-26T04:52:49.672 INFO:tasks.workunit.client.1.smithi078.stdout:8/308: dread - d0/d19/d52/d35/d44/f59 zero size 2024-04-26T04:52:49.674 INFO:tasks.workunit.client.1.smithi078.stdout:2/287: fdatasync d8/d11/d18/d5d/d61/f24 0 2024-04-26T04:52:49.674 INFO:tasks.workunit.client.1.smithi078.stdout:7/291: getdents d9/da/d33/d4b 0 2024-04-26T04:52:49.676 INFO:tasks.workunit.client.1.smithi078.stdout:1/356: symlink d7/d1e/l7d 0 2024-04-26T04:52:49.676 INFO:tasks.workunit.client.1.smithi078.stdout:5/240: write d7/d18/d32/f3b [1572118,129473] 0 2024-04-26T04:52:49.679 INFO:tasks.workunit.client.1.smithi078.stdout:4/337: creat d0/d25/d20/d31/d33/d4c/f81 x:0 0 0 2024-04-26T04:52:49.681 INFO:tasks.workunit.client.1.smithi078.stdout:8/309: creat d0/d19/d52/d35/f5f x:0 0 0 2024-04-26T04:52:49.682 INFO:tasks.workunit.client.1.smithi078.stdout:7/292: symlink d9/da/d11/d13/d1a/d52/l5d 0 2024-04-26T04:52:49.685 INFO:tasks.workunit.client.1.smithi078.stdout:2/288: symlink d8/d11/l65 0 2024-04-26T04:52:49.685 INFO:tasks.workunit.client.1.smithi078.stdout:0/399: creat d2/d9/d31/d73/f83 x:0 0 0 2024-04-26T04:52:49.686 INFO:tasks.workunit.client.1.smithi078.stdout:1/357: symlink d7/d39/db/d37/l7e 0 2024-04-26T04:52:49.686 INFO:tasks.workunit.client.1.smithi078.stdout:9/283: dwrite dc/df/d17/f4c [0,4194304] 0 2024-04-26T04:52:49.687 INFO:tasks.workunit.client.1.smithi078.stdout:6/364: rename d2/d1a/d5/d11/f15 to d2/d1a/d5/da/d35/f72 0 2024-04-26T04:52:49.688 INFO:tasks.workunit.client.1.smithi078.stdout:8/310: creat d0/d19/d52/d3c/f60 x:0 0 0 2024-04-26T04:52:49.689 INFO:tasks.workunit.client.1.smithi078.stdout:2/289: symlink d8/l66 0 2024-04-26T04:52:49.691 INFO:tasks.workunit.client.1.smithi078.stdout:8/311: chown d0/f10 209099640 1 2024-04-26T04:52:49.691 INFO:tasks.workunit.client.1.smithi078.stdout:1/358: creat d7/d1e/f7f x:0 0 0 2024-04-26T04:52:49.692 INFO:tasks.workunit.client.1.smithi078.stdout:0/400: fdatasync d2/d9/d18/f1c 0 2024-04-26T04:52:49.693 INFO:tasks.workunit.client.1.smithi078.stdout:5/241: rename d7/d11/c14 to d7/d18/d1c/c56 0 2024-04-26T04:52:49.694 INFO:tasks.workunit.client.1.smithi078.stdout:9/284: unlink f8 0 2024-04-26T04:52:49.695 INFO:tasks.workunit.client.1.smithi078.stdout:8/312: truncate d0/d19/d52/d35/d3f/f5e 558832 0 2024-04-26T04:52:49.695 INFO:tasks.workunit.client.1.smithi078.stdout:6/365: dread d2/d1a/fd [0,4194304] 0 2024-04-26T04:52:49.696 INFO:tasks.workunit.client.1.smithi078.stdout:4/338: creat d0/de/d22/d27/d7b/f82 x:0 0 0 2024-04-26T04:52:49.698 INFO:tasks.workunit.client.1.smithi078.stdout:2/290: fdatasync d8/d11/f2a 0 2024-04-26T04:52:49.700 INFO:tasks.workunit.client.1.smithi078.stdout:7/293: rename d9/da/d20/l46 to d9/da/d33/l5e 0 2024-04-26T04:52:49.702 INFO:tasks.workunit.client.1.smithi078.stdout:0/401: mkdir d2/d9/d78/d84 0 2024-04-26T04:52:49.705 INFO:tasks.workunit.client.1.smithi078.stdout:4/339: symlink d0/de/d22/d27/d56/d5a/l83 0 2024-04-26T04:52:49.705 INFO:tasks.workunit.client.1.smithi078.stdout:3/284: write d16/f20 [651276,80189] 0 2024-04-26T04:52:49.706 INFO:tasks.workunit.client.1.smithi078.stdout:3/285: write d16/d26/d2e/f43 [481156,79906] 0 2024-04-26T04:52:49.708 INFO:tasks.workunit.client.1.smithi078.stdout:3/286: chown d16/d26/d2b/d54/d5a 3131451 1 2024-04-26T04:52:49.708 INFO:tasks.workunit.client.1.smithi078.stdout:5/242: mkdir d7/d48/d57 0 2024-04-26T04:52:49.709 INFO:tasks.workunit.client.1.smithi078.stdout:7/294: symlink d9/da/d11/d13/d1a/d52/l5f 0 2024-04-26T04:52:49.711 INFO:tasks.workunit.client.1.smithi078.stdout:0/402: readlink d2/d9/d31/d4a/l51 0 2024-04-26T04:52:49.712 INFO:tasks.workunit.client.1.smithi078.stdout:8/313: mkdir d0/d19/d25/d36/d58/d5b/d61 0 2024-04-26T04:52:49.713 INFO:tasks.workunit.client.1.smithi078.stdout:4/340: creat d0/d1d/d46/f84 x:0 0 0 2024-04-26T04:52:49.714 INFO:tasks.workunit.client.1.smithi078.stdout:4/341: chown d0/d25/d20/d31/d33/d44/c41 2 1 2024-04-26T04:52:49.716 INFO:tasks.workunit.client.1.smithi078.stdout:3/287: unlink d16/d26/d2b/d54/f5c 0 2024-04-26T04:52:49.716 INFO:tasks.workunit.client.1.smithi078.stdout:3/288: write d16/f57 [307613,83340] 0 2024-04-26T04:52:49.717 INFO:tasks.workunit.client.1.smithi078.stdout:5/243: rmdir d7/d18/d1c 39 2024-04-26T04:52:49.718 INFO:tasks.workunit.client.1.smithi078.stdout:5/244: chown d7/c19 27106539 1 2024-04-26T04:52:49.720 INFO:tasks.workunit.client.1.smithi078.stdout:0/403: write d2/d1e/f2a [2037067,11376] 0 2024-04-26T04:52:49.720 INFO:tasks.workunit.client.1.smithi078.stdout:8/314: symlink d0/d19/d52/d3c/l62 0 2024-04-26T04:52:49.722 INFO:tasks.workunit.client.1.smithi078.stdout:3/289: dread d16/d26/d2b/d54/d5a/f5d [0,4194304] 0 2024-04-26T04:52:49.723 INFO:tasks.workunit.client.1.smithi078.stdout:3/290: write d16/d26/f48 [131772,43635] 0 2024-04-26T04:52:49.723 INFO:tasks.workunit.client.1.smithi078.stdout:9/285: rename dc/df/d4b/d22/d47 to dc/d55 0 2024-04-26T04:52:49.726 INFO:tasks.workunit.client.1.smithi078.stdout:1/359: truncate d7/d39/dd/d20/f3b 2144033 0 2024-04-26T04:52:49.726 INFO:tasks.workunit.client.1.smithi078.stdout:1/360: dread - d7/d39/d4c/f4e zero size 2024-04-26T04:52:49.727 INFO:tasks.workunit.client.1.smithi078.stdout:1/361: chown d7/d1e/f36 5376806 1 2024-04-26T04:52:49.728 INFO:tasks.workunit.client.1.smithi078.stdout:1/362: dread - d7/d39/d4c/f4e zero size 2024-04-26T04:52:49.729 INFO:tasks.workunit.client.1.smithi078.stdout:5/245: unlink d7/da/d1e/c33 0 2024-04-26T04:52:49.731 INFO:tasks.workunit.client.1.smithi078.stdout:7/295: truncate d9/da/d11/d13/f17 48957 0 2024-04-26T04:52:49.732 INFO:tasks.workunit.client.1.smithi078.stdout:2/291: link d8/d11/f54 d8/d11/d14/f67 0 2024-04-26T04:52:49.733 INFO:tasks.workunit.client.1.smithi078.stdout:0/404: readlink d2/d9/d31/d4a/l5b 0 2024-04-26T04:52:49.737 INFO:tasks.workunit.client.1.smithi078.stdout:6/366: rename d2/de/c19 to d2/d1a/d38/c73 0 2024-04-26T04:52:49.739 INFO:tasks.workunit.client.1.smithi078.stdout:3/291: creat d16/d26/d2b/d31/f65 x:0 0 0 2024-04-26T04:52:49.739 INFO:tasks.workunit.client.1.smithi078.stdout:6/367: truncate d2/de/f67 247850 0 2024-04-26T04:52:49.743 INFO:tasks.workunit.client.1.smithi078.stdout:2/292: write d8/d11/d18/d5d/d61/f62 [975617,49211] 0 2024-04-26T04:52:49.744 INFO:tasks.workunit.client.1.smithi078.stdout:2/293: dread - d8/d11/d14/f5f zero size 2024-04-26T04:52:49.744 INFO:tasks.workunit.client.1.smithi078.stdout:0/405: creat d2/d9/d31/f85 x:0 0 0 2024-04-26T04:52:49.745 INFO:tasks.workunit.client.1.smithi078.stdout:0/406: chown d2/d9/d31/l63 1147255169 1 2024-04-26T04:52:49.747 INFO:tasks.workunit.client.1.smithi078.stdout:4/342: truncate d0/f37 1246148 0 2024-04-26T04:52:49.748 INFO:tasks.workunit.client.1.smithi078.stdout:1/363: rename d7/d39/dd/d20/c7a to d7/d39/db/d49/c80 0 2024-04-26T04:52:49.749 INFO:tasks.workunit.client.1.smithi078.stdout:4/343: dread - d0/de/d22/d71/f79 zero size 2024-04-26T04:52:49.750 INFO:tasks.workunit.client.1.smithi078.stdout:8/315: truncate d0/d19/d25/d32/f4b 2650144 0 2024-04-26T04:52:49.750 INFO:tasks.workunit.client.1.smithi078.stdout:3/292: dread fd [0,4194304] 0 2024-04-26T04:52:49.750 INFO:tasks.workunit.client.1.smithi078.stdout:6/368: mknod d2/d1a/d5/da/d36/d4b/c74 0 2024-04-26T04:52:49.752 INFO:tasks.workunit.client.1.smithi078.stdout:2/294: rename d8/c36 to d8/c68 0 2024-04-26T04:52:49.752 INFO:tasks.workunit.client.1.smithi078.stdout:6/369: chown d2/de/f12 5407819 1 2024-04-26T04:52:49.753 INFO:tasks.workunit.client.1.smithi078.stdout:9/286: dwrite dc/df/d4b/d22/d2d/f4d [0,4194304] 0 2024-04-26T04:52:49.753 INFO:tasks.workunit.client.1.smithi078.stdout:1/364: rmdir d7/d1e 39 2024-04-26T04:52:49.756 INFO:tasks.workunit.client.1.smithi078.stdout:8/316: unlink d0/d19/c53 0 2024-04-26T04:52:49.757 INFO:tasks.workunit.client.1.smithi078.stdout:8/317: read - d0/d19/d52/d35/d44/f5d zero size 2024-04-26T04:52:49.758 INFO:tasks.workunit.client.1.smithi078.stdout:8/318: write d0/d19/d52/d35/f5f [183346,61972] 0 2024-04-26T04:52:49.759 INFO:tasks.workunit.client.1.smithi078.stdout:4/344: rename d0/d2f to d0/de/d22/d26/d2e/d85 0 2024-04-26T04:52:49.759 INFO:tasks.workunit.client.1.smithi078.stdout:8/319: read - d0/d19/d25/d36/f37 zero size 2024-04-26T04:52:49.762 INFO:tasks.workunit.client.1.smithi078.stdout:8/320: truncate d0/d19/d52/d35/d3f/f57 2907068 0 2024-04-26T04:52:49.763 INFO:tasks.workunit.client.1.smithi078.stdout:6/370: unlink d2/d1a/d5/d11/f21 0 2024-04-26T04:52:49.767 INFO:tasks.workunit.client.1.smithi078.stdout:2/295: rmdir d8/d11/d18/d5d/d61 39 2024-04-26T04:52:49.769 INFO:tasks.workunit.client.1.smithi078.stdout:7/296: link d9/da/d11/d13/d1a/d2d/l4a d9/d57/l60 0 2024-04-26T04:52:49.770 INFO:tasks.workunit.client.1.smithi078.stdout:3/293: mkdir d16/d26/d2b/d66 0 2024-04-26T04:52:49.774 INFO:tasks.workunit.client.1.smithi078.stdout:6/371: mknod d2/d1a/d5/d2a/d4d/c75 0 2024-04-26T04:52:49.776 INFO:tasks.workunit.client.1.smithi078.stdout:4/345: dwrite d0/d25/d20/d31/d33/f40 [0,4194304] 0 2024-04-26T04:52:49.777 INFO:tasks.workunit.client.1.smithi078.stdout:1/365: rename d7/d1e/c48 to d7/d39/db/d49/c81 0 2024-04-26T04:52:49.780 INFO:tasks.workunit.client.1.smithi078.stdout:7/297: readlink d9/da/d11/d13/d1a/d2d/l4a 0 2024-04-26T04:52:49.781 INFO:tasks.workunit.client.1.smithi078.stdout:3/294: mknod d16/d26/d2e/d5f/d49/c67 0 2024-04-26T04:52:49.785 INFO:tasks.workunit.client.1.smithi078.stdout:7/298: dread - d9/da/d11/d13/d1a/d52/d5a/f53 zero size 2024-04-26T04:52:49.785 INFO:tasks.workunit.client.1.smithi078.stdout:2/296: mkdir d8/d11/d14/d69 0 2024-04-26T04:52:49.786 INFO:tasks.workunit.client.1.smithi078.stdout:4/346: dwrite d0/d25/d20/d31/d33/d44/f59 [0,4194304] 0 2024-04-26T04:52:49.787 INFO:tasks.workunit.client.1.smithi078.stdout:1/366: dwrite d7/d39/db/d37/d58/f5e [0,4194304] 0 2024-04-26T04:52:49.788 INFO:tasks.workunit.client.1.smithi078.stdout:4/347: dread - d0/d25/d20/f2b zero size 2024-04-26T04:52:49.792 INFO:tasks.workunit.client.1.smithi078.stdout:3/295: creat d16/d26/d2b/d54/f68 x:0 0 0 2024-04-26T04:52:49.795 INFO:tasks.workunit.client.1.smithi078.stdout:2/297: creat d8/d3e/f6a x:0 0 0 2024-04-26T04:52:49.796 INFO:tasks.workunit.client.1.smithi078.stdout:8/321: getdents d0 0 2024-04-26T04:52:49.798 INFO:tasks.workunit.client.1.smithi078.stdout:9/287: link dc/df/d27/c40 dc/df/d4b/d3e/c56 0 2024-04-26T04:52:49.798 INFO:tasks.workunit.client.1.smithi078.stdout:6/372: getdents d2/d1a/d41 0 2024-04-26T04:52:49.799 INFO:tasks.workunit.client.1.smithi078.stdout:8/322: mknod d0/d19/c63 0 2024-04-26T04:52:49.800 INFO:tasks.workunit.client.1.smithi078.stdout:7/299: dwrite d9/da/d11/f3a [0,4194304] 0 2024-04-26T04:52:49.803 INFO:tasks.workunit.client.1.smithi078.stdout:6/373: truncate d2/d1a/d38/f68 567627 0 2024-04-26T04:52:49.803 INFO:tasks.workunit.client.1.smithi078.stdout:4/348: dread d0/d25/d20/d31/f73 [0,4194304] 0 2024-04-26T04:52:49.804 INFO:tasks.workunit.client.1.smithi078.stdout:3/296: rename d16/f1c to d16/f69 0 2024-04-26T04:52:49.806 INFO:tasks.workunit.client.1.smithi078.stdout:7/300: symlink d9/da/d11/d13/d1a/d2d/l61 0 2024-04-26T04:52:49.808 INFO:tasks.workunit.client.1.smithi078.stdout:2/298: creat d8/d11/d18/d5d/d61/d1a/f6b x:0 0 0 2024-04-26T04:52:49.808 INFO:tasks.workunit.client.1.smithi078.stdout:9/288: dread dc/df/d17/f4c [0,4194304] 0 2024-04-26T04:52:49.809 INFO:tasks.workunit.client.1.smithi078.stdout:4/349: symlink d0/de/d22/d27/d56/d5a/l86 0 2024-04-26T04:52:49.813 INFO:tasks.workunit.client.1.smithi078.stdout:7/301: write d9/da/d11/d13/d1a/f42 [28425,1646] 0 2024-04-26T04:52:49.813 INFO:tasks.workunit.client.1.smithi078.stdout:4/350: stat d0/d25/d20/d31/d33/d44 0 2024-04-26T04:52:49.816 INFO:tasks.workunit.client.1.smithi078.stdout:6/374: dread d2/d1a/d5/d11/f4e [0,4194304] 0 2024-04-26T04:52:49.816 INFO:tasks.workunit.client.1.smithi078.stdout:7/302: chown d9/da/d33/f56 0 1 2024-04-26T04:52:49.817 INFO:tasks.workunit.client.1.smithi078.stdout:3/297: dwrite d16/f39 [0,4194304] 0 2024-04-26T04:52:49.823 INFO:tasks.workunit.client.1.smithi078.stdout:2/299: creat d8/d11/d18/d5d/d61/d1a/f6c x:0 0 0 2024-04-26T04:52:49.823 INFO:tasks.workunit.client.1.smithi078.stdout:3/298: rmdir d16/d26/d2b/d54 39 2024-04-26T04:52:49.826 INFO:tasks.workunit.client.1.smithi078.stdout:7/303: truncate d9/da/d11/d13/d1a/f40 5170278 0 2024-04-26T04:52:49.826 INFO:tasks.workunit.client.1.smithi078.stdout:9/289: dwrite dc/df/d17/f44 [0,4194304] 0 2024-04-26T04:52:49.827 INFO:tasks.workunit.client.1.smithi078.stdout:2/300: chown d8/d11/d18/d5d/d61/d1a 178447 1 2024-04-26T04:52:49.829 INFO:tasks.workunit.client.1.smithi078.stdout:9/290: fdatasync dc/df/d27/f13 0 2024-04-26T04:52:49.832 INFO:tasks.workunit.client.1.smithi078.stdout:2/301: truncate d8/d11/d14/f3b 13076623 0 2024-04-26T04:52:49.833 INFO:tasks.workunit.client.1.smithi078.stdout:1/367: dwrite d7/d39/d4c/f26 [0,4194304] 0 2024-04-26T04:52:49.834 INFO:tasks.workunit.client.1.smithi078.stdout:6/375: write d2/f71 [699705,71760] 0 2024-04-26T04:52:49.837 INFO:tasks.workunit.client.1.smithi078.stdout:7/304: symlink d9/da/l62 0 2024-04-26T04:52:49.838 INFO:tasks.workunit.client.0.smithi077.stdout:0/0: sync 2024-04-26T04:52:49.839 INFO:tasks.workunit.client.0.smithi077.stdout:0/1: mkdir d0 0 2024-04-26T04:52:49.841 INFO:tasks.workunit.client.0.smithi077.stdout:0/2: creat d0/f1 x:0 0 0 2024-04-26T04:52:49.842 INFO:tasks.workunit.client.1.smithi078.stdout:9/291: mkdir dc/df/d4b/d22/d2a/d57 0 2024-04-26T04:52:49.842 INFO:tasks.workunit.client.0.smithi077.stdout:0/3: fdatasync d0/f1 0 2024-04-26T04:52:49.843 INFO:tasks.workunit.client.1.smithi078.stdout:2/302: truncate d8/d11/f2a 1264281 0 2024-04-26T04:52:49.843 INFO:tasks.workunit.client.0.smithi077.stdout:0/4: truncate d0/f1 722885 0 2024-04-26T04:52:49.845 INFO:tasks.workunit.client.0.smithi077.stdout:0/5: creat d0/f2 x:0 0 0 2024-04-26T04:52:49.848 INFO:tasks.workunit.client.1.smithi078.stdout:8/323: dwrite d0/d19/d25/d32/f4b [0,4194304] 0 2024-04-26T04:52:49.850 INFO:tasks.workunit.client.1.smithi078.stdout:3/299: dwrite d16/d26/f32 [0,4194304] 0 2024-04-26T04:52:49.851 INFO:tasks.workunit.client.1.smithi078.stdout:8/324: chown d0/cf 21 1 2024-04-26T04:52:49.851 INFO:tasks.workunit.client.1.smithi078.stdout:2/303: write d8/d11/d18/d5d/d61/f62 [917798,82143] 0 2024-04-26T04:52:49.857 INFO:tasks.workunit.client.1.smithi078.stdout:3/300: unlink d16/f50 0 2024-04-26T04:52:49.858 INFO:tasks.workunit.client.1.smithi078.stdout:1/368: dwrite d7/fc [0,4194304] 0 2024-04-26T04:52:49.858 INFO:tasks.workunit.client.1.smithi078.stdout:9/292: dread dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:49.859 INFO:tasks.workunit.client.1.smithi078.stdout:7/305: dread f4 [4194304,4194304] 0 2024-04-26T04:52:49.860 INFO:tasks.workunit.client.1.smithi078.stdout:9/293: fsync f1 0 2024-04-26T04:52:49.864 INFO:tasks.workunit.client.1.smithi078.stdout:3/301: fdatasync d16/d26/d2b/d31/f34 0 2024-04-26T04:52:49.865 INFO:tasks.workunit.client.1.smithi078.stdout:2/304: read d8/d11/f2a [827044,22581] 0 2024-04-26T04:52:49.866 INFO:tasks.workunit.client.1.smithi078.stdout:3/302: stat d16/d26/d2b/l2d 0 2024-04-26T04:52:49.867 INFO:tasks.workunit.client.1.smithi078.stdout:9/294: mkdir dc/df/d4b/d58 0 2024-04-26T04:52:49.868 INFO:tasks.workunit.client.1.smithi078.stdout:2/305: fdatasync d8/f25 0 2024-04-26T04:52:49.869 INFO:tasks.workunit.client.1.smithi078.stdout:7/306: link d9/da/d33/f3e d9/da/d33/d4b/f63 0 2024-04-26T04:52:49.870 INFO:tasks.workunit.client.1.smithi078.stdout:1/369: dwrite d7/d39/db/f41 [0,4194304] 0 2024-04-26T04:52:49.873 INFO:tasks.workunit.client.1.smithi078.stdout:3/303: symlink d16/d26/d2b/l6a 0 2024-04-26T04:52:49.873 INFO:tasks.workunit.client.1.smithi078.stdout:2/306: write d8/d11/d14/f5f [619176,116197] 0 2024-04-26T04:52:49.874 INFO:tasks.workunit.client.0.smithi077.stdout:0/6: rmdir d0 39 2024-04-26T04:52:49.875 INFO:tasks.workunit.client.1.smithi078.stdout:1/370: write d7/f13 [1989249,27906] 0 2024-04-26T04:52:49.876 INFO:tasks.workunit.client.1.smithi078.stdout:2/307: stat d8/f3a 0 2024-04-26T04:52:49.876 INFO:tasks.workunit.client.1.smithi078.stdout:6/376: dwrite d2/d1a/d5/f8 [0,4194304] 0 2024-04-26T04:52:49.877 INFO:tasks.workunit.client.0.smithi077.stdout:0/7: truncate d0/f2 849877 0 2024-04-26T04:52:49.877 INFO:tasks.workunit.client.0.smithi077.stdout:0/8: chown d0 120785 1 2024-04-26T04:52:49.877 INFO:tasks.workunit.client.1.smithi078.stdout:3/304: chown d16/d26/d2e/l44 9947 1 2024-04-26T04:52:49.879 INFO:tasks.workunit.client.0.smithi077.stdout:0/9: creat d0/f3 x:0 0 0 2024-04-26T04:52:49.880 INFO:tasks.workunit.client.1.smithi078.stdout:6/377: stat d2/d1a/d5/da/d35/f70 0 2024-04-26T04:52:49.881 INFO:tasks.workunit.client.0.smithi077.stdout:0/10: dread - d0/f3 zero size 2024-04-26T04:52:49.881 INFO:tasks.workunit.client.0.smithi077.stdout:0/11: readlink - no filename 2024-04-26T04:52:49.882 INFO:tasks.workunit.client.1.smithi078.stdout:1/371: read d7/d39/db/d37/f65 [3738434,125858] 0 2024-04-26T04:52:49.882 INFO:tasks.workunit.client.1.smithi078.stdout:3/305: symlink d16/d26/d35/l6b 0 2024-04-26T04:52:49.884 INFO:tasks.workunit.client.1.smithi078.stdout:7/307: dwrite d9/da/d20/f21 [4194304,4194304] 0 2024-04-26T04:52:49.884 INFO:tasks.workunit.client.1.smithi078.stdout:6/378: write d2/d1a/d5/da/d35/f3e [409303,99646] 0 2024-04-26T04:52:49.884 INFO:tasks.workunit.client.1.smithi078.stdout:1/372: dread - d7/d1e/f72 zero size 2024-04-26T04:52:49.887 INFO:tasks.workunit.client.1.smithi078.stdout:2/308: dread d8/d11/d18/d5d/d61/f4a [0,4194304] 0 2024-04-26T04:52:49.888 INFO:tasks.workunit.client.1.smithi078.stdout:6/379: write d2/d1a/d5/d2a/f3d [799015,111977] 0 2024-04-26T04:52:49.888 INFO:tasks.workunit.client.0.smithi077.stdout:0/12: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:52:49.891 INFO:tasks.workunit.client.1.smithi078.stdout:3/306: creat d16/d26/d2b/d54/f6c x:0 0 0 2024-04-26T04:52:49.892 INFO:tasks.workunit.client.1.smithi078.stdout:7/308: symlink d9/da/d20/l64 0 2024-04-26T04:52:49.893 INFO:tasks.workunit.client.1.smithi078.stdout:7/309: dread - d9/da/d33/f56 zero size 2024-04-26T04:52:49.894 INFO:tasks.workunit.client.1.smithi078.stdout:7/310: dread - d9/da/d11/d13/d1a/f4e zero size 2024-04-26T04:52:49.895 INFO:tasks.workunit.client.1.smithi078.stdout:1/373: mkdir d7/d39/dd/d82 0 2024-04-26T04:52:49.896 INFO:tasks.workunit.client.0.smithi077.stdout:0/13: link d0/f2 d0/f4 0 2024-04-26T04:52:49.901 INFO:tasks.workunit.client.0.smithi077.stdout:0/14: symlink d0/l5 0 2024-04-26T04:52:49.901 INFO:tasks.workunit.client.1.smithi078.stdout:8/325: truncate d0/f1a 8472332 0 2024-04-26T04:52:49.903 INFO:tasks.workunit.client.0.smithi077.stdout:0/15: symlink d0/l6 0 2024-04-26T04:52:49.904 INFO:tasks.workunit.client.1.smithi078.stdout:1/374: creat d7/d39/db/d47/d4a/f83 x:0 0 0 2024-04-26T04:52:49.905 INFO:tasks.workunit.client.1.smithi078.stdout:0/407: sync 2024-04-26T04:52:49.905 INFO:tasks.workunit.client.1.smithi078.stdout:5/246: sync 2024-04-26T04:52:49.906 INFO:tasks.workunit.client.0.smithi077.stdout:1/27: sync 2024-04-26T04:52:49.906 INFO:tasks.workunit.client.0.smithi077.stdout:9/57: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:6/67: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:7/19: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:2/31: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:8/84: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:5/9: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:5/10: dread - no filename 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:3/5: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.0.smithi077.stdout:4/13: sync 2024-04-26T04:52:49.907 INFO:tasks.workunit.client.1.smithi078.stdout:1/375: write d7/f13 [266035,8702] 0 2024-04-26T04:52:49.908 INFO:tasks.workunit.client.1.smithi078.stdout:6/380: link d2/d1a/d5/da/d35/f3e d2/de/d27/d58/f76 0 2024-04-26T04:52:49.908 INFO:tasks.workunit.client.0.smithi077.stdout:0/16: symlink d0/l7 0 2024-04-26T04:52:49.908 INFO:tasks.workunit.client.1.smithi078.stdout:8/326: dwrite d0/f10 [4194304,4194304] 0 2024-04-26T04:52:49.909 INFO:tasks.workunit.client.0.smithi077.stdout:0/17: read d0/f4 [137215,23133] 0 2024-04-26T04:52:49.909 INFO:tasks.workunit.client.0.smithi077.stdout:0/18: stat d0/f2 0 2024-04-26T04:52:49.913 INFO:tasks.workunit.client.0.smithi077.stdout:7/20: creat f8 x:0 0 0 2024-04-26T04:52:49.914 INFO:tasks.workunit.client.0.smithi077.stdout:7/21: chown f7 9677 1 2024-04-26T04:52:49.915 INFO:tasks.workunit.client.0.smithi077.stdout:1/28: read f2 [2614212,126860] 0 2024-04-26T04:52:49.917 INFO:tasks.workunit.client.1.smithi078.stdout:9/295: dwrite dc/df/f28 [0,4194304] 0 2024-04-26T04:52:49.917 INFO:tasks.workunit.client.0.smithi077.stdout:0/19: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:49.918 INFO:tasks.workunit.client.1.smithi078.stdout:9/296: write dc/f3d [893213,5853] 0 2024-04-26T04:52:49.919 INFO:tasks.workunit.client.1.smithi078.stdout:5/247: mknod d7/d21/c58 0 2024-04-26T04:52:49.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/14: write f2 [697317,82804] 0 2024-04-26T04:52:49.921 INFO:tasks.workunit.client.1.smithi078.stdout:5/248: read d7/d18/d32/d37/f3d [213189,71180] 0 2024-04-26T04:52:49.921 INFO:tasks.workunit.client.0.smithi077.stdout:2/32: mkdir d9/dc 0 2024-04-26T04:52:49.923 INFO:tasks.workunit.client.0.smithi077.stdout:1/29: dread f2 [0,4194304] 0 2024-04-26T04:52:49.923 INFO:tasks.workunit.client.0.smithi077.stdout:3/6: creat d0/f3 x:0 0 0 2024-04-26T04:52:49.927 INFO:tasks.workunit.client.0.smithi077.stdout:6/68: creat d8/f18 x:0 0 0 2024-04-26T04:52:49.928 INFO:tasks.workunit.client.0.smithi077.stdout:4/15: dread f2 [0,4194304] 0 2024-04-26T04:52:49.930 INFO:tasks.workunit.client.1.smithi078.stdout:4/351: sync 2024-04-26T04:52:49.930 INFO:tasks.workunit.client.0.smithi077.stdout:8/85: mkdir d4/df/d22 0 2024-04-26T04:52:49.932 INFO:tasks.workunit.client.1.smithi078.stdout:8/327: truncate d0/f2f 867678 0 2024-04-26T04:52:49.934 INFO:tasks.workunit.client.1.smithi078.stdout:2/309: dwrite d8/d11/f54 [0,4194304] 0 2024-04-26T04:52:49.934 INFO:tasks.workunit.client.1.smithi078.stdout:7/311: dread d9/da/d11/d13/d1a/f40 [0,4194304] 0 2024-04-26T04:52:49.937 INFO:tasks.workunit.client.1.smithi078.stdout:7/312: write d9/f14 [799582,124287] 0 2024-04-26T04:52:49.938 INFO:tasks.workunit.client.1.smithi078.stdout:6/381: dread d2/de/d27/f34 [0,4194304] 0 2024-04-26T04:52:49.939 INFO:tasks.workunit.client.1.smithi078.stdout:7/313: dread - d9/da/d33/f56 zero size 2024-04-26T04:52:49.941 INFO:tasks.workunit.client.1.smithi078.stdout:3/307: truncate fd 1104039 0 2024-04-26T04:52:49.941 INFO:tasks.workunit.client.1.smithi078.stdout:2/310: dread d8/d11/d14/f5f [0,4194304] 0 2024-04-26T04:52:49.943 INFO:tasks.workunit.client.1.smithi078.stdout:7/314: write d9/da/d11/d13/d1a/f42 [991408,75951] 0 2024-04-26T04:52:49.945 INFO:tasks.workunit.client.1.smithi078.stdout:8/328: dwrite d0/d19/d25/d36/f48 [0,4194304] 0 2024-04-26T04:52:49.946 INFO:tasks.workunit.client.1.smithi078.stdout:5/249: creat d7/d29/f59 x:0 0 0 2024-04-26T04:52:49.947 INFO:tasks.workunit.client.0.smithi077.stdout:5/11: mknod d1/c2 0 2024-04-26T04:52:49.950 INFO:tasks.workunit.client.0.smithi077.stdout:0/20: mknod d0/c8 0 2024-04-26T04:52:49.955 INFO:tasks.workunit.client.1.smithi078.stdout:5/250: dread d7/f8 [4194304,4194304] 0 2024-04-26T04:52:49.958 INFO:tasks.workunit.client.0.smithi077.stdout:2/33: dread f8 [0,4194304] 0 2024-04-26T04:52:49.960 INFO:tasks.workunit.client.1.smithi078.stdout:0/408: dwrite d2/d9/d18/fd [0,4194304] 0 2024-04-26T04:52:49.967 INFO:tasks.workunit.client.0.smithi077.stdout:4/16: mkdir d4 0 2024-04-26T04:52:49.969 INFO:tasks.workunit.client.0.smithi077.stdout:8/86: truncate d4/d1a/f1e 390849 0 2024-04-26T04:52:49.971 INFO:tasks.workunit.client.0.smithi077.stdout:9/58: unlink d9/dd/c16 0 2024-04-26T04:52:49.972 INFO:tasks.workunit.client.1.smithi078.stdout:2/311: dread d8/d11/d14/f3b [0,4194304] 0 2024-04-26T04:52:49.972 INFO:tasks.workunit.client.1.smithi078.stdout:4/352: write d0/d25/d20/d31/f3f [1414786,128191] 0 2024-04-26T04:52:49.973 INFO:tasks.workunit.client.0.smithi077.stdout:7/22: dwrite f1 [4194304,4194304] 0 2024-04-26T04:52:49.975 INFO:tasks.workunit.client.0.smithi077.stdout:5/12: creat d1/f3 x:0 0 0 2024-04-26T04:52:49.976 INFO:tasks.workunit.client.0.smithi077.stdout:3/7: write d0/f2 [881283,47485] 0 2024-04-26T04:52:49.977 INFO:tasks.workunit.client.0.smithi077.stdout:0/21: creat d0/f9 x:0 0 0 2024-04-26T04:52:49.980 INFO:tasks.workunit.client.1.smithi078.stdout:3/308: creat d16/d26/d2e/f6d x:0 0 0 2024-04-26T04:52:49.980 INFO:tasks.workunit.client.1.smithi078.stdout:2/312: dwrite d8/d11/d14/f1c [0,4194304] 0 2024-04-26T04:52:49.981 INFO:tasks.workunit.client.1.smithi078.stdout:3/309: chown l8 38 1 2024-04-26T04:52:49.982 INFO:tasks.workunit.client.1.smithi078.stdout:2/313: write d8/d11/d14/f1c [3190133,131027] 0 2024-04-26T04:52:49.982 INFO:tasks.workunit.client.0.smithi077.stdout:6/69: write d8/fb [318223,80334] 0 2024-04-26T04:52:49.987 INFO:tasks.workunit.client.1.smithi078.stdout:7/315: mkdir d9/da/d11/d13/d1a/d65 0 2024-04-26T04:52:49.988 INFO:tasks.workunit.client.0.smithi077.stdout:2/34: truncate f7 4448433 0 2024-04-26T04:52:49.989 INFO:tasks.workunit.client.0.smithi077.stdout:2/35: chown l6 2238521 1 2024-04-26T04:52:49.989 INFO:tasks.workunit.client.1.smithi078.stdout:1/376: rmdir d7/d39/dd/d82 0 2024-04-26T04:52:49.990 INFO:tasks.workunit.client.1.smithi078.stdout:1/377: write d7/d1e/f2f [1886075,25105] 0 2024-04-26T04:52:49.992 INFO:tasks.workunit.client.0.smithi077.stdout:8/87: dread - d4/d1a/f1c zero size 2024-04-26T04:52:49.993 INFO:tasks.workunit.client.0.smithi077.stdout:8/88: chown d4/fe 3553 1 2024-04-26T04:52:49.993 INFO:tasks.workunit.client.0.smithi077.stdout:8/89: chown d4/d1a 1144046 1 2024-04-26T04:52:49.993 INFO:tasks.workunit.client.0.smithi077.stdout:9/59: read f2 [1171294,111612] 0 2024-04-26T04:52:49.994 INFO:tasks.workunit.client.0.smithi077.stdout:8/90: chown d4/c14 3 1 2024-04-26T04:52:49.996 INFO:tasks.workunit.client.1.smithi078.stdout:9/297: creat dc/df/f59 x:0 0 0 2024-04-26T04:52:49.997 INFO:tasks.workunit.client.0.smithi077.stdout:7/23: dread - f7 zero size 2024-04-26T04:52:49.997 INFO:tasks.workunit.client.0.smithi077.stdout:7/24: write f1 [3311093,83611] 0 2024-04-26T04:52:49.998 INFO:tasks.workunit.client.0.smithi077.stdout:7/25: dread - f8 zero size 2024-04-26T04:52:49.998 INFO:tasks.workunit.client.0.smithi077.stdout:5/13: unlink d1/f3 0 2024-04-26T04:52:49.998 INFO:tasks.workunit.client.0.smithi077.stdout:5/14: dwrite - no filename 2024-04-26T04:52:49.998 INFO:tasks.workunit.client.0.smithi077.stdout:5/15: dwrite - no filename 2024-04-26T04:52:50.000 INFO:tasks.workunit.client.0.smithi077.stdout:3/8: mkdir d0/d4 0 2024-04-26T04:52:50.002 INFO:tasks.workunit.client.0.smithi077.stdout:0/22: mknod d0/ca 0 2024-04-26T04:52:50.002 INFO:tasks.workunit.client.0.smithi077.stdout:0/23: chown d0/l6 108141 1 2024-04-26T04:52:50.005 INFO:tasks.workunit.client.0.smithi077.stdout:1/30: rmdir d5 0 2024-04-26T04:52:50.008 INFO:tasks.workunit.client.0.smithi077.stdout:2/36: creat d9/fd x:0 0 0 2024-04-26T04:52:50.008 INFO:tasks.workunit.client.1.smithi078.stdout:3/310: creat d16/d26/d2b/d54/f6e x:0 0 0 2024-04-26T04:52:50.011 INFO:tasks.workunit.client.1.smithi078.stdout:2/314: symlink d8/d11/d18/d5d/d61/l6d 0 2024-04-26T04:52:50.015 INFO:tasks.workunit.client.1.smithi078.stdout:7/316: chown d9/da/d11/d13/l36 221746066 1 2024-04-26T04:52:50.016 INFO:tasks.workunit.client.1.smithi078.stdout:7/317: readlink d9/da/d11/d13/d1a/d52/l5d 0 2024-04-26T04:52:50.017 INFO:tasks.workunit.client.1.smithi078.stdout:7/318: write d9/da/d20/f21 [6889897,74192] 0 2024-04-26T04:52:50.021 INFO:tasks.workunit.client.1.smithi078.stdout:5/251: mkdir d7/d18/d1c/d5a 0 2024-04-26T04:52:50.021 INFO:tasks.workunit.client.1.smithi078.stdout:0/409: symlink d2/d9/d60/d7f/l86 0 2024-04-26T04:52:50.022 INFO:tasks.workunit.client.0.smithi077.stdout:7/26: symlink l9 0 2024-04-26T04:52:50.022 INFO:tasks.workunit.client.0.smithi077.stdout:7/27: fdatasync f1 0 2024-04-26T04:52:50.022 INFO:tasks.workunit.client.0.smithi077.stdout:5/16: creat d1/f4 x:0 0 0 2024-04-26T04:52:50.022 INFO:tasks.workunit.client.0.smithi077.stdout:5/17: fdatasync d1/f4 0 2024-04-26T04:52:50.023 INFO:tasks.workunit.client.0.smithi077.stdout:5/18: fdatasync d1/f4 0 2024-04-26T04:52:50.027 INFO:tasks.workunit.client.0.smithi077.stdout:0/24: mkdir d0/db 0 2024-04-26T04:52:50.030 INFO:tasks.workunit.client.0.smithi077.stdout:1/31: creat f6 x:0 0 0 2024-04-26T04:52:50.032 INFO:tasks.workunit.client.0.smithi077.stdout:2/37: creat d9/fe x:0 0 0 2024-04-26T04:52:50.033 INFO:tasks.workunit.client.0.smithi077.stdout:4/17: link l3 d4/l5 0 2024-04-26T04:52:50.034 INFO:tasks.workunit.client.1.smithi078.stdout:7/319: dread d9/da/d11/d13/d1a/d2d/f3f [4194304,4194304] 0 2024-04-26T04:52:50.034 INFO:tasks.workunit.client.0.smithi077.stdout:9/60: creat d9/dd/f1a x:0 0 0 2024-04-26T04:52:50.036 INFO:tasks.workunit.client.0.smithi077.stdout:9/61: chown d9/f10 15 1 2024-04-26T04:52:50.037 INFO:tasks.workunit.client.1.smithi078.stdout:1/378: fdatasync d7/d39/dd/d20/f3b 0 2024-04-26T04:52:50.038 INFO:tasks.workunit.client.0.smithi077.stdout:9/62: stat d9/c14 0 2024-04-26T04:52:50.039 INFO:tasks.workunit.client.0.smithi077.stdout:1/32: dwrite f6 [0,4194304] 0 2024-04-26T04:52:50.042 INFO:tasks.workunit.client.0.smithi077.stdout:5/19: creat d1/f5 x:0 0 0 2024-04-26T04:52:50.042 INFO:tasks.workunit.client.1.smithi078.stdout:6/382: getdents d2/d52 0 2024-04-26T04:52:50.043 INFO:tasks.workunit.client.1.smithi078.stdout:6/383: write d2/de/d27/f64 [997589,83419] 0 2024-04-26T04:52:50.043 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:49 smithi077 ceph-mon[26311]: pgmap v14: 65 pgs: 65 active+clean; 753 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 80 MiB/s wr, 360 op/s 2024-04-26T04:52:50.044 INFO:tasks.workunit.client.1.smithi078.stdout:6/384: chown d2/d1a/d5/d3a 485708 1 2024-04-26T04:52:50.045 INFO:tasks.workunit.client.0.smithi077.stdout:3/9: mkdir d0/d4/d5 0 2024-04-26T04:52:50.048 INFO:tasks.workunit.client.0.smithi077.stdout:0/25: creat d0/fc x:0 0 0 2024-04-26T04:52:50.049 INFO:tasks.workunit.client.0.smithi077.stdout:6/70: link l5 d8/l19 0 2024-04-26T04:52:50.049 INFO:tasks.workunit.client.1.smithi078.stdout:6/385: dwrite d2/d1a/d5/da/d35/f70 [0,4194304] 0 2024-04-26T04:52:50.051 INFO:tasks.workunit.client.1.smithi078.stdout:6/386: write d2/d1a/d5/da/d35/f70 [3342750,61482] 0 2024-04-26T04:52:50.052 INFO:tasks.workunit.client.1.smithi078.stdout:6/387: dread - d2/d1a/f43 zero size 2024-04-26T04:52:50.053 INFO:tasks.workunit.client.1.smithi078.stdout:2/315: mkdir d8/d11/d6e 0 2024-04-26T04:52:50.055 INFO:tasks.workunit.client.0.smithi077.stdout:4/18: rename d4/l5 to d4/l6 0 2024-04-26T04:52:50.055 INFO:tasks.workunit.client.0.smithi077.stdout:8/91: dwrite d4/f13 [0,4194304] 0 2024-04-26T04:52:50.059 INFO:tasks.workunit.client.1.smithi078.stdout:1/379: creat d7/d39/db/d49/f84 x:0 0 0 2024-04-26T04:52:50.059 INFO:tasks.workunit.client.0.smithi077.stdout:2/38: dread f7 [0,4194304] 0 2024-04-26T04:52:50.060 INFO:tasks.workunit.client.0.smithi077.stdout:1/33: creat f7 x:0 0 0 2024-04-26T04:52:50.061 INFO:tasks.workunit.client.0.smithi077.stdout:7/28: write f7 [418781,13231] 0 2024-04-26T04:52:50.061 INFO:tasks.workunit.client.1.smithi078.stdout:8/329: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:50.062 INFO:tasks.workunit.client.0.smithi077.stdout:7/29: fdatasync f1 0 2024-04-26T04:52:50.064 INFO:tasks.workunit.client.0.smithi077.stdout:5/20: fsync d1/f5 0 2024-04-26T04:52:50.064 INFO:tasks.workunit.client.1.smithi078.stdout:4/353: getdents d0/d25/d20/d31/d47 0 2024-04-26T04:52:50.065 INFO:tasks.workunit.client.0.smithi077.stdout:5/21: read - d1/f5 zero size 2024-04-26T04:52:50.066 INFO:tasks.workunit.client.0.smithi077.stdout:6/71: read f3 [296947,40493] 0 2024-04-26T04:52:50.070 INFO:tasks.workunit.client.0.smithi077.stdout:7/30: dwrite f1 [4194304,4194304] 0 2024-04-26T04:52:50.071 INFO:tasks.workunit.client.0.smithi077.stdout:7/31: write f8 [736705,52159] 0 2024-04-26T04:52:50.071 INFO:tasks.workunit.client.1.smithi078.stdout:6/388: mknod d2/d1a/d5/da/c77 0 2024-04-26T04:52:50.072 INFO:tasks.workunit.client.0.smithi077.stdout:6/72: dread f3 [0,4194304] 0 2024-04-26T04:52:50.073 INFO:tasks.workunit.client.0.smithi077.stdout:4/19: mknod d4/c7 0 2024-04-26T04:52:50.073 INFO:tasks.workunit.client.1.smithi078.stdout:9/298: dwrite dc/df/f1e [0,4194304] 0 2024-04-26T04:52:50.077 INFO:tasks.workunit.client.1.smithi078.stdout:3/311: dwrite d16/f3f [0,4194304] 0 2024-04-26T04:52:50.077 INFO:tasks.workunit.client.1.smithi078.stdout:0/410: truncate d2/d9/f1a 3259238 0 2024-04-26T04:52:50.078 INFO:tasks.workunit.client.1.smithi078.stdout:0/411: stat d2/d1e 0 2024-04-26T04:52:50.081 INFO:tasks.workunit.client.1.smithi078.stdout:1/380: mknod d7/d39/d4c/d3d/c85 0 2024-04-26T04:52:50.082 INFO:tasks.workunit.client.1.smithi078.stdout:8/330: truncate d0/f18 4186972 0 2024-04-26T04:52:50.083 INFO:tasks.workunit.client.1.smithi078.stdout:4/354: truncate d0/de/f34 13120764 0 2024-04-26T04:52:50.084 INFO:tasks.workunit.client.1.smithi078.stdout:4/355: dread - d0/d25/d20/d31/d33/f4d zero size 2024-04-26T04:52:50.085 INFO:tasks.workunit.client.1.smithi078.stdout:4/356: chown d0/d25/d20/d31/d33 248465995 1 2024-04-26T04:52:50.086 INFO:tasks.workunit.client.1.smithi078.stdout:6/389: creat d2/d1a/d5/d2a/d4d/f78 x:0 0 0 2024-04-26T04:52:50.086 INFO:tasks.workunit.client.1.smithi078.stdout:4/357: truncate d0/de/d22/d27/d56/d5a/f7d 236601 0 2024-04-26T04:52:50.088 INFO:tasks.workunit.client.0.smithi077.stdout:9/63: fdatasync f7 0 2024-04-26T04:52:50.088 INFO:tasks.workunit.client.0.smithi077.stdout:1/34: mknod c8 0 2024-04-26T04:52:50.088 INFO:tasks.workunit.client.0.smithi077.stdout:5/22: rename d1/c2 to d1/c6 0 2024-04-26T04:52:50.088 INFO:tasks.workunit.client.0.smithi077.stdout:1/35: chown f6 187576963 1 2024-04-26T04:52:50.089 INFO:tasks.workunit.client.0.smithi077.stdout:1/36: rmdir - no directory 2024-04-26T04:52:50.090 INFO:tasks.workunit.client.0.smithi077.stdout:7/32: mkdir da 0 2024-04-26T04:52:50.090 INFO:tasks.workunit.client.0.smithi077.stdout:7/33: chown f7 28 1 2024-04-26T04:52:50.091 INFO:tasks.workunit.client.1.smithi078.stdout:7/320: rename d9/da/d33/d4b/f63 to d9/da/f66 0 2024-04-26T04:52:50.091 INFO:tasks.workunit.client.1.smithi078.stdout:7/321: read - d9/da/d11/d13/d1a/f4e zero size 2024-04-26T04:52:50.093 INFO:tasks.workunit.client.1.smithi078.stdout:3/312: unlink d16/d26/f48 0 2024-04-26T04:52:50.093 INFO:tasks.workunit.client.0.smithi077.stdout:6/73: symlink d8/l1a 0 2024-04-26T04:52:50.094 INFO:tasks.workunit.client.0.smithi077.stdout:7/34: dread f7 [0,4194304] 0 2024-04-26T04:52:50.098 INFO:tasks.workunit.client.1.smithi078.stdout:8/331: mknod d0/d19/c64 0 2024-04-26T04:52:50.098 INFO:tasks.workunit.client.1.smithi078.stdout:2/316: write d8/d11/d14/f3b [11996243,19448] 0 2024-04-26T04:52:50.099 INFO:tasks.workunit.client.0.smithi077.stdout:4/20: fsync f2 0 2024-04-26T04:52:50.100 INFO:tasks.workunit.client.1.smithi078.stdout:2/317: write d8/f55 [955410,49218] 0 2024-04-26T04:52:50.101 INFO:tasks.workunit.client.1.smithi078.stdout:6/390: unlink d2/d1a/d41/f5e 0 2024-04-26T04:52:50.103 INFO:tasks.workunit.client.0.smithi077.stdout:3/10: rmdir d0/d4/d5 0 2024-04-26T04:52:50.103 INFO:tasks.workunit.client.1.smithi078.stdout:2/318: dread d8/f2b [0,4194304] 0 2024-04-26T04:52:50.104 INFO:tasks.workunit.client.1.smithi078.stdout:4/358: mknod d0/d25/d20/d31/d33/d4c/c87 0 2024-04-26T04:52:50.104 INFO:tasks.workunit.client.0.smithi077.stdout:0/26: link d0/ca d0/db/cd 0 2024-04-26T04:52:50.105 INFO:tasks.workunit.client.0.smithi077.stdout:0/27: stat d0 0 2024-04-26T04:52:50.106 INFO:tasks.workunit.client.0.smithi077.stdout:9/64: mkdir d9/dd/d1b 0 2024-04-26T04:52:50.107 INFO:tasks.workunit.client.1.smithi078.stdout:6/391: dwrite d2/d1a/d5/d2a/f3d [0,4194304] 0 2024-04-26T04:52:50.108 INFO:tasks.workunit.client.1.smithi078.stdout:6/392: stat d2/de/d27/l3f 0 2024-04-26T04:52:50.108 INFO:tasks.workunit.client.1.smithi078.stdout:6/393: chown d2/d1a/f6 190 1 2024-04-26T04:52:50.109 INFO:tasks.workunit.client.1.smithi078.stdout:7/322: rename d9/d57/l60 to d9/da/d11/d13/d1a/l67 0 2024-04-26T04:52:50.110 INFO:tasks.workunit.client.0.smithi077.stdout:1/37: chown l4 1206986 1 2024-04-26T04:52:50.112 INFO:tasks.workunit.client.0.smithi077.stdout:0/28: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:50.113 INFO:tasks.workunit.client.0.smithi077.stdout:0/29: dread - d0/f9 zero size 2024-04-26T04:52:50.115 INFO:tasks.workunit.client.1.smithi078.stdout:0/412: unlink d2/d9/d31/f36 0 2024-04-26T04:52:50.115 INFO:tasks.workunit.client.1.smithi078.stdout:7/323: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/f47 [0,4194304] 0 2024-04-26T04:52:50.121 INFO:tasks.workunit.client.0.smithi077.stdout:6/74: fdatasync d8/fe 0 2024-04-26T04:52:50.121 INFO:tasks.workunit.client.0.smithi077.stdout:1/38: dwrite f6 [0,4194304] 0 2024-04-26T04:52:50.129 INFO:tasks.workunit.client.0.smithi077.stdout:8/92: link d4/l8 d4/d17/l23 0 2024-04-26T04:52:50.131 INFO:tasks.workunit.client.0.smithi077.stdout:1/39: dwrite f6 [0,4194304] 0 2024-04-26T04:52:50.135 INFO:tasks.workunit.client.0.smithi077.stdout:4/21: creat d4/f8 x:0 0 0 2024-04-26T04:52:50.137 INFO:tasks.workunit.client.1.smithi078.stdout:2/319: symlink d8/l6f 0 2024-04-26T04:52:50.139 INFO:tasks.workunit.client.0.smithi077.stdout:3/11: creat d0/f6 x:0 0 0 2024-04-26T04:52:50.140 INFO:tasks.workunit.client.0.smithi077.stdout:3/12: truncate d0/f2 1333314 0 2024-04-26T04:52:50.144 INFO:tasks.workunit.client.0.smithi077.stdout:9/65: truncate d9/fc 8740960 0 2024-04-26T04:52:50.145 INFO:tasks.workunit.client.1.smithi078.stdout:4/359: dwrite d0/d25/d20/d31/f73 [0,4194304] 0 2024-04-26T04:52:50.146 INFO:tasks.workunit.client.1.smithi078.stdout:6/394: creat d2/d1a/d38/f79 x:0 0 0 2024-04-26T04:52:50.147 INFO:tasks.workunit.client.1.smithi078.stdout:6/395: read d2/de/d27/f34 [3660446,32055] 0 2024-04-26T04:52:50.147 INFO:tasks.workunit.client.1.smithi078.stdout:5/252: sync 2024-04-26T04:52:50.148 INFO:tasks.workunit.client.1.smithi078.stdout:5/253: dread - d7/d18/d32/f4a zero size 2024-04-26T04:52:50.149 INFO:tasks.workunit.client.1.smithi078.stdout:5/254: chown d7/d18/d32/d37 175176917 1 2024-04-26T04:52:50.150 INFO:tasks.workunit.client.1.smithi078.stdout:5/255: chown d7/d18/f44 5 1 2024-04-26T04:52:50.150 INFO:tasks.workunit.client.1.smithi078.stdout:5/256: write d7/d29/f36 [3953695,29095] 0 2024-04-26T04:52:50.154 INFO:tasks.workunit.client.0.smithi077.stdout:0/30: creat d0/fe x:0 0 0 2024-04-26T04:52:50.156 INFO:tasks.workunit.client.1.smithi078.stdout:3/313: link d16/d26/d2b/d54/d5a/f5d d16/d26/d2b/d66/f6f 0 2024-04-26T04:52:50.157 INFO:tasks.workunit.client.1.smithi078.stdout:3/314: dread - d16/d26/d2b/d31/f65 zero size 2024-04-26T04:52:50.157 INFO:tasks.workunit.client.1.smithi078.stdout:3/315: readlink d16/l64 0 2024-04-26T04:52:50.158 INFO:tasks.workunit.client.1.smithi078.stdout:0/413: dread - d2/d9/d31/f66 zero size 2024-04-26T04:52:50.158 INFO:tasks.workunit.client.1.smithi078.stdout:3/316: write d16/f39 [3515888,54377] 0 2024-04-26T04:52:50.160 INFO:tasks.workunit.client.1.smithi078.stdout:7/324: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68 0 2024-04-26T04:52:50.161 INFO:tasks.workunit.client.1.smithi078.stdout:1/381: truncate d7/d39/f12 1097935 0 2024-04-26T04:52:50.163 INFO:tasks.workunit.client.1.smithi078.stdout:4/360: dread d0/de/f34 [4194304,4194304] 0 2024-04-26T04:52:50.164 INFO:tasks.workunit.client.0.smithi077.stdout:6/75: rename d8/l12 to d8/l1b 0 2024-04-26T04:52:50.165 INFO:tasks.workunit.client.0.smithi077.stdout:5/23: chown d1/c6 3 1 2024-04-26T04:52:50.166 INFO:tasks.workunit.client.0.smithi077.stdout:5/24: write d1/f5 [1013495,113899] 0 2024-04-26T04:52:50.166 INFO:tasks.workunit.client.0.smithi077.stdout:5/25: fsync d1/f4 0 2024-04-26T04:52:50.170 INFO:tasks.workunit.client.0.smithi077.stdout:7/35: rename f7 to da/fb 0 2024-04-26T04:52:50.174 INFO:tasks.workunit.client.0.smithi077.stdout:1/40: mkdir d9 0 2024-04-26T04:52:50.175 INFO:tasks.workunit.client.1.smithi078.stdout:6/396: dwrite d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:50.175 INFO:tasks.workunit.client.0.smithi077.stdout:7/36: dwrite f1 [4194304,4194304] 0 2024-04-26T04:52:50.177 INFO:tasks.workunit.client.0.smithi077.stdout:4/22: symlink d4/l9 0 2024-04-26T04:52:50.183 INFO:tasks.workunit.client.0.smithi077.stdout:7/37: dread f8 [0,4194304] 0 2024-04-26T04:52:50.184 INFO:tasks.workunit.client.0.smithi077.stdout:7/38: write f1 [4293731,73913] 0 2024-04-26T04:52:50.184 INFO:tasks.workunit.client.0.smithi077.stdout:7/39: fsync f8 0 2024-04-26T04:52:50.185 INFO:tasks.workunit.client.0.smithi077.stdout:7/40: chown f8 20 1 2024-04-26T04:52:50.186 INFO:tasks.workunit.client.0.smithi077.stdout:6/76: mkdir d8/d1c 0 2024-04-26T04:52:50.189 INFO:tasks.workunit.client.0.smithi077.stdout:5/26: symlink d1/l7 0 2024-04-26T04:52:50.189 INFO:tasks.workunit.client.0.smithi077.stdout:6/77: dread d8/fb [0,4194304] 0 2024-04-26T04:52:50.190 INFO:tasks.workunit.client.0.smithi077.stdout:5/27: chown d1/f5 26215704 1 2024-04-26T04:52:50.192 INFO:tasks.workunit.client.1.smithi078.stdout:8/332: truncate d0/d19/d52/f24 223749 0 2024-04-26T04:52:50.193 INFO:tasks.workunit.client.1.smithi078.stdout:8/333: write d0/d19/d25/d32/f4b [1530150,1196] 0 2024-04-26T04:52:50.195 INFO:tasks.workunit.client.0.smithi077.stdout:8/93: creat d4/df/d22/f24 x:0 0 0 2024-04-26T04:52:50.196 INFO:tasks.workunit.client.1.smithi078.stdout:3/317: creat d16/d26/d2b/d54/d5a/f70 x:0 0 0 2024-04-26T04:52:50.198 INFO:tasks.workunit.client.0.smithi077.stdout:5/28: dwrite d1/f5 [0,4194304] 0 2024-04-26T04:52:50.202 INFO:tasks.workunit.client.1.smithi078.stdout:1/382: mkdir d7/d39/db/d37/d58/d86 0 2024-04-26T04:52:50.202 INFO:tasks.workunit.client.1.smithi078.stdout:9/299: sync 2024-04-26T04:52:50.204 INFO:tasks.workunit.client.0.smithi077.stdout:4/23: truncate f2 1494065 0 2024-04-26T04:52:50.205 INFO:tasks.workunit.client.1.smithi078.stdout:4/361: mknod d0/de/d22/c88 0 2024-04-26T04:52:50.207 INFO:tasks.workunit.client.1.smithi078.stdout:2/320: mkdir d8/d11/d18/d5d/d61/d1a/d34/d70 0 2024-04-26T04:52:50.208 INFO:tasks.workunit.client.0.smithi077.stdout:0/31: creat d0/db/ff x:0 0 0 2024-04-26T04:52:50.208 INFO:tasks.workunit.client.0.smithi077.stdout:0/32: readlink d0/l6 0 2024-04-26T04:52:50.210 INFO:tasks.workunit.client.0.smithi077.stdout:0/33: dread - d0/db/ff zero size 2024-04-26T04:52:50.210 INFO:tasks.workunit.client.1.smithi078.stdout:9/300: dwrite dc/df/d17/f44 [0,4194304] 0 2024-04-26T04:52:50.210 INFO:tasks.workunit.client.1.smithi078.stdout:9/301: chown dc/df/d4b/l1f 13446544 1 2024-04-26T04:52:50.210 INFO:tasks.workunit.client.1.smithi078.stdout:6/397: write d2/d49/f47 [558916,34090] 0 2024-04-26T04:52:50.213 INFO:tasks.workunit.client.1.smithi078.stdout:6/398: write d2/d1a/d5/da/d35/f66 [968553,94485] 0 2024-04-26T04:52:50.219 INFO:tasks.workunit.client.0.smithi077.stdout:8/94: unlink d4/df/d18/f19 0 2024-04-26T04:52:50.219 INFO:tasks.workunit.client.1.smithi078.stdout:2/321: dread d8/d11/d14/f19 [0,4194304] 0 2024-04-26T04:52:50.220 INFO:tasks.workunit.client.1.smithi078.stdout:1/383: dread d7/d39/d17/f3c [0,4194304] 0 2024-04-26T04:52:50.220 INFO:tasks.workunit.client.1.smithi078.stdout:2/322: fsync d8/d11/d18/d5d/d61/f1b 0 2024-04-26T04:52:50.221 INFO:tasks.workunit.client.1.smithi078.stdout:2/323: readlink d8/d11/d18/d5d/d61/d1a/l44 0 2024-04-26T04:52:50.222 INFO:tasks.workunit.client.1.smithi078.stdout:3/318: mkdir d16/d26/d2b/d31/d71 0 2024-04-26T04:52:50.223 INFO:tasks.workunit.client.1.smithi078.stdout:3/319: chown d16/d26/d2b/l2d 1 1 2024-04-26T04:52:50.223 INFO:tasks.workunit.client.0.smithi077.stdout:1/41: mknod d9/ca 0 2024-04-26T04:52:50.225 INFO:tasks.workunit.client.1.smithi078.stdout:0/414: link d2/d1e/f67 d2/d9/d60/d6a/f87 0 2024-04-26T04:52:50.226 INFO:tasks.workunit.client.1.smithi078.stdout:3/320: dread d16/d26/d2b/d54/d5a/f5d [0,4194304] 0 2024-04-26T04:52:50.227 INFO:tasks.workunit.client.1.smithi078.stdout:3/321: write d16/d26/d2e/f43 [278963,86301] 0 2024-04-26T04:52:50.228 INFO:tasks.workunit.client.1.smithi078.stdout:7/325: sync 2024-04-26T04:52:50.228 INFO:tasks.workunit.client.0.smithi077.stdout:4/24: rename l0 to d4/la 0 2024-04-26T04:52:50.230 INFO:tasks.workunit.client.1.smithi078.stdout:7/326: read d9/da/d11/d13/d1a/d2d/f4f [791054,58089] 0 2024-04-26T04:52:50.230 INFO:tasks.workunit.client.1.smithi078.stdout:7/327: stat d9/d57 0 2024-04-26T04:52:50.231 INFO:tasks.workunit.client.0.smithi077.stdout:3/13: rmdir d0/d4 0 2024-04-26T04:52:50.233 INFO:tasks.workunit.client.0.smithi077.stdout:0/34: symlink d0/l10 0 2024-04-26T04:52:50.234 INFO:tasks.workunit.client.1.smithi078.stdout:5/257: truncate f5 925590 0 2024-04-26T04:52:50.234 INFO:tasks.workunit.client.1.smithi078.stdout:5/258: write d7/d29/f59 [500115,94817] 0 2024-04-26T04:52:50.235 INFO:tasks.workunit.client.1.smithi078.stdout:5/259: dread - d7/d29/d46/f54 zero size 2024-04-26T04:52:50.236 INFO:tasks.workunit.client.1.smithi078.stdout:5/260: stat d7/d11 0 2024-04-26T04:52:50.239 INFO:tasks.workunit.client.0.smithi077.stdout:6/78: symlink d8/d1c/l1d 0 2024-04-26T04:52:50.240 INFO:tasks.workunit.client.0.smithi077.stdout:7/41: dwrite da/fb [0,4194304] 0 2024-04-26T04:52:50.240 INFO:tasks.workunit.client.0.smithi077.stdout:3/14: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:52:50.242 INFO:tasks.workunit.client.1.smithi078.stdout:6/399: creat d2/d52/f7a x:0 0 0 2024-04-26T04:52:50.242 INFO:tasks.workunit.client.0.smithi077.stdout:3/15: truncate d0/f6 650072 0 2024-04-26T04:52:50.245 INFO:tasks.workunit.client.0.smithi077.stdout:6/79: dread f4 [0,4194304] 0 2024-04-26T04:52:50.247 INFO:tasks.workunit.client.0.smithi077.stdout:3/16: dread d0/f3 [0,4194304] 0 2024-04-26T04:52:50.247 INFO:tasks.workunit.client.1.smithi078.stdout:6/400: dwrite d2/f71 [0,4194304] 0 2024-04-26T04:52:50.248 INFO:tasks.workunit.client.1.smithi078.stdout:6/401: dread - d2/f53 zero size 2024-04-26T04:52:50.250 INFO:tasks.workunit.client.0.smithi077.stdout:1/42: mkdir d9/db 0 2024-04-26T04:52:50.251 INFO:tasks.workunit.client.1.smithi078.stdout:1/384: fdatasync d7/d1e/f31 0 2024-04-26T04:52:50.252 INFO:tasks.workunit.client.1.smithi078.stdout:1/385: dread - d7/d1e/f72 zero size 2024-04-26T04:52:50.254 INFO:tasks.workunit.client.1.smithi078.stdout:2/324: creat d8/d53/f71 x:0 0 0 2024-04-26T04:52:50.254 INFO:tasks.workunit.client.1.smithi078.stdout:1/386: fdatasync d7/d39/dd/d20/f7b 0 2024-04-26T04:52:50.255 INFO:tasks.workunit.client.0.smithi077.stdout:4/25: symlink d4/lb 0 2024-04-26T04:52:50.256 INFO:tasks.workunit.client.0.smithi077.stdout:4/26: fdatasync d4/f8 0 2024-04-26T04:52:50.256 INFO:tasks.workunit.client.1.smithi078.stdout:1/387: stat d7/d39/d17/l22 0 2024-04-26T04:52:50.260 INFO:tasks.workunit.client.0.smithi077.stdout:0/35: mkdir d0/db/d11 0 2024-04-26T04:52:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:49 smithi078 ceph-mon[33746]: pgmap v14: 65 pgs: 65 active+clean; 753 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 80 MiB/s wr, 360 op/s 2024-04-26T04:52:50.267 INFO:tasks.workunit.client.0.smithi077.stdout:7/42: creat da/fc x:0 0 0 2024-04-26T04:52:50.270 INFO:tasks.workunit.client.0.smithi077.stdout:5/29: truncate d1/f5 723615 0 2024-04-26T04:52:50.274 INFO:tasks.workunit.client.0.smithi077.stdout:6/80: fdatasync d8/f17 0 2024-04-26T04:52:50.275 INFO:tasks.workunit.client.1.smithi078.stdout:6/402: dread d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:50.275 INFO:tasks.workunit.client.0.smithi077.stdout:6/81: chown d8/f13 6219620 1 2024-04-26T04:52:50.275 INFO:tasks.workunit.client.0.smithi077.stdout:6/82: stat d8/f13 0 2024-04-26T04:52:50.276 INFO:tasks.workunit.client.0.smithi077.stdout:3/17: mkdir d0/d7 0 2024-04-26T04:52:50.277 INFO:tasks.workunit.client.0.smithi077.stdout:1/43: fsync f2 0 2024-04-26T04:52:50.279 INFO:tasks.workunit.client.1.smithi078.stdout:4/362: mknod d0/d25/d20/d31/d33/d44/d55/d5e/d75/c89 0 2024-04-26T04:52:50.280 INFO:tasks.workunit.client.1.smithi078.stdout:4/363: write d0/d25/d20/d31/d33/d44/d55/f5d [3534701,74656] 0 2024-04-26T04:52:50.280 INFO:tasks.workunit.client.1.smithi078.stdout:5/261: creat d7/d29/f5b x:0 0 0 2024-04-26T04:52:50.281 INFO:tasks.workunit.client.0.smithi077.stdout:0/36: creat d0/f12 x:0 0 0 2024-04-26T04:52:50.282 INFO:tasks.workunit.client.1.smithi078.stdout:5/262: truncate d7/d29/f53 38303 0 2024-04-26T04:52:50.282 INFO:tasks.workunit.client.1.smithi078.stdout:5/263: write d7/d21/f42 [515432,26156] 0 2024-04-26T04:52:50.285 INFO:tasks.workunit.client.0.smithi077.stdout:5/30: mkdir d1/d8 0 2024-04-26T04:52:50.290 INFO:tasks.workunit.client.1.smithi078.stdout:0/415: creat d2/d9/d78/d84/f88 x:0 0 0 2024-04-26T04:52:50.292 INFO:tasks.workunit.client.0.smithi077.stdout:3/18: fsync d0/f3 0 2024-04-26T04:52:50.292 INFO:tasks.workunit.client.0.smithi077.stdout:7/43: dwrite da/fb [4194304,4194304] 0 2024-04-26T04:52:50.293 INFO:tasks.workunit.client.0.smithi077.stdout:3/19: chown d0/f3 91864589 1 2024-04-26T04:52:50.296 INFO:tasks.workunit.client.1.smithi078.stdout:1/388: dread d7/d39/f12 [0,4194304] 0 2024-04-26T04:52:50.296 INFO:tasks.workunit.client.1.smithi078.stdout:1/389: chown d7/d39/d4c/d3d 211439 1 2024-04-26T04:52:50.297 INFO:tasks.workunit.client.0.smithi077.stdout:3/20: dread d0/f3 [0,4194304] 0 2024-04-26T04:52:50.298 INFO:tasks.workunit.client.0.smithi077.stdout:3/21: write d0/f3 [3301939,105946] 0 2024-04-26T04:52:50.299 INFO:tasks.workunit.client.1.smithi078.stdout:6/403: symlink d2/de/l7b 0 2024-04-26T04:52:50.299 INFO:tasks.workunit.client.0.smithi077.stdout:3/22: stat d0 0 2024-04-26T04:52:50.301 INFO:tasks.workunit.client.1.smithi078.stdout:7/328: symlink d9/da/d11/l69 0 2024-04-26T04:52:50.302 INFO:tasks.workunit.client.0.smithi077.stdout:0/37: mkdir d0/db/d13 0 2024-04-26T04:52:50.305 INFO:tasks.workunit.client.1.smithi078.stdout:0/416: dread d2/f10 [0,4194304] 0 2024-04-26T04:52:50.306 INFO:tasks.workunit.client.0.smithi077.stdout:5/31: symlink d1/l9 0 2024-04-26T04:52:50.306 INFO:tasks.workunit.client.1.smithi078.stdout:6/404: dwrite d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:52:50.307 INFO:tasks.workunit.client.1.smithi078.stdout:4/364: mknod d0/d25/d20/d31/d33/d44/c8a 0 2024-04-26T04:52:50.308 INFO:tasks.workunit.client.1.smithi078.stdout:5/264: chown d7/d18/d1c/c56 12549342 1 2024-04-26T04:52:50.310 INFO:tasks.workunit.client.0.smithi077.stdout:7/44: creat da/fd x:0 0 0 2024-04-26T04:52:50.311 INFO:tasks.workunit.client.1.smithi078.stdout:9/302: mknod dc/df/d37/c5a 0 2024-04-26T04:52:50.311 INFO:tasks.workunit.client.1.smithi078.stdout:9/303: chown dc/f54 249 1 2024-04-26T04:52:50.315 INFO:tasks.workunit.client.1.smithi078.stdout:8/334: getdents d0/d19/d25 0 2024-04-26T04:52:50.319 INFO:tasks.workunit.client.1.smithi078.stdout:8/335: dwrite d0/d19/d25/d36/f37 [0,4194304] 0 2024-04-26T04:52:50.320 INFO:tasks.workunit.client.0.smithi077.stdout:5/32: rename d1/l7 to d1/la 0 2024-04-26T04:52:50.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/83: dwrite d8/ff [0,4194304] 0 2024-04-26T04:52:50.322 INFO:tasks.workunit.client.1.smithi078.stdout:3/322: creat d16/d26/d2b/f72 x:0 0 0 2024-04-26T04:52:50.324 INFO:tasks.workunit.client.1.smithi078.stdout:1/390: mkdir d7/d39/db/d47/d4a/d87 0 2024-04-26T04:52:50.325 INFO:tasks.workunit.client.1.smithi078.stdout:1/391: stat d7/d39/d4c/f4e 0 2024-04-26T04:52:50.325 INFO:tasks.workunit.client.1.smithi078.stdout:8/336: dwrite d0/d19/d25/d32/f4b [4194304,4194304] 0 2024-04-26T04:52:50.327 INFO:tasks.workunit.client.0.smithi077.stdout:3/23: creat d0/d7/f8 x:0 0 0 2024-04-26T04:52:50.329 INFO:tasks.workunit.client.0.smithi077.stdout:3/24: chown d0/d7 23284456 1 2024-04-26T04:52:50.331 INFO:tasks.workunit.client.0.smithi077.stdout:0/38: creat d0/db/d11/f14 x:0 0 0 2024-04-26T04:52:50.331 INFO:tasks.workunit.client.0.smithi077.stdout:4/27: dwrite f2 [0,4194304] 0 2024-04-26T04:52:50.332 INFO:tasks.workunit.client.1.smithi078.stdout:0/417: creat d2/d9/d60/d6a/f89 x:0 0 0 2024-04-26T04:52:50.336 INFO:tasks.workunit.client.1.smithi078.stdout:6/405: creat d2/de/d27/d58/d5b/f7c x:0 0 0 2024-04-26T04:52:50.342 INFO:tasks.workunit.client.0.smithi077.stdout:4/28: dread f2 [0,4194304] 0 2024-04-26T04:52:50.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/44: getdents d9 0 2024-04-26T04:52:50.491 INFO:tasks.workunit.client.1.smithi078.stdout:9/304: dwrite dc/df/f1d [0,4194304] 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:1/392: unlink d7/d39/dd/d20/f3b 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:1/393: readlink d7/d39/db/d47/l59 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:8/337: rename d0/d19/c63 to d0/d19/d25/d32/c65 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:9/305: dwrite dc/df/f59 [0,4194304] 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:8/338: write d0/d19/d52/d35/d3f/f5e [72580,100011] 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:7/329: dread d9/da/d11/d13/d1a/d52/d5a/f44 [0,4194304] 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:5/265: mknod d7/da/d1e/d4d/c5c 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:2/325: link d8/d11/d18/d5d/d61/d1a/c5a d8/d11/d18/d5d/d61/d1a/c72 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:3/323: link d16/d26/d2e/f51 d16/d26/d2e/f73 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:4/365: truncate d0/de/d22/d26/f57 819017 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:4/366: truncate d0/de/d22/d27/d56/f5c 4351193 0 2024-04-26T04:52:50.492 INFO:tasks.workunit.client.1.smithi078.stdout:6/406: unlink d2/d1a/d5/d3a/d42/c51 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:2/326: dread d8/d11/d14/f40 [0,4194304] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:5/266: mkdir d7/d11/d5d 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:5/267: write d7/d21/f4e [43370,23242] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:4/367: mknod d0/de/d22/d27/d56/d5a/c8b 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:9/306: mknod dc/df/d4b/d22/c5b 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:8/339: dread d0/f18 [0,4194304] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:9/307: dwrite dc/df/f59 [0,4194304] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:9/308: write dc/df/d4b/d22/d2d/f4d [1195513,119970] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:6/407: rmdir d2/d1a/d5/d2a/d4d 39 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/418: sync 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/419: write d2/f6c [1908067,17690] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:3/324: write d16/d26/d2b/d66/f6f [1736788,44056] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:3/325: truncate d16/f57 1083109 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:7/330: dwrite d9/f10 [0,4194304] 0 2024-04-26T04:52:50.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/420: dwrite d2/f2e [0,4194304] 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:9/309: truncate dc/df/d17/f4c 3005095 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:6/408: truncate d2/d1a/d5/d3a/f62 985825 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:6/409: readlink d2/de/l6e 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:7/331: creat d9/f6a x:0 0 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:1/394: sync 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:4/368: rename d0/de/d22/d26/l74 to d0/d25/d20/l8c 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:7/332: mknod d9/c6b 0 2024-04-26T04:52:50.494 INFO:tasks.workunit.client.1.smithi078.stdout:6/410: creat d2/d1a/d5/d3a/d42/f7d x:0 0 0 2024-04-26T04:52:50.495 INFO:tasks.workunit.client.1.smithi078.stdout:4/369: rmdir d0/de/d22/d71 39 2024-04-26T04:52:50.495 INFO:tasks.workunit.client.1.smithi078.stdout:9/310: dwrite dc/df/d27/f53 [0,4194304] 0 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:0/39: dwrite d0/fc [0,4194304] 0 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:4/29: dread - d4/f8 zero size 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:3/25: unlink d0/f2 0 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:0/40: dread d0/fc [0,4194304] 0 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:0/41: fsync d0/db/d11/f14 0 2024-04-26T04:52:50.496 INFO:tasks.workunit.client.0.smithi077.stdout:4/30: mknod d4/cc 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/42: dread d0/f4 [0,4194304] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/43: chown d0 9751 1 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/44: truncate d0/fc 4382364 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/45: dread - d0/f12 zero size 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:4/31: dread f2 [0,4194304] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:4/32: stat d4/lb 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:6/84: write d8/fe [2674308,130367] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/46: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:1/45: creat d9/db/fc x:0 0 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/47: write d0/db/d11/f14 [1024364,5746] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/48: chown d0 1700 1 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:1/46: dwrite d9/db/fc [0,4194304] 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:4/33: mknod d4/cd 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/49: creat d0/f15 x:0 0 0 2024-04-26T04:52:50.497 INFO:tasks.workunit.client.0.smithi077.stdout:1/47: mkdir d9/db/dd 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:4/34: mkdir d4/de 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:4/35: write d4/f8 [1024277,52247] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:0/50: creat d0/db/f16 x:0 0 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:5/33: dwrite d1/f5 [0,4194304] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:0/51: mknod d0/c17 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:5/34: dwrite d1/f4 [0,4194304] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:0/52: truncate d0/db/ff 934959 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:6/85: dwrite d8/f13 [0,4194304] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:5/35: write d1/f4 [585575,85944] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:3/26: write d0/f1 [392716,124534] 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:4/36: mknod d4/de/cf 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:0/53: symlink d0/db/d11/l18 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:1/48: truncate d9/db/fc 1578215 0 2024-04-26T04:52:50.498 INFO:tasks.workunit.client.0.smithi077.stdout:3/27: mkdir d0/d9 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/28: write d0/d7/f8 [214124,31286] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:4/37: fsync f2 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/29: write d0/f6 [1508195,44921] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/30: dread d0/d7/f8 [0,4194304] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/31: write d0/d7/f8 [130591,29930] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:4/38: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:1/49: truncate f3 297197 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:1/50: stat f7 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:1/51: write f6 [4503140,18137] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/32: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/33: dread d0/f3 [0,4194304] 0 2024-04-26T04:52:50.499 INFO:tasks.workunit.client.0.smithi077.stdout:0/54: getdents d0/db/d13 0 2024-04-26T04:52:50.500 INFO:tasks.workunit.client.0.smithi077.stdout:4/39: creat d4/f10 x:0 0 0 2024-04-26T04:52:50.500 INFO:tasks.workunit.client.0.smithi077.stdout:1/52: creat d9/db/fe x:0 0 0 2024-04-26T04:52:50.500 INFO:tasks.workunit.client.0.smithi077.stdout:6/86: link l5 d8/l1e 0 2024-04-26T04:52:50.500 INFO:tasks.workunit.client.1.smithi078.stdout:0/421: link d2/d9/d60/d6a/f89 d2/d9/d18/f8a 0 2024-04-26T04:52:50.500 INFO:tasks.workunit.client.1.smithi078.stdout:9/311: dwrite dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:50.501 INFO:tasks.workunit.client.1.smithi078.stdout:8/340: rename d0/c49 to d0/d19/d25/d36/c66 0 2024-04-26T04:52:50.503 INFO:tasks.workunit.client.0.smithi077.stdout:3/34: mknod d0/ca 0 2024-04-26T04:52:50.505 INFO:tasks.workunit.client.1.smithi078.stdout:0/422: dwrite d2/d39/f7b [0,4194304] 0 2024-04-26T04:52:50.506 INFO:tasks.workunit.client.0.smithi077.stdout:0/55: mkdir d0/d19 0 2024-04-26T04:52:50.514 INFO:tasks.workunit.client.0.smithi077.stdout:5/36: link d1/la d1/lb 0 2024-04-26T04:52:50.517 INFO:tasks.workunit.client.0.smithi077.stdout:4/40: symlink d4/l11 0 2024-04-26T04:52:50.519 INFO:tasks.workunit.client.1.smithi078.stdout:9/312: creat dc/df/d27/f5c x:0 0 0 2024-04-26T04:52:50.520 INFO:tasks.workunit.client.1.smithi078.stdout:9/313: write dc/df/d4b/f1c [1463551,64194] 0 2024-04-26T04:52:50.522 INFO:tasks.workunit.client.1.smithi078.stdout:3/326: rename d16/d26/d35 to d16/d26/d2e/d74 0 2024-04-26T04:52:50.522 INFO:tasks.workunit.client.1.smithi078.stdout:3/327: fsync d16/f20 0 2024-04-26T04:52:50.523 INFO:tasks.workunit.client.0.smithi077.stdout:6/87: fsync d8/f11 0 2024-04-26T04:52:50.523 INFO:tasks.workunit.client.1.smithi078.stdout:3/328: stat d16/d26/d2e/d5f/d49/c63 0 2024-04-26T04:52:50.524 INFO:tasks.workunit.client.1.smithi078.stdout:9/314: dread dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:50.525 INFO:tasks.workunit.client.1.smithi078.stdout:9/315: write dc/df/f28 [2043593,130950] 0 2024-04-26T04:52:50.527 INFO:tasks.workunit.client.0.smithi077.stdout:3/35: creat d0/fb x:0 0 0 2024-04-26T04:52:50.535 INFO:tasks.workunit.client.0.smithi077.stdout:1/53: symlink d9/db/dd/lf 0 2024-04-26T04:52:50.536 INFO:tasks.workunit.client.0.smithi077.stdout:1/54: write f6 [931177,89503] 0 2024-04-26T04:52:50.539 INFO:tasks.workunit.client.0.smithi077.stdout:6/88: mknod d8/c1f 0 2024-04-26T04:52:50.541 INFO:tasks.workunit.client.1.smithi078.stdout:3/329: creat d16/d26/d2e/f75 x:0 0 0 2024-04-26T04:52:50.542 INFO:tasks.workunit.client.1.smithi078.stdout:3/330: truncate d16/d26/d2e/f75 769117 0 2024-04-26T04:52:50.542 INFO:tasks.workunit.client.0.smithi077.stdout:3/36: fdatasync d0/d7/f8 0 2024-04-26T04:52:50.544 INFO:tasks.workunit.client.0.smithi077.stdout:0/56: creat d0/d19/f1a x:0 0 0 2024-04-26T04:52:50.546 INFO:tasks.workunit.client.0.smithi077.stdout:5/37: creat d1/d8/fc x:0 0 0 2024-04-26T04:52:50.547 INFO:tasks.workunit.client.1.smithi078.stdout:6/411: truncate d2/d1a/d38/f79 37124 0 2024-04-26T04:52:50.547 INFO:tasks.workunit.client.0.smithi077.stdout:5/38: write d1/f5 [44775,19896] 0 2024-04-26T04:52:50.548 INFO:tasks.workunit.client.1.smithi078.stdout:0/423: truncate d2/d1e/f67 734727 0 2024-04-26T04:52:50.549 INFO:tasks.workunit.client.1.smithi078.stdout:4/370: creat d0/de/d22/d26/f8d x:0 0 0 2024-04-26T04:52:50.550 INFO:tasks.workunit.client.1.smithi078.stdout:4/371: write d0/f8 [2186930,48082] 0 2024-04-26T04:52:50.551 INFO:tasks.workunit.client.0.smithi077.stdout:4/41: unlink d4/f10 0 2024-04-26T04:52:50.551 INFO:tasks.workunit.client.0.smithi077.stdout:4/42: fsync d4/f8 0 2024-04-26T04:52:50.552 INFO:tasks.workunit.client.0.smithi077.stdout:4/43: truncate f2 5009995 0 2024-04-26T04:52:50.554 INFO:tasks.workunit.client.1.smithi078.stdout:2/327: sync 2024-04-26T04:52:50.554 INFO:tasks.workunit.client.1.smithi078.stdout:5/268: sync 2024-04-26T04:52:50.556 INFO:tasks.workunit.client.1.smithi078.stdout:7/333: getdents d9/da/d11/d13/d1a/d52 0 2024-04-26T04:52:50.556 INFO:tasks.workunit.client.1.smithi078.stdout:7/334: write d9/f14 [7892749,91752] 0 2024-04-26T04:52:50.558 INFO:tasks.workunit.client.0.smithi077.stdout:1/55: dread d9/db/fc [0,4194304] 0 2024-04-26T04:52:50.559 INFO:tasks.workunit.client.1.smithi078.stdout:3/331: read f13 [1129764,91102] 0 2024-04-26T04:52:50.560 INFO:tasks.workunit.client.0.smithi077.stdout:4/44: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:50.561 INFO:tasks.workunit.client.0.smithi077.stdout:0/57: rename d0/fc to d0/db/f1b 0 2024-04-26T04:52:50.562 INFO:tasks.workunit.client.1.smithi078.stdout:6/412: creat d2/d1a/d41/f7e x:0 0 0 2024-04-26T04:52:50.564 INFO:tasks.workunit.client.1.smithi078.stdout:0/424: dread - d2/d9/d31/d4a/f64 zero size 2024-04-26T04:52:50.568 INFO:tasks.workunit.client.1.smithi078.stdout:4/372: mkdir d0/d25/d20/d31/d33/d4c/d8e 0 2024-04-26T04:52:50.569 INFO:tasks.workunit.client.0.smithi077.stdout:3/37: symlink d0/d9/lc 0 2024-04-26T04:52:50.570 INFO:tasks.workunit.client.1.smithi078.stdout:2/328: mknod d8/d11/d14/d5b/c73 0 2024-04-26T04:52:50.571 INFO:tasks.workunit.client.0.smithi077.stdout:1/56: unlink d9/db/fc 0 2024-04-26T04:52:50.571 INFO:tasks.workunit.client.0.smithi077.stdout:1/57: write f7 [612828,16160] 0 2024-04-26T04:52:50.572 INFO:tasks.workunit.client.0.smithi077.stdout:1/58: stat d9/db/dd/lf 0 2024-04-26T04:52:50.573 INFO:tasks.workunit.client.0.smithi077.stdout:1/59: fsync f6 0 2024-04-26T04:52:50.574 INFO:tasks.workunit.client.0.smithi077.stdout:1/60: fdatasync d9/db/fe 0 2024-04-26T04:52:50.574 INFO:tasks.workunit.client.1.smithi078.stdout:7/335: mkdir d9/da/d33/d4b/d4c/d6c 0 2024-04-26T04:52:50.574 INFO:tasks.workunit.client.1.smithi078.stdout:6/413: dread d2/de/f12 [0,4194304] 0 2024-04-26T04:52:50.575 INFO:tasks.workunit.client.0.smithi077.stdout:0/58: mknod d0/db/d11/c1c 0 2024-04-26T04:52:50.575 INFO:tasks.workunit.client.1.smithi078.stdout:7/336: chown d9/da/d11/d13/d1a/f1c 1 1 2024-04-26T04:52:50.575 INFO:tasks.workunit.client.0.smithi077.stdout:0/59: write d0/f9 [664266,96251] 0 2024-04-26T04:52:50.576 INFO:tasks.workunit.client.1.smithi078.stdout:3/332: creat d16/d26/d2e/d74/f76 x:0 0 0 2024-04-26T04:52:50.576 INFO:tasks.workunit.client.1.smithi078.stdout:7/337: chown d9/da/d11/f2f 18 1 2024-04-26T04:52:50.576 INFO:tasks.workunit.client.1.smithi078.stdout:3/333: dread - d16/d26/d2b/d31/f52 zero size 2024-04-26T04:52:50.577 INFO:tasks.workunit.client.0.smithi077.stdout:3/38: mkdir d0/d7/dd 0 2024-04-26T04:52:50.578 INFO:tasks.workunit.client.1.smithi078.stdout:3/334: write d16/d26/d2e/f4d [151679,73036] 0 2024-04-26T04:52:50.578 INFO:tasks.workunit.client.0.smithi077.stdout:5/39: rmdir d1/d8 39 2024-04-26T04:52:50.582 INFO:tasks.workunit.client.1.smithi078.stdout:7/338: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/f47 [4194304,4194304] 0 2024-04-26T04:52:50.583 INFO:tasks.workunit.client.1.smithi078.stdout:7/339: chown d9/da/d33/l38 89620 1 2024-04-26T04:52:50.589 INFO:tasks.workunit.client.0.smithi077.stdout:1/61: creat d9/db/dd/f10 x:0 0 0 2024-04-26T04:52:50.589 INFO:tasks.workunit.client.0.smithi077.stdout:1/62: readlink l4 0 2024-04-26T04:52:50.589 INFO:tasks.workunit.client.1.smithi078.stdout:1/395: sync 2024-04-26T04:52:50.590 INFO:tasks.workunit.client.0.smithi077.stdout:3/39: truncate d0/f3 4779175 0 2024-04-26T04:52:50.592 INFO:tasks.workunit.client.1.smithi078.stdout:4/373: rename d0/de/d22/d26/f64 to d0/de/d22/d27/d7b/f8f 0 2024-04-26T04:52:50.594 INFO:tasks.workunit.client.1.smithi078.stdout:4/374: write d0/de/d22/d27/d56/d5a/f7d [819154,64055] 0 2024-04-26T04:52:50.598 INFO:tasks.workunit.client.1.smithi078.stdout:6/414: rmdir d2 39 2024-04-26T04:52:50.598 INFO:tasks.workunit.client.1.smithi078.stdout:7/340: dread d9/da/d20/f21 [0,4194304] 0 2024-04-26T04:52:50.599 INFO:tasks.workunit.client.0.smithi077.stdout:1/63: creat d9/db/dd/f11 x:0 0 0 2024-04-26T04:52:50.599 INFO:tasks.workunit.client.0.smithi077.stdout:1/64: chown d9/db/dd 52031830 1 2024-04-26T04:52:50.600 INFO:tasks.workunit.client.0.smithi077.stdout:1/65: truncate f7 745792 0 2024-04-26T04:52:50.603 INFO:tasks.workunit.client.1.smithi078.stdout:9/316: getdents dc/df/d4b 0 2024-04-26T04:52:50.604 INFO:tasks.workunit.client.0.smithi077.stdout:3/40: truncate d0/d7/f8 938348 0 2024-04-26T04:52:50.604 INFO:tasks.workunit.client.0.smithi077.stdout:5/40: mknod d1/d8/cd 0 2024-04-26T04:52:50.606 INFO:tasks.workunit.client.0.smithi077.stdout:5/41: chown d1/d8/fc 177526 1 2024-04-26T04:52:50.606 INFO:tasks.workunit.client.1.smithi078.stdout:3/335: mkdir d16/d26/d2e/d5f/d49/d77 0 2024-04-26T04:52:50.607 INFO:tasks.workunit.client.1.smithi078.stdout:2/329: rmdir d8 39 2024-04-26T04:52:50.607 INFO:tasks.workunit.client.0.smithi077.stdout:5/42: fsync d1/f5 0 2024-04-26T04:52:50.608 INFO:tasks.workunit.client.0.smithi077.stdout:1/66: dwrite d9/db/fe [0,4194304] 0 2024-04-26T04:52:50.608 INFO:tasks.workunit.client.0.smithi077.stdout:4/45: getdents d4/de 0 2024-04-26T04:52:50.611 INFO:tasks.workunit.client.0.smithi077.stdout:4/46: write f2 [4549810,9611] 0 2024-04-26T04:52:50.612 INFO:tasks.workunit.client.1.smithi078.stdout:3/336: dwrite d16/d26/d2e/f6d [0,4194304] 0 2024-04-26T04:52:50.613 INFO:tasks.workunit.client.0.smithi077.stdout:4/47: write d4/f8 [1015709,106307] 0 2024-04-26T04:52:50.613 INFO:tasks.workunit.client.1.smithi078.stdout:3/337: write d16/d26/d2b/f72 [201047,45751] 0 2024-04-26T04:52:50.615 INFO:tasks.workunit.client.0.smithi077.stdout:3/41: dwrite d0/fb [0,4194304] 0 2024-04-26T04:52:50.617 INFO:tasks.workunit.client.1.smithi078.stdout:0/425: symlink d2/l8b 0 2024-04-26T04:52:50.618 INFO:tasks.workunit.client.1.smithi078.stdout:0/426: truncate d2/d1e/f61 428523 0 2024-04-26T04:52:50.619 INFO:tasks.workunit.client.1.smithi078.stdout:0/427: readlink d2/d9/d18/l41 0 2024-04-26T04:52:50.620 INFO:tasks.workunit.client.0.smithi077.stdout:4/48: dread f2 [0,4194304] 0 2024-04-26T04:52:50.620 INFO:tasks.workunit.client.1.smithi078.stdout:0/428: fdatasync d2/d9/d60/d6a/f6d 0 2024-04-26T04:52:50.621 INFO:tasks.workunit.client.0.smithi077.stdout:4/49: chown d4/cd 347 1 2024-04-26T04:52:50.623 INFO:tasks.workunit.client.1.smithi078.stdout:0/429: read d2/d9/d18/f45 [3068643,18369] 0 2024-04-26T04:52:50.624 INFO:tasks.workunit.client.0.smithi077.stdout:1/67: mknod d9/db/dd/c12 0 2024-04-26T04:52:50.625 INFO:tasks.workunit.client.1.smithi078.stdout:1/396: dread d7/d39/d17/f3c [0,4194304] 0 2024-04-26T04:52:50.626 INFO:tasks.workunit.client.1.smithi078.stdout:1/397: dread - d7/d39/d4c/f4e zero size 2024-04-26T04:52:50.633 INFO:tasks.workunit.client.1.smithi078.stdout:4/375: creat d0/de/d22/d26/d2e/f90 x:0 0 0 2024-04-26T04:52:50.633 INFO:tasks.workunit.client.1.smithi078.stdout:7/341: mkdir d9/da/d11/d13/d1a/d52/d5a/d28/d6d 0 2024-04-26T04:52:50.634 INFO:tasks.workunit.client.1.smithi078.stdout:9/317: creat dc/df/d4b/d3e/f5d x:0 0 0 2024-04-26T04:52:50.634 INFO:tasks.workunit.client.0.smithi077.stdout:4/50: symlink d4/l12 0 2024-04-26T04:52:50.634 INFO:tasks.workunit.client.0.smithi077.stdout:4/51: write d4/f8 [4518753,42001] 0 2024-04-26T04:52:50.635 INFO:tasks.workunit.client.0.smithi077.stdout:1/68: creat d9/db/f13 x:0 0 0 2024-04-26T04:52:50.640 INFO:tasks.workunit.client.1.smithi078.stdout:3/338: mkdir d16/d26/d2b/d31/d78 0 2024-04-26T04:52:50.640 INFO:tasks.workunit.client.1.smithi078.stdout:3/339: stat d16/l64 0 2024-04-26T04:52:50.642 INFO:tasks.workunit.client.0.smithi077.stdout:3/42: link d0/f1 d0/d7/fe 0 2024-04-26T04:52:50.642 INFO:tasks.workunit.client.0.smithi077.stdout:4/52: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:50.644 INFO:tasks.workunit.client.1.smithi078.stdout:5/269: getdents d7/d21/d2f 0 2024-04-26T04:52:50.645 INFO:tasks.workunit.client.1.smithi078.stdout:8/341: sync 2024-04-26T04:52:50.651 INFO:tasks.workunit.client.0.smithi077.stdout:4/53: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:50.653 INFO:tasks.workunit.client.1.smithi078.stdout:7/342: dread f4 [8388608,4194304] 0 2024-04-26T04:52:50.653 INFO:tasks.workunit.client.1.smithi078.stdout:0/430: creat d2/d39/f8c x:0 0 0 2024-04-26T04:52:50.654 INFO:tasks.workunit.client.1.smithi078.stdout:7/343: chown d9/da/d11/d13/d1a/f29 151342074 1 2024-04-26T04:52:50.654 INFO:tasks.workunit.client.1.smithi078.stdout:0/431: chown d2/d9/d31/f50 11 1 2024-04-26T04:52:50.655 INFO:tasks.workunit.client.1.smithi078.stdout:0/432: fdatasync d2/f14 0 2024-04-26T04:52:50.655 INFO:tasks.workunit.client.1.smithi078.stdout:7/344: chown d9/da/d33/d4b 7782073 1 2024-04-26T04:52:50.655 INFO:tasks.workunit.client.1.smithi078.stdout:1/398: rmdir d7/d39/db/d49 39 2024-04-26T04:52:50.656 INFO:tasks.workunit.client.1.smithi078.stdout:7/345: write d9/f6a [468607,84452] 0 2024-04-26T04:52:50.656 INFO:tasks.workunit.client.1.smithi078.stdout:7/346: chown d9/d57 139144652 1 2024-04-26T04:52:50.658 INFO:tasks.workunit.client.1.smithi078.stdout:4/376: rmdir d0 39 2024-04-26T04:52:50.661 INFO:tasks.workunit.client.1.smithi078.stdout:6/415: rmdir d2/d1a/d5/d3a/d42 39 2024-04-26T04:52:50.662 INFO:tasks.workunit.client.1.smithi078.stdout:6/416: truncate d2/d1a/f43 621224 0 2024-04-26T04:52:50.662 INFO:tasks.workunit.client.0.smithi077.stdout:5/43: getdents d1/d8 0 2024-04-26T04:52:50.663 INFO:tasks.workunit.client.1.smithi078.stdout:9/318: rmdir dc/df/d4b/d22/d2d 39 2024-04-26T04:52:50.665 INFO:tasks.workunit.client.0.smithi077.stdout:4/54: rename d4/l9 to d4/de/l13 0 2024-04-26T04:52:50.667 INFO:tasks.workunit.client.1.smithi078.stdout:5/270: rename d7/d21/c55 to d7/d18/d32/c5e 0 2024-04-26T04:52:50.667 INFO:tasks.workunit.client.0.smithi077.stdout:3/43: mknod d0/d7/dd/cf 0 2024-04-26T04:52:50.669 INFO:tasks.workunit.client.0.smithi077.stdout:4/55: creat d4/f14 x:0 0 0 2024-04-26T04:52:50.670 INFO:tasks.workunit.client.1.smithi078.stdout:8/342: dread d0/d19/d52/d35/f4e [0,4194304] 0 2024-04-26T04:52:50.670 INFO:tasks.workunit.client.1.smithi078.stdout:1/399: dread - d7/d39/db/d49/f84 zero size 2024-04-26T04:52:50.672 INFO:tasks.workunit.client.0.smithi077.stdout:3/44: symlink d0/d7/l10 0 2024-04-26T04:52:50.675 INFO:tasks.workunit.client.1.smithi078.stdout:4/377: dread - d0/d25/d20/d31/f6b zero size 2024-04-26T04:52:50.675 INFO:tasks.workunit.client.0.smithi077.stdout:4/56: mkdir d4/de/d15 0 2024-04-26T04:52:50.675 INFO:tasks.workunit.client.0.smithi077.stdout:4/57: dread - d4/f14 zero size 2024-04-26T04:52:50.676 INFO:tasks.workunit.client.1.smithi078.stdout:6/417: mknod d2/d1a/d38/c7f 0 2024-04-26T04:52:50.676 INFO:tasks.workunit.client.0.smithi077.stdout:4/58: fdatasync d4/f8 0 2024-04-26T04:52:50.679 INFO:tasks.workunit.client.0.smithi077.stdout:5/44: creat d1/fe x:0 0 0 2024-04-26T04:52:50.681 INFO:tasks.workunit.client.1.smithi078.stdout:7/347: rename d9/da/d33/f3e to d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e 0 2024-04-26T04:52:50.682 INFO:tasks.workunit.client.0.smithi077.stdout:4/59: readlink d4/l6 0 2024-04-26T04:52:50.682 INFO:tasks.workunit.client.0.smithi077.stdout:1/69: dwrite f2 [4194304,4194304] 0 2024-04-26T04:52:50.683 INFO:tasks.workunit.client.1.smithi078.stdout:7/348: write d9/da/d11/d13/d1a/f4e [620627,128887] 0 2024-04-26T04:52:50.683 INFO:tasks.workunit.client.0.smithi077.stdout:4/60: dread - d4/f14 zero size 2024-04-26T04:52:50.684 INFO:tasks.workunit.client.0.smithi077.stdout:1/70: truncate d9/db/dd/f11 1001062 0 2024-04-26T04:52:50.686 INFO:tasks.workunit.client.0.smithi077.stdout:1/71: write f2 [2448931,37587] 0 2024-04-26T04:52:50.687 INFO:tasks.workunit.client.0.smithi077.stdout:5/45: creat d1/d8/ff x:0 0 0 2024-04-26T04:52:50.688 INFO:tasks.workunit.client.1.smithi078.stdout:1/400: creat d7/d39/db/d37/f88 x:0 0 0 2024-04-26T04:52:50.689 INFO:tasks.workunit.client.1.smithi078.stdout:0/433: link d2/d1e/f2a d2/d9/d31/f8d 0 2024-04-26T04:52:50.690 INFO:tasks.workunit.client.1.smithi078.stdout:0/434: dread - d2/d9/d31/f66 zero size 2024-04-26T04:52:50.690 INFO:tasks.workunit.client.1.smithi078.stdout:0/435: readlink d2/d9/d31/d4a/l5b 0 2024-04-26T04:52:50.691 INFO:tasks.workunit.client.0.smithi077.stdout:3/45: link d0/f1 d0/d7/f11 0 2024-04-26T04:52:50.692 INFO:tasks.workunit.client.1.smithi078.stdout:4/378: chown d0/de/d22/d71/f79 13 1 2024-04-26T04:52:50.694 INFO:tasks.workunit.client.0.smithi077.stdout:5/46: readlink d1/la 0 2024-04-26T04:52:50.695 INFO:tasks.workunit.client.1.smithi078.stdout:9/319: mknod dc/df/d4b/d22/c5e 0 2024-04-26T04:52:50.697 INFO:tasks.workunit.client.1.smithi078.stdout:2/330: creat d8/d11/f74 x:0 0 0 2024-04-26T04:52:50.697 INFO:tasks.workunit.client.1.smithi078.stdout:6/418: dwrite d2/d1a/d5/f6b [0,4194304] 0 2024-04-26T04:52:50.698 INFO:tasks.workunit.client.1.smithi078.stdout:6/419: readlink d2/de/l6e 0 2024-04-26T04:52:50.700 INFO:tasks.workunit.client.0.smithi077.stdout:3/46: mknod d0/c12 0 2024-04-26T04:52:50.702 INFO:tasks.workunit.client.1.smithi078.stdout:7/349: creat d9/da/d20/f6f x:0 0 0 2024-04-26T04:52:50.702 INFO:tasks.workunit.client.1.smithi078.stdout:8/343: write d0/f23 [125986,34084] 0 2024-04-26T04:52:50.702 INFO:tasks.workunit.client.1.smithi078.stdout:7/350: stat d9/fe 0 2024-04-26T04:52:50.702 INFO:tasks.workunit.client.0.smithi077.stdout:5/47: creat d1/d8/f10 x:0 0 0 2024-04-26T04:52:50.704 INFO:tasks.workunit.client.1.smithi078.stdout:7/351: chown d9/da/d11/d13/d1a/d2d/f4f 4445 1 2024-04-26T04:52:50.708 INFO:tasks.workunit.client.1.smithi078.stdout:0/436: chown d2/f37 888 1 2024-04-26T04:52:50.708 INFO:tasks.workunit.client.1.smithi078.stdout:8/344: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:52:50.709 INFO:tasks.workunit.client.0.smithi077.stdout:3/47: mknod d0/c13 0 2024-04-26T04:52:50.709 INFO:tasks.workunit.client.0.smithi077.stdout:3/48: truncate d0/fb 4468861 0 2024-04-26T04:52:50.710 INFO:tasks.workunit.client.0.smithi077.stdout:3/49: read d0/f6 [1178568,38365] 0 2024-04-26T04:52:50.710 INFO:tasks.workunit.client.0.smithi077.stdout:1/72: link d9/ca d9/c14 0 2024-04-26T04:52:50.711 INFO:tasks.workunit.client.1.smithi078.stdout:4/379: fsync d0/de/f1a 0 2024-04-26T04:52:50.712 INFO:tasks.workunit.client.0.smithi077.stdout:4/61: getdents d4/de 0 2024-04-26T04:52:50.713 INFO:tasks.workunit.client.0.smithi077.stdout:4/62: write d4/f8 [352890,119066] 0 2024-04-26T04:52:50.714 INFO:tasks.workunit.client.0.smithi077.stdout:4/63: write d4/f14 [382884,67724] 0 2024-04-26T04:52:50.715 INFO:tasks.workunit.client.1.smithi078.stdout:1/401: fdatasync d7/d39/dd/d20/d53/f76 0 2024-04-26T04:52:50.716 INFO:tasks.workunit.client.1.smithi078.stdout:1/402: stat d7/d39/db/d37/c43 0 2024-04-26T04:52:50.716 INFO:tasks.workunit.client.0.smithi077.stdout:5/48: truncate d1/f5 500919 0 2024-04-26T04:52:50.717 INFO:tasks.workunit.client.1.smithi078.stdout:7/352: rmdir d9/da/d11/d13/d1a/d52/d5a/d28 39 2024-04-26T04:52:50.720 INFO:tasks.workunit.client.1.smithi078.stdout:2/331: dread d8/d11/d18/d5d/d61/f1b [0,4194304] 0 2024-04-26T04:52:50.720 INFO:tasks.workunit.client.0.smithi077.stdout:3/50: creat d0/d9/f14 x:0 0 0 2024-04-26T04:52:50.721 INFO:tasks.workunit.client.0.smithi077.stdout:1/73: creat d9/db/dd/f15 x:0 0 0 2024-04-26T04:52:50.721 INFO:tasks.workunit.client.0.smithi077.stdout:1/74: readlink l4 0 2024-04-26T04:52:50.722 INFO:tasks.workunit.client.0.smithi077.stdout:1/75: chown c8 0 1 2024-04-26T04:52:50.722 INFO:tasks.workunit.client.1.smithi078.stdout:9/320: mkdir dc/df/d4b/d22/d2a/d57/d5f 0 2024-04-26T04:52:50.725 INFO:tasks.workunit.client.0.smithi077.stdout:5/49: rename d1/d8/cd to d1/d8/c11 0 2024-04-26T04:52:50.726 INFO:tasks.workunit.client.1.smithi078.stdout:6/420: rename d2/d1a/d38/c73 to d2/d49/c80 0 2024-04-26T04:52:50.728 INFO:tasks.workunit.client.1.smithi078.stdout:4/380: truncate d0/de/f34 7165137 0 2024-04-26T04:52:50.730 INFO:tasks.workunit.client.1.smithi078.stdout:3/340: sync 2024-04-26T04:52:50.761 INFO:tasks.workunit.client.0.smithi077.stdout:3/51: mkdir d0/d7/d15 0 2024-04-26T04:52:50.763 INFO:tasks.workunit.client.0.smithi077.stdout:4/64: creat d4/de/d15/f16 x:0 0 0 2024-04-26T04:52:50.764 INFO:tasks.workunit.client.1.smithi078.stdout:7/353: creat d9/da/d33/f70 x:0 0 0 2024-04-26T04:52:50.764 INFO:tasks.workunit.client.1.smithi078.stdout:6/421: dread d2/d49/f57 [0,4194304] 0 2024-04-26T04:52:50.765 INFO:tasks.workunit.client.0.smithi077.stdout:1/76: mknod d9/db/dd/c16 0 2024-04-26T04:52:50.767 INFO:tasks.workunit.client.0.smithi077.stdout:3/52: symlink d0/d7/dd/l16 0 2024-04-26T04:52:50.770 INFO:tasks.workunit.client.0.smithi077.stdout:4/65: mkdir d4/d17 0 2024-04-26T04:52:50.770 INFO:tasks.workunit.client.0.smithi077.stdout:4/66: dread - d4/de/d15/f16 zero size 2024-04-26T04:52:50.771 INFO:tasks.workunit.client.0.smithi077.stdout:1/77: chown d9/ca 8 1 2024-04-26T04:52:50.772 INFO:tasks.workunit.client.1.smithi078.stdout:5/271: sync 2024-04-26T04:52:50.773 INFO:tasks.workunit.client.0.smithi077.stdout:5/50: mkdir d1/d12 0 2024-04-26T04:52:50.773 INFO:tasks.workunit.client.0.smithi077.stdout:5/51: dread - d1/d8/fc zero size 2024-04-26T04:52:50.776 INFO:tasks.workunit.client.0.smithi077.stdout:3/53: write d0/f6 [5003556,11420] 0 2024-04-26T04:52:50.776 INFO:tasks.workunit.client.1.smithi078.stdout:1/403: mknod d7/d39/db/d37/d58/d5c/d68/c89 0 2024-04-26T04:52:50.778 INFO:tasks.workunit.client.1.smithi078.stdout:7/354: creat d9/f71 x:0 0 0 2024-04-26T04:52:50.779 INFO:tasks.workunit.client.0.smithi077.stdout:1/78: rmdir d9 39 2024-04-26T04:52:50.781 INFO:tasks.workunit.client.0.smithi077.stdout:5/52: dwrite d1/f4 [0,4194304] 0 2024-04-26T04:52:50.790 INFO:tasks.workunit.client.1.smithi078.stdout:2/332: creat d8/d11/d18/d5d/d61/d1a/d34/d70/f75 x:0 0 0 2024-04-26T04:52:50.791 INFO:tasks.workunit.client.1.smithi078.stdout:2/333: write d8/fa [2651865,6662] 0 2024-04-26T04:52:50.792 INFO:tasks.workunit.client.0.smithi077.stdout:4/67: unlink d4/la 0 2024-04-26T04:52:50.793 INFO:tasks.workunit.client.0.smithi077.stdout:3/54: dread d0/d7/fe [0,4194304] 0 2024-04-26T04:52:50.796 INFO:tasks.workunit.client.0.smithi077.stdout:3/55: write d0/f6 [4796738,16180] 0 2024-04-26T04:52:50.796 INFO:tasks.workunit.client.1.smithi078.stdout:2/334: dwrite d8/d11/d18/d5d/d61/f32 [0,4194304] 0 2024-04-26T04:52:50.798 INFO:tasks.workunit.client.1.smithi078.stdout:5/272: creat d7/da/f5f x:0 0 0 2024-04-26T04:52:50.800 INFO:tasks.workunit.client.1.smithi078.stdout:6/422: write d2/de/f12 [259163,66429] 0 2024-04-26T04:52:50.800 INFO:tasks.workunit.client.1.smithi078.stdout:5/273: write d7/d18/d32/f3b [2005820,79438] 0 2024-04-26T04:52:50.802 INFO:tasks.workunit.client.1.smithi078.stdout:3/341: fsync f9 0 2024-04-26T04:52:50.803 INFO:tasks.workunit.client.1.smithi078.stdout:0/437: truncate d2/f3 2083713 0 2024-04-26T04:52:50.805 INFO:tasks.workunit.client.0.smithi077.stdout:1/79: rename l4 to d9/db/dd/l17 0 2024-04-26T04:52:50.806 INFO:tasks.workunit.client.1.smithi078.stdout:7/355: symlink d9/l72 0 2024-04-26T04:52:50.807 INFO:tasks.workunit.client.1.smithi078.stdout:7/356: fdatasync d9/da/d11/f45 0 2024-04-26T04:52:50.808 INFO:tasks.workunit.client.0.smithi077.stdout:5/53: creat d1/d12/f13 x:0 0 0 2024-04-26T04:52:50.808 INFO:tasks.workunit.client.1.smithi078.stdout:6/423: dwrite d2/d1a/d5/da/f2c [0,4194304] 0 2024-04-26T04:52:50.808 INFO:tasks.workunit.client.1.smithi078.stdout:7/357: dread - d9/da/d11/d13/d50/f59 zero size 2024-04-26T04:52:50.809 INFO:tasks.workunit.client.0.smithi077.stdout:5/54: chown d1/d8/ff 21 1 2024-04-26T04:52:50.809 INFO:tasks.workunit.client.0.smithi077.stdout:1/80: dread f2 [0,4194304] 0 2024-04-26T04:52:50.810 INFO:tasks.workunit.client.1.smithi078.stdout:7/358: write d9/da/d11/d13/d50/f59 [612385,34426] 0 2024-04-26T04:52:50.811 INFO:tasks.workunit.client.0.smithi077.stdout:4/68: write f2 [4026900,128192] 0 2024-04-26T04:52:50.812 INFO:tasks.workunit.client.1.smithi078.stdout:8/345: getdents d0/d19/d52/d35/d3f 0 2024-04-26T04:52:50.813 INFO:tasks.workunit.client.1.smithi078.stdout:7/359: write d9/da/d11/d13/d1a/d52/d5a/d41/f47 [3407995,114775] 0 2024-04-26T04:52:50.813 INFO:tasks.workunit.client.1.smithi078.stdout:6/424: dread d2/d1a/d5/d2a/f3d [0,4194304] 0 2024-04-26T04:52:50.819 INFO:tasks.workunit.client.1.smithi078.stdout:6/425: dwrite d2/f71 [0,4194304] 0 2024-04-26T04:52:50.820 INFO:tasks.workunit.client.0.smithi077.stdout:5/55: rmdir d1/d8 39 2024-04-26T04:52:50.822 INFO:tasks.workunit.client.1.smithi078.stdout:9/321: creat dc/f60 x:0 0 0 2024-04-26T04:52:50.823 INFO:tasks.workunit.client.1.smithi078.stdout:9/322: chown dc/df/d4b/d22/c5b 819518929 1 2024-04-26T04:52:50.824 INFO:tasks.workunit.client.1.smithi078.stdout:2/335: creat d8/d11/d18/d5d/d61/f76 x:0 0 0 2024-04-26T04:52:50.826 INFO:tasks.workunit.client.1.smithi078.stdout:4/381: creat d0/de/f91 x:0 0 0 2024-04-26T04:52:50.827 INFO:tasks.workunit.client.1.smithi078.stdout:6/426: dwrite d2/de/d27/f64 [0,4194304] 0 2024-04-26T04:52:50.831 INFO:tasks.workunit.client.0.smithi077.stdout:3/56: dwrite d0/d7/fe [0,4194304] 0 2024-04-26T04:52:50.833 INFO:tasks.workunit.client.0.smithi077.stdout:4/69: mkdir d4/d17/d18 0 2024-04-26T04:52:50.834 INFO:tasks.workunit.client.1.smithi078.stdout:4/382: dwrite d0/de/d22/d27/d56/f5c [0,4194304] 0 2024-04-26T04:52:50.834 INFO:tasks.workunit.client.0.smithi077.stdout:4/70: chown l3 7 1 2024-04-26T04:52:50.838 INFO:tasks.workunit.client.1.smithi078.stdout:0/438: dread d2/d9/f3e [0,4194304] 0 2024-04-26T04:52:50.839 INFO:tasks.workunit.client.0.smithi077.stdout:4/71: dread d4/f8 [0,4194304] 0 2024-04-26T04:52:50.840 INFO:tasks.workunit.client.0.smithi077.stdout:3/57: mknod d0/d7/d15/c17 0 2024-04-26T04:52:50.844 INFO:tasks.workunit.client.0.smithi077.stdout:4/72: mkdir d4/de/d19 0 2024-04-26T04:52:50.844 INFO:tasks.workunit.client.0.smithi077.stdout:4/73: chown d4/de/d15/f16 7 1 2024-04-26T04:52:50.850 INFO:tasks.workunit.client.0.smithi077.stdout:4/74: rename d4/l11 to d4/d17/d18/l1a 0 2024-04-26T04:52:50.850 INFO:tasks.workunit.client.0.smithi077.stdout:1/81: truncate f2 1686782 0 2024-04-26T04:52:50.851 INFO:tasks.workunit.client.0.smithi077.stdout:4/75: write d4/de/d15/f16 [1040872,50785] 0 2024-04-26T04:52:50.851 INFO:tasks.workunit.client.1.smithi078.stdout:7/360: dread d9/da/d20/f21 [0,4194304] 0 2024-04-26T04:52:50.864 INFO:tasks.workunit.client.1.smithi078.stdout:2/336: stat d8/d11/l4f 0 2024-04-26T04:52:50.867 INFO:tasks.workunit.client.0.smithi077.stdout:1/82: mknod d9/c18 0 2024-04-26T04:52:50.868 INFO:tasks.workunit.client.0.smithi077.stdout:4/76: write d4/f8 [3067050,39800] 0 2024-04-26T04:52:50.872 INFO:tasks.workunit.client.1.smithi078.stdout:1/404: creat d7/f8a x:0 0 0 2024-04-26T04:52:50.872 INFO:tasks.workunit.client.1.smithi078.stdout:1/405: chown d7/d39/d4c/f4e 1032702577 1 2024-04-26T04:52:50.874 INFO:tasks.workunit.client.0.smithi077.stdout:1/83: dwrite d9/db/dd/f10 [0,4194304] 0 2024-04-26T04:52:50.877 INFO:tasks.workunit.client.0.smithi077.stdout:4/77: link f2 d4/de/d19/f1b 0 2024-04-26T04:52:50.879 INFO:tasks.workunit.client.0.smithi077.stdout:4/78: dread f2 [4194304,4194304] 0 2024-04-26T04:52:50.883 INFO:tasks.workunit.client.1.smithi078.stdout:8/346: mkdir d0/d19/d25/d36/d58/d5b/d61/d67 0 2024-04-26T04:52:50.883 INFO:tasks.workunit.client.0.smithi077.stdout:3/58: truncate d0/f1 335845 0 2024-04-26T04:52:50.884 INFO:tasks.workunit.client.1.smithi078.stdout:8/347: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:50.885 INFO:tasks.workunit.client.1.smithi078.stdout:7/361: fsync d9/da/d11/d13/d1a/f29 0 2024-04-26T04:52:50.889 INFO:tasks.workunit.client.0.smithi077.stdout:4/79: rename d4/cd to d4/d17/d18/c1c 0 2024-04-26T04:52:50.890 INFO:tasks.workunit.client.1.smithi078.stdout:3/342: sync 2024-04-26T04:52:50.890 INFO:tasks.workunit.client.0.smithi077.stdout:4/80: fsync d4/de/d15/f16 0 2024-04-26T04:52:50.891 INFO:tasks.workunit.client.1.smithi078.stdout:5/274: write d7/d29/f3a [1175037,72704] 0 2024-04-26T04:52:50.891 INFO:tasks.workunit.client.1.smithi078.stdout:8/348: dwrite d0/d19/d25/d32/f4b [8388608,4194304] 0 2024-04-26T04:52:50.892 INFO:tasks.workunit.client.1.smithi078.stdout:7/362: dwrite d9/da/d11/d13/d1a/f1c [0,4194304] 0 2024-04-26T04:52:50.894 INFO:tasks.workunit.client.1.smithi078.stdout:6/427: mknod d2/de/d27/d4c/d5f/c81 0 2024-04-26T04:52:50.903 INFO:tasks.workunit.client.1.smithi078.stdout:1/406: symlink d7/d39/db/d37/d58/d5c/l8b 0 2024-04-26T04:52:50.908 INFO:tasks.workunit.client.0.smithi077.stdout:1/84: dwrite f3 [0,4194304] 0 2024-04-26T04:52:50.908 INFO:tasks.workunit.client.1.smithi078.stdout:1/407: dwrite d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:50.913 INFO:tasks.workunit.client.1.smithi078.stdout:2/337: mknod d8/d11/d18/d5d/d61/d1a/c77 0 2024-04-26T04:52:50.915 INFO:tasks.workunit.client.1.smithi078.stdout:9/323: rmdir dc/df/d4b/d22/d2a/d57/d5f 0 2024-04-26T04:52:50.918 INFO:tasks.workunit.client.0.smithi077.stdout:3/59: dwrite d0/f3 [4194304,4194304] 0 2024-04-26T04:52:50.918 INFO:tasks.workunit.client.1.smithi078.stdout:5/275: creat d7/d18/d32/f60 x:0 0 0 2024-04-26T04:52:50.920 INFO:tasks.workunit.client.0.smithi077.stdout:3/60: read d0/f3 [2331940,31094] 0 2024-04-26T04:52:50.921 INFO:tasks.workunit.client.0.smithi077.stdout:1/85: dwrite d9/db/dd/f11 [0,4194304] 0 2024-04-26T04:52:50.921 INFO:tasks.workunit.client.0.smithi077.stdout:3/61: readlink d0/d7/l10 0 2024-04-26T04:52:50.924 INFO:tasks.workunit.client.0.smithi077.stdout:4/81: truncate d4/f8 2649191 0 2024-04-26T04:52:50.927 INFO:tasks.workunit.client.1.smithi078.stdout:6/428: creat d2/de/f82 x:0 0 0 2024-04-26T04:52:50.929 INFO:tasks.workunit.client.1.smithi078.stdout:4/383: link d0/de/d22/d27/d56/d5a/c8b d0/d25/d20/d31/d33/d4c/d62/c92 0 2024-04-26T04:52:50.930 INFO:tasks.workunit.client.0.smithi077.stdout:1/86: rmdir d9/db/dd 39 2024-04-26T04:52:50.932 INFO:tasks.workunit.client.0.smithi077.stdout:3/62: symlink d0/d7/d15/l18 0 2024-04-26T04:52:50.933 INFO:tasks.workunit.client.0.smithi077.stdout:3/63: stat d0/f3 0 2024-04-26T04:52:50.934 INFO:tasks.workunit.client.1.smithi078.stdout:1/408: unlink d7/d39/db/d37/d58/d5c/l8b 0 2024-04-26T04:52:50.935 INFO:tasks.workunit.client.1.smithi078.stdout:9/324: chown dc/df/d17/c1b 251088 1 2024-04-26T04:52:50.937 INFO:tasks.workunit.client.0.smithi077.stdout:1/87: truncate d9/db/dd/f15 811564 0 2024-04-26T04:52:50.937 INFO:tasks.workunit.client.1.smithi078.stdout:9/325: write dc/df/d27/f53 [1305,110020] 0 2024-04-26T04:52:50.938 INFO:tasks.workunit.client.1.smithi078.stdout:8/349: mkdir d0/d19/d52/d5c/d68 0 2024-04-26T04:52:50.938 INFO:tasks.workunit.client.0.smithi077.stdout:4/82: truncate d4/de/d19/f1b 625022 0 2024-04-26T04:52:50.940 INFO:tasks.workunit.client.1.smithi078.stdout:7/363: creat d9/da/d11/d13/d1a/d52/d5a/d28/f73 x:0 0 0 2024-04-26T04:52:50.941 INFO:tasks.workunit.client.1.smithi078.stdout:2/338: dwrite d8/d11/f48 [0,4194304] 0 2024-04-26T04:52:50.942 INFO:tasks.workunit.client.1.smithi078.stdout:6/429: symlink d2/d49/l83 0 2024-04-26T04:52:50.942 INFO:tasks.workunit.client.1.smithi078.stdout:2/339: chown l2 2023490104 1 2024-04-26T04:52:50.945 INFO:tasks.workunit.client.1.smithi078.stdout:4/384: creat d0/f93 x:0 0 0 2024-04-26T04:52:50.946 INFO:tasks.workunit.client.1.smithi078.stdout:4/385: chown d0/d25/d20/d31/d33/d44/d55/f7f 11526 1 2024-04-26T04:52:50.946 INFO:tasks.workunit.client.1.smithi078.stdout:0/439: link d2/f15 d2/d9/d78/d84/f8e 0 2024-04-26T04:52:50.947 INFO:tasks.workunit.client.1.smithi078.stdout:0/440: readlink d2/d1e/l47 0 2024-04-26T04:52:50.952 INFO:tasks.workunit.client.1.smithi078.stdout:1/409: dread d7/d1e/f54 [0,4194304] 0 2024-04-26T04:52:50.953 INFO:tasks.workunit.client.1.smithi078.stdout:1/410: chown d7/d2e/l52 2961696 1 2024-04-26T04:52:50.956 INFO:tasks.workunit.client.1.smithi078.stdout:7/364: symlink d9/da/d11/d13/d50/l74 0 2024-04-26T04:52:50.958 INFO:tasks.workunit.client.1.smithi078.stdout:2/340: mknod d8/d11/d18/d5d/d61/d1a/d34/d70/c78 0 2024-04-26T04:52:50.959 INFO:tasks.workunit.client.1.smithi078.stdout:4/386: symlink d0/d25/d20/d31/d33/d44/l94 0 2024-04-26T04:52:50.959 INFO:tasks.workunit.client.1.smithi078.stdout:2/341: chown d8/d11/l4f 381 1 2024-04-26T04:52:50.960 INFO:tasks.workunit.client.1.smithi078.stdout:9/326: dwrite dc/df/d4b/d22/d2d/f4d [4194304,4194304] 0 2024-04-26T04:52:50.963 INFO:tasks.workunit.client.1.smithi078.stdout:0/441: mkdir d2/d9/d60/d6a/d8f 0 2024-04-26T04:52:50.966 INFO:tasks.workunit.client.1.smithi078.stdout:1/411: mknod d7/d39/d17/c8c 0 2024-04-26T04:52:50.967 INFO:tasks.workunit.client.1.smithi078.stdout:9/327: dwrite dc/df/d17/f48 [0,4194304] 0 2024-04-26T04:52:50.968 INFO:tasks.workunit.client.1.smithi078.stdout:9/328: chown dc/df/d37/c5a 116112 1 2024-04-26T04:52:50.968 INFO:tasks.workunit.client.0.smithi077.stdout:3/64: dwrite d0/d7/f8 [0,4194304] 0 2024-04-26T04:52:50.973 INFO:tasks.workunit.client.1.smithi078.stdout:7/365: unlink d9/da/d11/f12 0 2024-04-26T04:52:50.974 INFO:tasks.workunit.client.1.smithi078.stdout:1/412: dread d7/d1e/f2f [0,4194304] 0 2024-04-26T04:52:50.976 INFO:tasks.workunit.client.1.smithi078.stdout:4/387: write d0/de/f1e [279650,55072] 0 2024-04-26T04:52:50.979 INFO:tasks.workunit.client.1.smithi078.stdout:2/342: symlink d8/d11/d18/d5d/d61/d1a/d34/l79 0 2024-04-26T04:52:50.980 INFO:tasks.workunit.client.0.smithi077.stdout:4/83: write f2 [50590,26282] 0 2024-04-26T04:52:50.980 INFO:tasks.workunit.client.1.smithi078.stdout:2/343: chown d8/d11/d18/d5d/d61 6 1 2024-04-26T04:52:50.981 INFO:tasks.workunit.client.1.smithi078.stdout:1/413: dwrite d7/d39/db/f5a [0,4194304] 0 2024-04-26T04:52:50.981 INFO:tasks.workunit.client.0.smithi077.stdout:3/65: link d0/d7/fe d0/f19 0 2024-04-26T04:52:50.982 INFO:tasks.workunit.client.1.smithi078.stdout:1/414: read d7/f11 [694199,51585] 0 2024-04-26T04:52:50.982 INFO:tasks.workunit.client.0.smithi077.stdout:3/66: chown d0/d7/d15 913765 1 2024-04-26T04:52:50.983 INFO:tasks.workunit.client.1.smithi078.stdout:5/276: getdents d7/d29/d46 0 2024-04-26T04:52:50.983 INFO:tasks.workunit.client.1.smithi078.stdout:1/415: truncate d7/d39/db/d47/f73 449306 0 2024-04-26T04:52:50.984 INFO:tasks.workunit.client.0.smithi077.stdout:3/67: truncate d0/f3 9009699 0 2024-04-26T04:52:50.984 INFO:tasks.workunit.client.1.smithi078.stdout:0/442: creat d2/d9/d31/d73/f90 x:0 0 0 2024-04-26T04:52:50.987 INFO:tasks.workunit.client.1.smithi078.stdout:8/350: link d0/d19/d52/d35/f4e d0/d19/d52/f69 0 2024-04-26T04:52:50.988 INFO:tasks.workunit.client.0.smithi077.stdout:4/84: dwrite d4/de/d15/f16 [0,4194304] 0 2024-04-26T04:52:50.992 INFO:tasks.workunit.client.0.smithi077.stdout:3/68: fdatasync d0/d7/fe 0 2024-04-26T04:52:50.997 INFO:tasks.workunit.client.0.smithi077.stdout:4/85: dwrite f2 [0,4194304] 0 2024-04-26T04:52:50.997 INFO:tasks.workunit.client.1.smithi078.stdout:7/366: rename d9/da/d20/f21 to d9/da/d33/d4b/f75 0 2024-04-26T04:52:50.998 INFO:tasks.workunit.client.1.smithi078.stdout:7/367: write d9/da/d11/d13/d1a/d52/d5a/f53 [158057,53757] 0 2024-04-26T04:52:51.001 INFO:tasks.workunit.client.1.smithi078.stdout:4/388: rmdir d0/de/d22/d71 39 2024-04-26T04:52:51.002 INFO:tasks.workunit.client.0.smithi077.stdout:4/86: dread d4/de/d19/f1b [0,4194304] 0 2024-04-26T04:52:51.005 INFO:tasks.workunit.client.1.smithi078.stdout:7/368: dwrite d9/da/d11/d13/d1a/d52/d5a/f53 [0,4194304] 0 2024-04-26T04:52:51.006 INFO:tasks.workunit.client.0.smithi077.stdout:3/69: fsync d0/f1 0 2024-04-26T04:52:51.008 INFO:tasks.workunit.client.1.smithi078.stdout:4/389: read d0/de/d22/d27/d7b/f8f [6863207,96030] 0 2024-04-26T04:52:51.008 INFO:tasks.workunit.client.1.smithi078.stdout:2/344: creat d8/d53/f7a x:0 0 0 2024-04-26T04:52:51.011 INFO:tasks.workunit.client.1.smithi078.stdout:4/390: dread d0/d1d/d46/f6e [0,4194304] 0 2024-04-26T04:52:51.011 INFO:tasks.workunit.client.1.smithi078.stdout:5/277: chown d7/da/f12 492733139 1 2024-04-26T04:52:51.011 INFO:tasks.workunit.client.0.smithi077.stdout:4/87: write d4/de/d15/f16 [4337887,129480] 0 2024-04-26T04:52:51.012 INFO:tasks.workunit.client.1.smithi078.stdout:4/391: chown d0/d25/d20/d31/c43 723131 1 2024-04-26T04:52:51.012 INFO:tasks.workunit.client.0.smithi077.stdout:3/70: rmdir d0/d7 39 2024-04-26T04:52:51.014 INFO:tasks.workunit.client.1.smithi078.stdout:8/351: symlink d0/d19/d52/d35/d44/l6a 0 2024-04-26T04:52:51.020 INFO:tasks.workunit.client.0.smithi077.stdout:3/71: read d0/d7/f8 [115007,119760] 0 2024-04-26T04:52:51.021 INFO:tasks.workunit.client.0.smithi077.stdout:4/88: symlink d4/de/d19/l1d 0 2024-04-26T04:52:51.023 INFO:tasks.workunit.client.1.smithi078.stdout:1/416: write d7/d39/d17/f3c [499936,60023] 0 2024-04-26T04:52:51.026 INFO:tasks.workunit.client.1.smithi078.stdout:9/329: write dc/df/d17/f4c [2029133,40661] 0 2024-04-26T04:52:51.027 INFO:tasks.workunit.client.1.smithi078.stdout:3/343: sync 2024-04-26T04:52:51.028 INFO:tasks.workunit.client.1.smithi078.stdout:3/344: fdatasync d16/d26/d2b/d31/f65 0 2024-04-26T04:52:51.029 INFO:tasks.workunit.client.0.smithi077.stdout:4/89: symlink d4/d17/d18/l1e 0 2024-04-26T04:52:51.029 INFO:tasks.workunit.client.1.smithi078.stdout:3/345: chown d16/d26/d2b/d66/f6f 50117400 1 2024-04-26T04:52:51.029 INFO:tasks.workunit.client.1.smithi078.stdout:3/346: readlink d16/d26/d2b/l53 0 2024-04-26T04:52:51.030 INFO:tasks.workunit.client.1.smithi078.stdout:3/347: stat d16/d3c/f61 0 2024-04-26T04:52:51.030 INFO:tasks.workunit.client.1.smithi078.stdout:3/348: stat d16/d26/d2e/f75 0 2024-04-26T04:52:51.031 INFO:tasks.workunit.client.1.smithi078.stdout:0/443: symlink d2/l91 0 2024-04-26T04:52:51.032 INFO:tasks.workunit.client.0.smithi077.stdout:2/39: sync 2024-04-26T04:52:51.032 INFO:tasks.workunit.client.0.smithi077.stdout:9/66: sync 2024-04-26T04:52:51.032 INFO:tasks.workunit.client.1.smithi078.stdout:4/392: creat d0/d25/d20/d31/d33/d44/d55/f95 x:0 0 0 2024-04-26T04:52:51.034 INFO:tasks.workunit.client.1.smithi078.stdout:8/352: mkdir d0/d19/d52/d35/d44/d6b 0 2024-04-26T04:52:51.034 INFO:tasks.workunit.client.0.smithi077.stdout:4/90: rmdir d4 39 2024-04-26T04:52:51.035 INFO:tasks.workunit.client.1.smithi078.stdout:8/353: stat d0/f10 0 2024-04-26T04:52:51.035 INFO:tasks.workunit.client.1.smithi078.stdout:7/369: mknod d9/da/d11/d13/c76 0 2024-04-26T04:52:51.036 INFO:tasks.workunit.client.1.smithi078.stdout:5/278: dwrite d7/d29/d46/f4f [0,4194304] 0 2024-04-26T04:52:51.038 INFO:tasks.workunit.client.0.smithi077.stdout:2/40: fsync f7 0 2024-04-26T04:52:51.041 INFO:tasks.workunit.client.1.smithi078.stdout:2/345: creat d8/d11/d6e/f7b x:0 0 0 2024-04-26T04:52:51.041 INFO:tasks.workunit.client.1.smithi078.stdout:4/393: dwrite d0/de/d22/d27/d7b/f82 [0,4194304] 0 2024-04-26T04:52:51.041 INFO:tasks.workunit.client.0.smithi077.stdout:3/72: getdents d0/d7 0 2024-04-26T04:52:51.043 INFO:tasks.workunit.client.1.smithi078.stdout:4/394: stat d0/de/d22/d26/f8d 0 2024-04-26T04:52:51.043 INFO:tasks.workunit.client.1.smithi078.stdout:1/417: mknod d7/d39/db/d37/d58/d5c/d68/c8d 0 2024-04-26T04:52:51.046 INFO:tasks.workunit.client.0.smithi077.stdout:2/41: dread f8 [0,4194304] 0 2024-04-26T04:52:51.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/279: dwrite d7/d11/f2c [0,4194304] 0 2024-04-26T04:52:51.047 INFO:tasks.workunit.client.1.smithi078.stdout:9/330: mknod dc/df/d17/c61 0 2024-04-26T04:52:51.047 INFO:tasks.workunit.client.0.smithi077.stdout:4/91: mkdir d4/de/d15/d1f 0 2024-04-26T04:52:51.049 INFO:tasks.workunit.client.0.smithi077.stdout:3/73: creat d0/d7/d15/f1a x:0 0 0 2024-04-26T04:52:51.052 INFO:tasks.workunit.client.0.smithi077.stdout:9/67: creat d9/f1c x:0 0 0 2024-04-26T04:52:51.053 INFO:tasks.workunit.client.0.smithi077.stdout:4/92: dread d4/f14 [0,4194304] 0 2024-04-26T04:52:51.054 INFO:tasks.workunit.client.0.smithi077.stdout:2/42: rmdir d9 39 2024-04-26T04:52:51.054 INFO:tasks.workunit.client.1.smithi078.stdout:3/349: dread fd [0,4194304] 0 2024-04-26T04:52:51.054 INFO:tasks.workunit.client.0.smithi077.stdout:4/93: stat d4/de/d15/d1f 0 2024-04-26T04:52:51.056 INFO:tasks.workunit.client.1.smithi078.stdout:8/354: chown d0/c11 485193 1 2024-04-26T04:52:51.056 INFO:tasks.workunit.client.0.smithi077.stdout:3/74: dread d0/f3 [4194304,4194304] 0 2024-04-26T04:52:51.057 INFO:tasks.workunit.client.0.smithi077.stdout:4/94: read d4/de/d19/f1b [2072309,48397] 0 2024-04-26T04:52:51.057 INFO:tasks.workunit.client.1.smithi078.stdout:6/430: sync 2024-04-26T04:52:51.063 INFO:tasks.workunit.client.0.smithi077.stdout:4/95: unlink l3 0 2024-04-26T04:52:51.064 INFO:tasks.workunit.client.1.smithi078.stdout:2/346: creat d8/d11/d18/d5d/d61/d1a/d34/f7c x:0 0 0 2024-04-26T04:52:51.064 INFO:tasks.workunit.client.0.smithi077.stdout:3/75: dwrite d0/d7/d15/f1a [0,4194304] 0 2024-04-26T04:52:51.067 INFO:tasks.workunit.client.1.smithi078.stdout:4/395: read d0/de/d22/d26/f57 [20560,18023] 0 2024-04-26T04:52:51.068 INFO:tasks.workunit.client.0.smithi077.stdout:2/43: creat d9/ff x:0 0 0 2024-04-26T04:52:51.069 INFO:tasks.workunit.client.1.smithi078.stdout:5/280: mknod d7/da/d1e/d4d/c61 0 2024-04-26T04:52:51.070 INFO:tasks.workunit.client.0.smithi077.stdout:9/68: truncate d9/fc 9272417 0 2024-04-26T04:52:51.072 INFO:tasks.workunit.client.0.smithi077.stdout:4/96: write d4/f14 [1493762,72095] 0 2024-04-26T04:52:51.073 INFO:tasks.workunit.client.1.smithi078.stdout:0/444: fsync d2/f21 0 2024-04-26T04:52:51.082 INFO:tasks.workunit.client.1.smithi078.stdout:7/370: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/d77 0 2024-04-26T04:52:51.083 INFO:tasks.workunit.client.1.smithi078.stdout:2/347: mknod d8/d53/c7d 0 2024-04-26T04:52:51.087 INFO:tasks.workunit.client.1.smithi078.stdout:4/396: rename d0/d25/d20/d31/d33/d4c/f81 to d0/de/d22/d27/f96 0 2024-04-26T04:52:51.088 INFO:tasks.workunit.client.0.smithi077.stdout:9/69: rmdir d9 39 2024-04-26T04:52:51.091 INFO:tasks.workunit.client.0.smithi077.stdout:3/76: write d0/f19 [820799,35017] 0 2024-04-26T04:52:51.091 INFO:tasks.workunit.client.1.smithi078.stdout:8/355: write d0/d19/f2a [1385484,62383] 0 2024-04-26T04:52:51.096 INFO:tasks.workunit.client.1.smithi078.stdout:3/350: mknod d16/d26/d2b/c79 0 2024-04-26T04:52:51.096 INFO:tasks.workunit.client.1.smithi078.stdout:3/351: dread - d16/d26/d2b/d54/f6e zero size 2024-04-26T04:52:51.099 INFO:tasks.workunit.client.0.smithi077.stdout:4/97: getdents d4/d17 0 2024-04-26T04:52:51.100 INFO:tasks.workunit.client.0.smithi077.stdout:2/44: dwrite f8 [4194304,4194304] 0 2024-04-26T04:52:51.101 INFO:tasks.workunit.client.0.smithi077.stdout:2/45: fsync d9/fe 0 2024-04-26T04:52:51.102 INFO:tasks.workunit.client.1.smithi078.stdout:3/352: dwrite d16/d26/d2b/d31/f3d [0,4194304] 0 2024-04-26T04:52:51.102 INFO:tasks.workunit.client.1.smithi078.stdout:6/431: creat d2/d1a/d5/d3a/d42/f84 x:0 0 0 2024-04-26T04:52:51.103 INFO:tasks.workunit.client.0.smithi077.stdout:3/77: dwrite d0/f19 [0,4194304] 0 2024-04-26T04:52:51.103 INFO:tasks.workunit.client.0.smithi077.stdout:3/78: stat d0/f3 0 2024-04-26T04:52:51.103 INFO:tasks.workunit.client.1.smithi078.stdout:3/353: chown d16/d26/d2e/d5f/d49 1856 1 2024-04-26T04:52:51.104 INFO:tasks.workunit.client.0.smithi077.stdout:9/70: fsync d9/f10 0 2024-04-26T04:52:51.105 INFO:tasks.workunit.client.1.smithi078.stdout:7/371: symlink d9/da/d11/d13/d1a/d2d/l78 0 2024-04-26T04:52:51.105 INFO:tasks.workunit.client.1.smithi078.stdout:6/432: read d2/d1a/d5/d11/f4e [485055,123315] 0 2024-04-26T04:52:51.105 INFO:tasks.workunit.client.1.smithi078.stdout:6/433: chown d2/d1a/l5d 9539780 1 2024-04-26T04:52:51.107 INFO:tasks.workunit.client.1.smithi078.stdout:6/434: fdatasync d2/d1a/d5/d3a/d42/f5a 0 2024-04-26T04:52:51.109 INFO:tasks.workunit.client.1.smithi078.stdout:6/435: fsync d2/d1a/d5/d3a/d42/f7d 0 2024-04-26T04:52:51.112 INFO:tasks.workunit.client.1.smithi078.stdout:4/397: write d0/de/d22/d26/d2e/f52 [1055503,19408] 0 2024-04-26T04:52:51.116 INFO:tasks.workunit.client.0.smithi077.stdout:2/46: link d9/fe d9/f10 0 2024-04-26T04:52:51.117 INFO:tasks.workunit.client.0.smithi077.stdout:2/47: readlink l6 0 2024-04-26T04:52:51.120 INFO:tasks.workunit.client.1.smithi078.stdout:7/372: unlink d9/l72 0 2024-04-26T04:52:51.120 INFO:tasks.workunit.client.1.smithi078.stdout:1/418: link d7/f11 d7/f8e 0 2024-04-26T04:52:51.121 INFO:tasks.workunit.client.0.smithi077.stdout:9/71: symlink d9/l1d 0 2024-04-26T04:52:51.121 INFO:tasks.workunit.client.0.smithi077.stdout:9/72: readlink d9/dd/le 0 2024-04-26T04:52:51.123 INFO:tasks.workunit.client.0.smithi077.stdout:2/48: mkdir d9/d11 0 2024-04-26T04:52:51.124 INFO:tasks.workunit.client.1.smithi078.stdout:9/331: getdents dc/d55 0 2024-04-26T04:52:51.125 INFO:tasks.workunit.client.0.smithi077.stdout:2/49: dread - d9/fe zero size 2024-04-26T04:52:51.125 INFO:tasks.workunit.client.0.smithi077.stdout:3/79: creat d0/f1b x:0 0 0 2024-04-26T04:52:51.125 INFO:tasks.workunit.client.1.smithi078.stdout:3/354: dread d16/f21 [0,4194304] 0 2024-04-26T04:52:51.126 INFO:tasks.workunit.client.0.smithi077.stdout:4/98: mkdir d4/de/d15/d1f/d20 0 2024-04-26T04:52:51.126 INFO:tasks.workunit.client.0.smithi077.stdout:3/80: read d0/d7/f8 [3072861,95195] 0 2024-04-26T04:52:51.127 INFO:tasks.workunit.client.1.smithi078.stdout:4/398: rename d0/d25/d20/d31/d33/l4e to d0/d25/d20/d31/d33/d4c/d8e/l97 0 2024-04-26T04:52:51.127 INFO:tasks.workunit.client.1.smithi078.stdout:6/436: dwrite d2/d1a/d5/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:52:51.129 INFO:tasks.workunit.client.1.smithi078.stdout:8/356: mkdir d0/d19/d52/d35/d44/d6b/d6c 0 2024-04-26T04:52:51.130 INFO:tasks.workunit.client.1.smithi078.stdout:8/357: chown d0/d19/d52/d5c/d68 130701752 1 2024-04-26T04:52:51.134 INFO:tasks.workunit.client.0.smithi077.stdout:2/50: rename f8 to d9/f12 0 2024-04-26T04:52:51.135 INFO:tasks.workunit.client.0.smithi077.stdout:2/51: chown d9/f12 54312310 1 2024-04-26T04:52:51.136 INFO:tasks.workunit.client.1.smithi078.stdout:1/419: symlink d7/d39/d4c/d3d/l8f 0 2024-04-26T04:52:51.137 INFO:tasks.workunit.client.1.smithi078.stdout:1/420: write d7/f13 [318406,1758] 0 2024-04-26T04:52:51.138 INFO:tasks.workunit.client.1.smithi078.stdout:0/445: getdents d2/d9/d60/d7f 0 2024-04-26T04:52:51.139 INFO:tasks.workunit.client.0.smithi077.stdout:4/99: write f2 [2310631,75290] 0 2024-04-26T04:52:51.145 INFO:tasks.workunit.client.1.smithi078.stdout:9/332: symlink dc/df/d4b/l62 0 2024-04-26T04:52:51.146 INFO:tasks.workunit.client.0.smithi077.stdout:8/95: sync 2024-04-26T04:52:51.146 INFO:tasks.workunit.client.0.smithi077.stdout:7/45: sync 2024-04-26T04:52:51.148 INFO:tasks.workunit.client.1.smithi078.stdout:7/373: write d9/da/d11/d13/d1a/d2d/f4f [257302,61305] 0 2024-04-26T04:52:51.149 INFO:tasks.workunit.client.0.smithi077.stdout:3/81: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:52:51.149 INFO:tasks.workunit.client.0.smithi077.stdout:4/100: dwrite d4/de/d15/f16 [0,4194304] 0 2024-04-26T04:52:51.153 INFO:tasks.workunit.client.1.smithi078.stdout:6/437: symlink d2/d52/l85 0 2024-04-26T04:52:51.154 INFO:tasks.workunit.client.1.smithi078.stdout:7/374: dwrite d9/da/d11/d13/d1a/d2d/f3d [0,4194304] 0 2024-04-26T04:52:51.154 INFO:tasks.workunit.client.0.smithi077.stdout:7/46: mknod da/ce 0 2024-04-26T04:52:51.157 INFO:tasks.workunit.client.0.smithi077.stdout:4/101: dread f2 [0,4194304] 0 2024-04-26T04:52:51.157 INFO:tasks.workunit.client.0.smithi077.stdout:3/82: dread d0/f3 [4194304,4194304] 0 2024-04-26T04:52:51.157 INFO:tasks.workunit.client.0.smithi077.stdout:8/96: creat d4/df/f25 x:0 0 0 2024-04-26T04:52:51.158 INFO:tasks.workunit.client.0.smithi077.stdout:2/52: creat d9/d11/f13 x:0 0 0 2024-04-26T04:52:51.159 INFO:tasks.workunit.client.1.smithi078.stdout:1/421: creat d7/d39/db/d47/f90 x:0 0 0 2024-04-26T04:52:51.159 INFO:tasks.workunit.client.0.smithi077.stdout:4/102: read f2 [1387801,71909] 0 2024-04-26T04:52:51.159 INFO:tasks.workunit.client.0.smithi077.stdout:2/53: readlink l6 0 2024-04-26T04:52:51.160 INFO:tasks.workunit.client.0.smithi077.stdout:2/54: readlink l6 0 2024-04-26T04:52:51.160 INFO:tasks.workunit.client.1.smithi078.stdout:9/333: rmdir dc/df/d4b 39 2024-04-26T04:52:51.162 INFO:tasks.workunit.client.0.smithi077.stdout:9/73: dwrite d9/fc [0,4194304] 0 2024-04-26T04:52:51.162 INFO:tasks.workunit.client.1.smithi078.stdout:1/422: truncate d7/d1e/f7f 159179 0 2024-04-26T04:52:51.163 INFO:tasks.workunit.client.1.smithi078.stdout:6/438: dwrite d2/d1a/d5/d3a/d42/f7d [0,4194304] 0 2024-04-26T04:52:51.163 INFO:tasks.workunit.client.1.smithi078.stdout:1/423: stat d7/d39/d4c/d3d/c77 0 2024-04-26T04:52:51.165 INFO:tasks.workunit.client.0.smithi077.stdout:8/97: unlink c3 0 2024-04-26T04:52:51.165 INFO:tasks.workunit.client.1.smithi078.stdout:1/424: chown f1 1133409 1 2024-04-26T04:52:51.165 INFO:tasks.workunit.client.1.smithi078.stdout:1/425: stat d7/ff 0 2024-04-26T04:52:51.168 INFO:tasks.workunit.client.1.smithi078.stdout:7/375: symlink d9/l79 0 2024-04-26T04:52:51.170 INFO:tasks.workunit.client.0.smithi077.stdout:4/103: symlink d4/de/d19/l21 0 2024-04-26T04:52:51.172 INFO:tasks.workunit.client.0.smithi077.stdout:9/74: rmdir d9 39 2024-04-26T04:52:51.175 INFO:tasks.workunit.client.1.smithi078.stdout:7/376: dwrite d9/da/d11/f45 [0,4194304] 0 2024-04-26T04:52:51.180 INFO:tasks.workunit.client.1.smithi078.stdout:4/399: rename d0/d25/d20/d31/d33/d44/f3d to d0/de/d22/f98 0 2024-04-26T04:52:51.184 INFO:tasks.workunit.client.0.smithi077.stdout:4/104: readlink d4/l6 0 2024-04-26T04:52:51.185 INFO:tasks.workunit.client.1.smithi078.stdout:7/377: dread d9/da/d11/d13/d1a/f42 [0,4194304] 0 2024-04-26T04:52:51.185 INFO:tasks.workunit.client.0.smithi077.stdout:4/105: write d4/de/d15/f16 [97434,20641] 0 2024-04-26T04:52:51.186 INFO:tasks.workunit.client.1.smithi078.stdout:7/378: write d9/da/d11/d13/d1a/f4e [686903,66915] 0 2024-04-26T04:52:51.186 INFO:tasks.workunit.client.1.smithi078.stdout:9/334: symlink dc/df/d4b/d3e/l63 0 2024-04-26T04:52:51.190 INFO:tasks.workunit.client.0.smithi077.stdout:8/98: unlink d4/d1a/l21 0 2024-04-26T04:52:51.191 INFO:tasks.workunit.client.0.smithi077.stdout:9/75: dread d9/fc [0,4194304] 0 2024-04-26T04:52:51.191 INFO:tasks.workunit.client.0.smithi077.stdout:9/76: readlink d9/l1d 0 2024-04-26T04:52:51.191 INFO:tasks.workunit.client.0.smithi077.stdout:3/83: creat d0/f1c x:0 0 0 2024-04-26T04:52:51.193 INFO:tasks.workunit.client.1.smithi078.stdout:8/358: rename d0/d19/d52/d3c/c56 to d0/d19/d52/d35/d3f/c6d 0 2024-04-26T04:52:51.194 INFO:tasks.workunit.client.1.smithi078.stdout:6/439: dwrite d2/d1a/d5/d2a/d4d/f60 [4194304,4194304] 0 2024-04-26T04:52:51.194 INFO:tasks.workunit.client.1.smithi078.stdout:4/400: mknod d0/de/d22/d27/d56/c99 0 2024-04-26T04:52:51.196 INFO:tasks.workunit.client.1.smithi078.stdout:6/440: write d2/d52/f7a [872966,50893] 0 2024-04-26T04:52:51.196 INFO:tasks.workunit.client.1.smithi078.stdout:4/401: chown d0/de/f78 837 1 2024-04-26T04:52:51.196 INFO:tasks.workunit.client.1.smithi078.stdout:8/359: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:51.196 INFO:tasks.workunit.client.1.smithi078.stdout:6/441: chown d2/d1a/d5 426399633 1 2024-04-26T04:52:51.198 INFO:tasks.workunit.client.0.smithi077.stdout:8/99: dread - d4/f10 zero size 2024-04-26T04:52:51.199 INFO:tasks.workunit.client.1.smithi078.stdout:5/281: sync 2024-04-26T04:52:51.199 INFO:tasks.workunit.client.1.smithi078.stdout:8/360: write d0/d19/d52/d35/d44/f59 [685192,15075] 0 2024-04-26T04:52:51.200 INFO:tasks.workunit.client.0.smithi077.stdout:3/84: unlink d0/d7/fe 0 2024-04-26T04:52:51.201 INFO:tasks.workunit.client.1.smithi078.stdout:5/282: write d7/d11/f2c [4326576,101398] 0 2024-04-26T04:52:51.202 INFO:tasks.workunit.client.0.smithi077.stdout:4/106: dwrite f2 [0,4194304] 0 2024-04-26T04:52:51.203 INFO:tasks.workunit.client.1.smithi078.stdout:5/283: write d7/d18/f1a [1025818,41735] 0 2024-04-26T04:52:51.204 INFO:tasks.workunit.client.1.smithi078.stdout:5/284: readlink d7/da/l45 0 2024-04-26T04:52:51.204 INFO:tasks.workunit.client.1.smithi078.stdout:4/402: mknod d0/de/d22/d27/d56/d5a/c9a 0 2024-04-26T04:52:51.206 INFO:tasks.workunit.client.0.smithi077.stdout:9/77: rename d9/c15 to d9/dd/c1e 0 2024-04-26T04:52:51.208 INFO:tasks.workunit.client.0.smithi077.stdout:8/100: creat d4/df/d18/f26 x:0 0 0 2024-04-26T04:52:51.210 INFO:tasks.workunit.client.1.smithi078.stdout:5/285: dwrite d7/d21/f42 [0,4194304] 0 2024-04-26T04:52:51.211 INFO:tasks.workunit.client.1.smithi078.stdout:5/286: dread - d7/d11/f47 zero size 2024-04-26T04:52:51.211 INFO:tasks.workunit.client.1.smithi078.stdout:3/355: truncate d16/f17 3386247 0 2024-04-26T04:52:51.212 INFO:tasks.workunit.client.1.smithi078.stdout:3/356: chown d16/d26/d2b/d66/f6f 104798128 1 2024-04-26T04:52:51.216 INFO:tasks.workunit.client.0.smithi077.stdout:4/107: unlink d4/de/d19/l1d 0 2024-04-26T04:52:51.216 INFO:tasks.workunit.client.0.smithi077.stdout:8/101: dwrite d4/df/f25 [0,4194304] 0 2024-04-26T04:52:51.219 INFO:tasks.workunit.client.1.smithi078.stdout:4/403: fdatasync d0/d25/d20/f23 0 2024-04-26T04:52:51.219 INFO:tasks.workunit.client.1.smithi078.stdout:5/287: dwrite d7/f38 [0,4194304] 0 2024-04-26T04:52:51.222 INFO:tasks.workunit.client.1.smithi078.stdout:1/426: write d7/d1e/f2f [1202070,36369] 0 2024-04-26T04:52:51.224 INFO:tasks.workunit.client.1.smithi078.stdout:1/427: write d7/d39/db/d49/f61 [4480461,60708] 0 2024-04-26T04:52:51.225 INFO:tasks.workunit.client.0.smithi077.stdout:2/55: write f5 [1467215,74545] 0 2024-04-26T04:52:51.226 INFO:tasks.workunit.client.1.smithi078.stdout:5/288: dwrite d7/d18/f40 [4194304,4194304] 0 2024-04-26T04:52:51.226 INFO:tasks.workunit.client.0.smithi077.stdout:2/56: dread - d9/fd zero size 2024-04-26T04:52:51.228 INFO:tasks.workunit.client.1.smithi078.stdout:5/289: write d7/f38 [3512713,122384] 0 2024-04-26T04:52:51.229 INFO:tasks.workunit.client.1.smithi078.stdout:8/361: fsync d0/d19/d52/d35/f4e 0 2024-04-26T04:52:51.234 INFO:tasks.workunit.client.1.smithi078.stdout:7/379: dread d9/da/f15 [0,4194304] 0 2024-04-26T04:52:51.236 INFO:tasks.workunit.client.1.smithi078.stdout:1/428: dwrite d7/d39/d17/f66 [0,4194304] 0 2024-04-26T04:52:51.238 INFO:tasks.workunit.client.0.smithi077.stdout:3/85: creat d0/f1d x:0 0 0 2024-04-26T04:52:51.239 INFO:tasks.workunit.client.0.smithi077.stdout:3/86: fdatasync d0/f1b 0 2024-04-26T04:52:51.240 INFO:tasks.workunit.client.1.smithi078.stdout:1/429: fsync d7/d1e/f31 0 2024-04-26T04:52:51.241 INFO:tasks.workunit.client.1.smithi078.stdout:1/430: dread - d7/f8a zero size 2024-04-26T04:52:51.241 INFO:tasks.workunit.client.0.smithi077.stdout:2/57: rmdir d9 39 2024-04-26T04:52:51.242 INFO:tasks.workunit.client.1.smithi078.stdout:3/357: dwrite d16/f21 [0,4194304] 0 2024-04-26T04:52:51.244 INFO:tasks.workunit.client.0.smithi077.stdout:2/58: fdatasync f5 0 2024-04-26T04:52:51.256 INFO:tasks.workunit.client.1.smithi078.stdout:7/380: dread d9/da/d33/d4b/f75 [0,4194304] 0 2024-04-26T04:52:51.258 INFO:tasks.workunit.client.1.smithi078.stdout:6/442: truncate d2/d1a/d5/da/d35/f70 3801274 0 2024-04-26T04:52:51.260 INFO:tasks.workunit.client.0.smithi077.stdout:9/78: getdents d9/dd 0 2024-04-26T04:52:51.260 INFO:tasks.workunit.client.1.smithi078.stdout:7/381: dread d9/da/d11/d13/d1a/d52/d5a/d41/f47 [4194304,4194304] 0 2024-04-26T04:52:51.266 INFO:tasks.workunit.client.0.smithi077.stdout:2/59: symlink d9/l14 0 2024-04-26T04:52:51.266 INFO:tasks.workunit.client.0.smithi077.stdout:2/60: write f5 [3618827,94794] 0 2024-04-26T04:52:51.274 INFO:tasks.workunit.client.0.smithi077.stdout:8/102: chown d4/df/d22/f24 9 1 2024-04-26T04:52:51.278 INFO:tasks.workunit.client.1.smithi078.stdout:6/443: write d2/d1a/d5/d2a/f3d [2668247,68493] 0 2024-04-26T04:52:51.278 INFO:tasks.workunit.client.1.smithi078.stdout:4/404: write d0/de/f34 [3518511,93358] 0 2024-04-26T04:52:51.278 INFO:tasks.workunit.client.0.smithi077.stdout:2/61: dwrite d9/d11/f13 [0,4194304] 0 2024-04-26T04:52:51.280 INFO:tasks.workunit.client.0.smithi077.stdout:4/108: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:51.286 INFO:tasks.workunit.client.1.smithi078.stdout:3/358: fdatasync f6 0 2024-04-26T04:52:51.290 INFO:tasks.workunit.client.1.smithi078.stdout:8/362: link d0/d19/d25/d36/l54 d0/d19/d52/d3c/l6e 0 2024-04-26T04:52:51.291 INFO:tasks.workunit.client.0.smithi077.stdout:4/109: symlink d4/de/d15/l22 0 2024-04-26T04:52:51.294 INFO:tasks.workunit.client.0.smithi077.stdout:3/87: dwrite d0/f19 [0,4194304] 0 2024-04-26T04:52:51.294 INFO:tasks.workunit.client.0.smithi077.stdout:3/88: chown d0/c13 1912 1 2024-04-26T04:52:51.295 INFO:tasks.workunit.client.0.smithi077.stdout:2/62: unlink d9/ca 0 2024-04-26T04:52:51.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/110: creat d4/de/d15/d1f/f23 x:0 0 0 2024-04-26T04:52:51.298 INFO:tasks.workunit.client.0.smithi077.stdout:4/111: dread - d4/de/d15/d1f/f23 zero size 2024-04-26T04:52:51.298 INFO:tasks.workunit.client.0.smithi077.stdout:3/89: symlink d0/d7/dd/l1e 0 2024-04-26T04:52:51.301 INFO:tasks.workunit.client.1.smithi078.stdout:4/405: mkdir d0/de/d22/d26/d2e/d85/d9b 0 2024-04-26T04:52:51.301 INFO:tasks.workunit.client.0.smithi077.stdout:4/112: fdatasync d4/f8 0 2024-04-26T04:52:51.305 INFO:tasks.workunit.client.0.smithi077.stdout:9/79: write f2 [4769133,11704] 0 2024-04-26T04:52:51.306 INFO:tasks.workunit.client.0.smithi077.stdout:3/90: mkdir d0/d7/d1f 0 2024-04-26T04:52:51.308 INFO:tasks.workunit.client.1.smithi078.stdout:3/359: unlink d16/d26/d2e/d5f/d49/c67 0 2024-04-26T04:52:51.309 INFO:tasks.workunit.client.0.smithi077.stdout:2/63: stat d9/cb 0 2024-04-26T04:52:51.309 INFO:tasks.workunit.client.0.smithi077.stdout:3/91: dread d0/fb [0,4194304] 0 2024-04-26T04:52:51.310 INFO:tasks.workunit.client.1.smithi078.stdout:3/360: read d16/d26/d2b/d31/f34 [175227,30853] 0 2024-04-26T04:52:51.312 INFO:tasks.workunit.client.1.smithi078.stdout:7/382: dwrite d9/da/d11/d13/d1a/d52/d5a/f44 [0,4194304] 0 2024-04-26T04:52:51.312 INFO:tasks.workunit.client.0.smithi077.stdout:4/113: chown d4/cc 652694103 1 2024-04-26T04:52:51.313 INFO:tasks.workunit.client.1.smithi078.stdout:7/383: readlink d9/da/d11/l69 0 2024-04-26T04:52:51.313 INFO:tasks.workunit.client.1.smithi078.stdout:8/363: symlink d0/l6f 0 2024-04-26T04:52:51.314 INFO:tasks.workunit.client.0.smithi077.stdout:9/80: mkdir d9/dd/d1f 0 2024-04-26T04:52:51.318 INFO:tasks.workunit.client.0.smithi077.stdout:2/64: dwrite d9/fd [0,4194304] 0 2024-04-26T04:52:51.320 INFO:tasks.workunit.client.1.smithi078.stdout:7/384: dwrite d9/da/d11/d13/d50/f59 [0,4194304] 0 2024-04-26T04:52:51.325 INFO:tasks.workunit.client.0.smithi077.stdout:3/92: creat d0/d7/d15/f20 x:0 0 0 2024-04-26T04:52:51.330 INFO:tasks.workunit.client.1.smithi078.stdout:1/431: getdents d7/d39/db/d47 0 2024-04-26T04:52:51.331 INFO:tasks.workunit.client.1.smithi078.stdout:1/432: truncate d7/d39/db/f41 4325842 0 2024-04-26T04:52:51.332 INFO:tasks.workunit.client.0.smithi077.stdout:9/81: symlink d9/dd/l20 0 2024-04-26T04:52:51.332 INFO:tasks.workunit.client.1.smithi078.stdout:2/348: sync 2024-04-26T04:52:51.332 INFO:tasks.workunit.client.1.smithi078.stdout:0/446: sync 2024-04-26T04:52:51.332 INFO:tasks.workunit.client.1.smithi078.stdout:6/444: write d2/d1a/d5/d3a/f62 [980933,42417] 0 2024-04-26T04:52:51.333 INFO:tasks.workunit.client.1.smithi078.stdout:6/445: chown d2/d49 395547788 1 2024-04-26T04:52:51.333 INFO:tasks.workunit.client.1.smithi078.stdout:0/447: stat d2/d9/d78 0 2024-04-26T04:52:51.334 INFO:tasks.workunit.client.1.smithi078.stdout:6/446: write d2/d1a/d5/da/f4a [1255083,55640] 0 2024-04-26T04:52:51.337 INFO:tasks.workunit.client.1.smithi078.stdout:3/361: symlink d16/d26/d2e/d74/l7a 0 2024-04-26T04:52:51.337 INFO:tasks.workunit.client.0.smithi077.stdout:1/88: sync 2024-04-26T04:52:51.337 INFO:tasks.workunit.client.0.smithi077.stdout:0/60: sync 2024-04-26T04:52:51.337 INFO:tasks.workunit.client.0.smithi077.stdout:6/89: sync 2024-04-26T04:52:51.337 INFO:tasks.workunit.client.0.smithi077.stdout:5/56: sync 2024-04-26T04:52:51.340 INFO:tasks.workunit.client.0.smithi077.stdout:4/114: creat d4/d17/f24 x:0 0 0 2024-04-26T04:52:51.341 INFO:tasks.workunit.client.1.smithi078.stdout:4/406: symlink d0/de/d22/d26/l9c 0 2024-04-26T04:52:51.348 INFO:tasks.workunit.client.1.smithi078.stdout:5/290: sync 2024-04-26T04:52:51.348 INFO:tasks.workunit.client.1.smithi078.stdout:9/335: sync 2024-04-26T04:52:51.395 INFO:tasks.workunit.client.0.smithi077.stdout:5/57: dwrite d1/fe [0,4194304] 0 2024-04-26T04:52:51.397 INFO:tasks.workunit.client.0.smithi077.stdout:9/82: dwrite d9/fc [0,4194304] 0 2024-04-26T04:52:51.398 INFO:tasks.workunit.client.0.smithi077.stdout:6/90: symlink d8/l20 0 2024-04-26T04:52:51.404 INFO:tasks.workunit.client.0.smithi077.stdout:4/115: mkdir d4/de/d15/d1f/d25 0 2024-04-26T04:52:51.405 INFO:tasks.workunit.client.0.smithi077.stdout:4/116: chown d4/de/d15/d1f/f23 1447 1 2024-04-26T04:52:51.407 INFO:tasks.workunit.client.0.smithi077.stdout:4/117: write d4/de/d15/f16 [709064,42078] 0 2024-04-26T04:52:51.408 INFO:tasks.workunit.client.1.smithi078.stdout:3/362: rename d16/f21 to d16/d26/d2e/d5f/d49/d77/f7b 0 2024-04-26T04:52:51.411 INFO:tasks.workunit.client.1.smithi078.stdout:8/364: link d0/d19/d25/d36/f48 d0/d19/d25/d36/d58/f70 0 2024-04-26T04:52:51.414 INFO:tasks.workunit.client.1.smithi078.stdout:4/407: rmdir d0/d25 39 2024-04-26T04:52:51.415 INFO:tasks.workunit.client.0.smithi077.stdout:4/118: dwrite d4/d17/f24 [0,4194304] 0 2024-04-26T04:52:51.416 INFO:tasks.workunit.client.0.smithi077.stdout:5/58: unlink d1/lb 0 2024-04-26T04:52:51.418 INFO:tasks.workunit.client.1.smithi078.stdout:5/291: creat d7/da/f62 x:0 0 0 2024-04-26T04:52:51.420 INFO:tasks.workunit.client.1.smithi078.stdout:9/336: mkdir dc/df/d37/d64 0 2024-04-26T04:52:51.420 INFO:tasks.workunit.client.1.smithi078.stdout:9/337: dread - dc/d55/f52 zero size 2024-04-26T04:52:51.421 INFO:tasks.workunit.client.0.smithi077.stdout:6/91: mknod d8/d1c/c21 0 2024-04-26T04:52:51.422 INFO:tasks.workunit.client.0.smithi077.stdout:4/119: dread d4/f8 [0,4194304] 0 2024-04-26T04:52:51.426 INFO:tasks.workunit.client.0.smithi077.stdout:5/59: dwrite d1/f4 [0,4194304] 0 2024-04-26T04:52:51.426 INFO:tasks.workunit.client.1.smithi078.stdout:1/433: link d7/d2e/f5b d7/d39/d4c/d3d/f91 0 2024-04-26T04:52:51.428 INFO:tasks.workunit.client.0.smithi077.stdout:4/120: dread d4/d17/f24 [0,4194304] 0 2024-04-26T04:52:51.431 INFO:tasks.workunit.client.1.smithi078.stdout:0/448: mkdir d2/d9/d60/d6a/d8f/d92 0 2024-04-26T04:52:51.433 INFO:tasks.workunit.client.1.smithi078.stdout:6/447: mknod d2/d1a/d5/d3a/d69/c86 0 2024-04-26T04:52:51.438 INFO:tasks.workunit.client.1.smithi078.stdout:7/385: dwrite d9/da/d11/d13/f17 [0,4194304] 0 2024-04-26T04:52:51.438 INFO:tasks.workunit.client.1.smithi078.stdout:3/363: creat d16/d26/d2e/d5f/d49/f7c x:0 0 0 2024-04-26T04:52:51.439 INFO:tasks.workunit.client.0.smithi077.stdout:9/83: mknod d9/dd/d1b/c21 0 2024-04-26T04:52:51.440 INFO:tasks.workunit.client.1.smithi078.stdout:3/364: stat d16/d26/d2b/d54/d5a/f5d 0 2024-04-26T04:52:51.440 INFO:tasks.workunit.client.1.smithi078.stdout:8/365: dread d0/d19/f1c [0,4194304] 0 2024-04-26T04:52:51.441 INFO:tasks.workunit.client.0.smithi077.stdout:6/92: fdatasync f4 0 2024-04-26T04:52:51.442 INFO:tasks.workunit.client.1.smithi078.stdout:4/408: mknod d0/de/d22/d27/d56/d5a/c9d 0 2024-04-26T04:52:51.445 INFO:tasks.workunit.client.1.smithi078.stdout:5/292: symlink d7/da/d1e/l63 0 2024-04-26T04:52:51.445 INFO:tasks.workunit.client.0.smithi077.stdout:9/84: dwrite f2 [0,4194304] 0 2024-04-26T04:52:51.447 INFO:tasks.workunit.client.1.smithi078.stdout:9/338: symlink dc/df/d4b/d3e/l65 0 2024-04-26T04:52:51.448 INFO:tasks.workunit.client.1.smithi078.stdout:9/339: read - dc/df/d27/f5c zero size 2024-04-26T04:52:51.449 INFO:tasks.workunit.client.0.smithi077.stdout:0/61: rmdir d0/db/d13 0 2024-04-26T04:52:51.450 INFO:tasks.workunit.client.0.smithi077.stdout:0/62: read d0/db/ff [547475,119454] 0 2024-04-26T04:52:51.452 INFO:tasks.workunit.client.0.smithi077.stdout:0/63: dread d0/db/ff [0,4194304] 0 2024-04-26T04:52:51.453 INFO:tasks.workunit.client.1.smithi078.stdout:1/434: chown d7/d39/dd/d20/d53/f70 631 1 2024-04-26T04:52:51.453 INFO:tasks.workunit.client.1.smithi078.stdout:1/435: write d7/d39/db/d37/f88 [743518,20249] 0 2024-04-26T04:52:51.454 INFO:tasks.workunit.client.0.smithi077.stdout:1/89: rename d9/c18 to d9/db/c19 0 2024-04-26T04:52:51.455 INFO:tasks.workunit.client.0.smithi077.stdout:4/121: dread d4/f14 [0,4194304] 0 2024-04-26T04:52:51.458 INFO:tasks.workunit.client.1.smithi078.stdout:2/349: creat d8/d11/d18/d5d/f7e x:0 0 0 2024-04-26T04:52:51.458 INFO:tasks.workunit.client.1.smithi078.stdout:6/448: rmdir d2/de/d27/d4c 39 2024-04-26T04:52:51.458 INFO:tasks.workunit.client.1.smithi078.stdout:2/350: dread - d8/d11/d18/d5d/d61/d1a/f5c zero size 2024-04-26T04:52:51.460 INFO:tasks.workunit.client.1.smithi078.stdout:1/436: dwrite d7/d1e/f2f [0,4194304] 0 2024-04-26T04:52:51.460 INFO:tasks.workunit.client.0.smithi077.stdout:5/60: mknod d1/c14 0 2024-04-26T04:52:51.466 INFO:tasks.workunit.client.1.smithi078.stdout:0/449: write d2/d9/d60/d6a/f89 [585982,125836] 0 2024-04-26T04:52:51.467 INFO:tasks.workunit.client.0.smithi077.stdout:0/64: fdatasync d0/f4 0 2024-04-26T04:52:51.467 INFO:tasks.workunit.client.1.smithi078.stdout:0/450: write d2/d9/d31/f85 [279078,41134] 0 2024-04-26T04:52:51.469 INFO:tasks.workunit.client.0.smithi077.stdout:4/122: fsync d4/d17/f24 0 2024-04-26T04:52:51.471 INFO:tasks.workunit.client.1.smithi078.stdout:6/449: rmdir d2/d1a/d5/da 39 2024-04-26T04:52:51.472 INFO:tasks.workunit.client.1.smithi078.stdout:4/409: dwrite d0/de/d22/d71/f79 [0,4194304] 0 2024-04-26T04:52:51.472 INFO:tasks.workunit.client.0.smithi077.stdout:5/61: rename d1 to d1/d8/d15 22 2024-04-26T04:52:51.473 INFO:tasks.workunit.client.0.smithi077.stdout:5/62: fdatasync d1/d12/f13 0 2024-04-26T04:52:51.476 INFO:tasks.workunit.client.0.smithi077.stdout:2/65: truncate d9/f12 4295474 0 2024-04-26T04:52:51.478 INFO:tasks.workunit.client.0.smithi077.stdout:2/66: read f5 [3333260,102937] 0 2024-04-26T04:52:51.480 INFO:tasks.workunit.client.0.smithi077.stdout:0/65: rename d0/l5 to d0/l1d 0 2024-04-26T04:52:51.481 INFO:tasks.workunit.client.1.smithi078.stdout:2/351: dwrite d8/d11/f2a [0,4194304] 0 2024-04-26T04:52:51.482 INFO:tasks.workunit.client.0.smithi077.stdout:1/90: symlink d9/db/l1a 0 2024-04-26T04:52:51.483 INFO:tasks.workunit.client.0.smithi077.stdout:4/123: creat d4/de/f26 x:0 0 0 2024-04-26T04:52:51.483 INFO:tasks.workunit.client.0.smithi077.stdout:1/91: chown d9/db/dd/f10 4 1 2024-04-26T04:52:51.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/92: readlink d9/db/dd/lf 0 2024-04-26T04:52:51.485 INFO:tasks.workunit.client.1.smithi078.stdout:5/293: creat d7/d18/d1c/d5a/f64 x:0 0 0 2024-04-26T04:52:51.486 INFO:tasks.workunit.client.1.smithi078.stdout:9/340: creat dc/df/d37/d64/f66 x:0 0 0 2024-04-26T04:52:51.487 INFO:tasks.workunit.client.1.smithi078.stdout:9/341: fsync dc/f54 0 2024-04-26T04:52:51.488 INFO:tasks.workunit.client.1.smithi078.stdout:9/342: write dc/df/d4b/f1c [3318895,12924] 0 2024-04-26T04:52:51.488 INFO:tasks.workunit.client.0.smithi077.stdout:4/124: dread d4/f8 [0,4194304] 0 2024-04-26T04:52:51.489 INFO:tasks.workunit.client.0.smithi077.stdout:4/125: stat d4/de/d15/d1f/d25 0 2024-04-26T04:52:51.491 INFO:tasks.workunit.client.0.smithi077.stdout:1/93: dread d9/db/dd/f11 [0,4194304] 0 2024-04-26T04:52:51.491 INFO:tasks.workunit.client.1.smithi078.stdout:0/451: symlink d2/d9/d60/d6a/l93 0 2024-04-26T04:52:51.491 INFO:tasks.workunit.client.0.smithi077.stdout:1/94: read - d9/db/f13 zero size 2024-04-26T04:52:51.493 INFO:tasks.workunit.client.0.smithi077.stdout:1/95: read d9/db/dd/f15 [318468,41132] 0 2024-04-26T04:52:51.493 INFO:tasks.workunit.client.1.smithi078.stdout:9/343: dwrite dc/df/d27/f13 [0,4194304] 0 2024-04-26T04:52:51.495 INFO:tasks.workunit.client.0.smithi077.stdout:1/96: fdatasync d9/db/dd/f15 0 2024-04-26T04:52:51.495 INFO:tasks.workunit.client.0.smithi077.stdout:9/85: rename d9/fc to d9/f22 0 2024-04-26T04:52:51.495 INFO:tasks.workunit.client.0.smithi077.stdout:9/86: fsync d9/dd/f1a 0 2024-04-26T04:52:51.497 INFO:tasks.workunit.client.1.smithi078.stdout:8/366: truncate d0/d19/d25/d32/f4b 1586249 0 2024-04-26T04:52:51.498 INFO:tasks.workunit.client.1.smithi078.stdout:8/367: fsync d0/d19/d52/d35/d44/f59 0 2024-04-26T04:52:51.501 INFO:tasks.workunit.client.0.smithi077.stdout:0/66: mkdir d0/d19/d1e 0 2024-04-26T04:52:51.503 INFO:tasks.workunit.client.1.smithi078.stdout:1/437: mkdir d7/d39/d92 0 2024-04-26T04:52:51.504 INFO:tasks.workunit.client.1.smithi078.stdout:1/438: write d7/d39/db/d49/f84 [619196,32554] 0 2024-04-26T04:52:51.505 INFO:tasks.workunit.client.1.smithi078.stdout:7/386: link d9/da/d33/d4b/f75 d9/d57/f7a 0 2024-04-26T04:52:51.507 INFO:tasks.workunit.client.1.smithi078.stdout:7/387: read - d9/da/d11/d13/d1a/f1e zero size 2024-04-26T04:52:51.507 INFO:tasks.workunit.client.1.smithi078.stdout:3/365: link f6 d16/d26/d2b/d54/d56/f7d 0 2024-04-26T04:52:51.508 INFO:tasks.workunit.client.0.smithi077.stdout:6/93: getdents d8/d1c 0 2024-04-26T04:52:51.508 INFO:tasks.workunit.client.0.smithi077.stdout:0/67: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:52:51.508 INFO:tasks.workunit.client.0.smithi077.stdout:0/68: chown d0/l10 230 1 2024-04-26T04:52:51.513 INFO:tasks.workunit.client.1.smithi078.stdout:7/388: dwrite d9/da/d11/d13/d1a/f1e [0,4194304] 0 2024-04-26T04:52:51.519 INFO:tasks.workunit.client.0.smithi077.stdout:1/97: creat d9/f1b x:0 0 0 2024-04-26T04:52:51.524 INFO:tasks.workunit.client.1.smithi078.stdout:0/452: unlink d2/f14 0 2024-04-26T04:52:51.526 INFO:tasks.workunit.client.0.smithi077.stdout:2/67: fsync d9/fe 0 2024-04-26T04:52:51.527 INFO:tasks.workunit.client.0.smithi077.stdout:2/68: stat d9/fe 0 2024-04-26T04:52:51.528 INFO:tasks.workunit.client.1.smithi078.stdout:9/344: unlink dc/df/d17/f4c 0 2024-04-26T04:52:51.531 INFO:tasks.workunit.client.1.smithi078.stdout:4/410: mkdir d0/d25/d20/d31/d33/d44/d55/d5e/d9e 0 2024-04-26T04:52:51.532 INFO:tasks.workunit.client.0.smithi077.stdout:6/94: creat d8/f22 x:0 0 0 2024-04-26T04:52:51.532 INFO:tasks.workunit.client.0.smithi077.stdout:7/47: sync 2024-04-26T04:52:51.533 INFO:tasks.workunit.client.1.smithi078.stdout:8/368: mknod d0/d19/d52/d5c/c71 0 2024-04-26T04:52:51.534 INFO:tasks.workunit.client.1.smithi078.stdout:1/439: mkdir d7/d39/db/d49/d93 0 2024-04-26T04:52:51.535 INFO:tasks.workunit.client.0.smithi077.stdout:0/69: mknod d0/d19/c1f 0 2024-04-26T04:52:51.536 INFO:tasks.workunit.client.0.smithi077.stdout:0/70: write d0/db/d11/f14 [876226,107598] 0 2024-04-26T04:52:51.537 INFO:tasks.workunit.client.0.smithi077.stdout:0/71: write d0/f4 [45717,54298] 0 2024-04-26T04:52:51.539 INFO:tasks.workunit.client.1.smithi078.stdout:2/352: symlink d8/d11/d18/d5d/d61/d1a/l7f 0 2024-04-26T04:52:51.540 INFO:tasks.workunit.client.0.smithi077.stdout:1/98: readlink d9/db/dd/l17 0 2024-04-26T04:52:51.545 INFO:tasks.workunit.client.0.smithi077.stdout:6/95: chown d8/l1b 764447185 1 2024-04-26T04:52:51.545 INFO:tasks.workunit.client.0.smithi077.stdout:2/69: dread f7 [0,4194304] 0 2024-04-26T04:52:51.547 INFO:tasks.workunit.client.0.smithi077.stdout:7/48: unlink da/fd 0 2024-04-26T04:52:51.548 INFO:tasks.workunit.client.0.smithi077.stdout:7/49: fsync f1 0 2024-04-26T04:52:51.548 INFO:tasks.workunit.client.1.smithi078.stdout:9/345: dread dc/df/d27/f4f [0,4194304] 0 2024-04-26T04:52:51.549 INFO:tasks.workunit.client.1.smithi078.stdout:9/346: fdatasync dc/df/d4b/d3e/f5d 0 2024-04-26T04:52:51.549 INFO:tasks.workunit.client.0.smithi077.stdout:4/126: creat d4/f27 x:0 0 0 2024-04-26T04:52:51.550 INFO:tasks.workunit.client.1.smithi078.stdout:9/347: stat dc/df/d27 0 2024-04-26T04:52:51.550 INFO:tasks.workunit.client.0.smithi077.stdout:4/127: dread - d4/f27 zero size 2024-04-26T04:52:51.550 INFO:tasks.workunit.client.1.smithi078.stdout:9/348: write dc/df/f28 [4229863,100160] 0 2024-04-26T04:52:51.551 INFO:tasks.workunit.client.0.smithi077.stdout:4/128: stat d4/de/d15/d1f/d25 0 2024-04-26T04:52:51.551 INFO:tasks.workunit.client.0.smithi077.stdout:5/63: getdents d1/d8 0 2024-04-26T04:52:51.555 INFO:tasks.workunit.client.0.smithi077.stdout:5/64: dread d1/fe [0,4194304] 0 2024-04-26T04:52:51.555 INFO:tasks.workunit.client.1.smithi078.stdout:7/389: rename d9/da/d33/d4b/f75 to d9/da/d11/d13/d1a/d52/d5a/d41/d68/f7b 0 2024-04-26T04:52:51.556 INFO:tasks.workunit.client.1.smithi078.stdout:7/390: write d9/da/d11/d13/f17 [941086,39513] 0 2024-04-26T04:52:51.556 INFO:tasks.workunit.client.0.smithi077.stdout:0/72: unlink d0/db/f1b 0 2024-04-26T04:52:51.558 INFO:tasks.workunit.client.1.smithi078.stdout:0/453: symlink d2/d9/d60/d7f/l94 0 2024-04-26T04:52:51.559 INFO:tasks.workunit.client.1.smithi078.stdout:0/454: dread d2/d9/d60/d6a/f6d [0,4194304] 0 2024-04-26T04:52:51.560 INFO:tasks.workunit.client.0.smithi077.stdout:9/87: link f3 d9/dd/d1b/f23 0 2024-04-26T04:52:51.562 INFO:tasks.workunit.client.1.smithi078.stdout:6/450: link d2/d1a/l5d d2/de/d27/d58/l87 0 2024-04-26T04:52:51.562 INFO:tasks.workunit.client.0.smithi077.stdout:6/96: creat d8/d1c/f23 x:0 0 0 2024-04-26T04:52:51.562 INFO:tasks.workunit.client.0.smithi077.stdout:9/88: chown d9/dd/l20 78926 1 2024-04-26T04:52:51.562 INFO:tasks.workunit.client.0.smithi077.stdout:5/65: dwrite d1/d8/f10 [0,4194304] 0 2024-04-26T04:52:51.563 INFO:tasks.workunit.client.1.smithi078.stdout:6/451: chown d2/de/f82 29 1 2024-04-26T04:52:51.564 INFO:tasks.workunit.client.0.smithi077.stdout:2/70: fsync d9/ff 0 2024-04-26T04:52:51.565 INFO:tasks.workunit.client.1.smithi078.stdout:8/369: creat d0/d19/d25/f72 x:0 0 0 2024-04-26T04:52:51.566 INFO:tasks.workunit.client.1.smithi078.stdout:1/440: symlink d7/d39/d4c/l94 0 2024-04-26T04:52:51.566 INFO:tasks.workunit.client.1.smithi078.stdout:1/441: dread - d7/f8a zero size 2024-04-26T04:52:51.567 INFO:tasks.workunit.client.0.smithi077.stdout:7/50: rename f8 to da/ff 0 2024-04-26T04:52:51.568 INFO:tasks.workunit.client.1.smithi078.stdout:3/366: mkdir d16/d7e 0 2024-04-26T04:52:51.571 INFO:tasks.workunit.client.1.smithi078.stdout:5/294: rmdir d7/d11/d5d 0 2024-04-26T04:52:51.573 INFO:tasks.workunit.client.1.smithi078.stdout:9/349: rmdir dc/df/d4b 39 2024-04-26T04:52:51.573 INFO:tasks.workunit.client.1.smithi078.stdout:3/367: dread d16/d26/d2e/f43 [0,4194304] 0 2024-04-26T04:52:51.574 INFO:tasks.workunit.client.1.smithi078.stdout:3/368: write d16/f57 [1843554,27318] 0 2024-04-26T04:52:51.576 INFO:tasks.workunit.client.1.smithi078.stdout:2/353: rename d8/d11/d18/d5d/d61/f24 to d8/d11/d6e/f80 0 2024-04-26T04:52:51.577 INFO:tasks.workunit.client.1.smithi078.stdout:2/354: readlink d8/d11/d18/d5d/d61/d1a/d34/l79 0 2024-04-26T04:52:51.578 INFO:tasks.workunit.client.1.smithi078.stdout:2/355: dread - d8/d11/d6e/f7b zero size 2024-04-26T04:52:51.581 INFO:tasks.workunit.client.1.smithi078.stdout:7/391: creat d9/da/d11/d13/d50/f7c x:0 0 0 2024-04-26T04:52:51.583 INFO:tasks.workunit.client.1.smithi078.stdout:2/356: dwrite d8/d11/f54 [0,4194304] 0 2024-04-26T04:52:51.584 INFO:tasks.workunit.client.1.smithi078.stdout:0/455: creat d2/d9/d60/d6a/d8f/f95 x:0 0 0 2024-04-26T04:52:51.587 INFO:tasks.workunit.client.1.smithi078.stdout:0/456: write d2/d9/d18/f13 [1224314,101334] 0 2024-04-26T04:52:51.589 INFO:tasks.workunit.client.1.smithi078.stdout:7/392: dwrite d9/da/d11/d13/d50/f59 [0,4194304] 0 2024-04-26T04:52:51.589 INFO:tasks.workunit.client.0.smithi077.stdout:7/51: rename l9 to da/l10 0 2024-04-26T04:52:51.592 INFO:tasks.workunit.client.1.smithi078.stdout:7/393: write d9/da/d11/d13/d1a/f1c [3528278,115071] 0 2024-04-26T04:52:51.592 INFO:tasks.workunit.client.1.smithi078.stdout:5/295: dread d7/d18/f1a [0,4194304] 0 2024-04-26T04:52:51.593 INFO:tasks.workunit.client.1.smithi078.stdout:1/442: creat d7/d39/d4c/d3d/f95 x:0 0 0 2024-04-26T04:52:51.594 INFO:tasks.workunit.client.1.smithi078.stdout:1/443: read - d7/d39/d4c/d3d/f95 zero size 2024-04-26T04:52:51.596 INFO:tasks.workunit.client.1.smithi078.stdout:1/444: fsync d7/d39/dd/d20/f7b 0 2024-04-26T04:52:51.596 INFO:tasks.workunit.client.0.smithi077.stdout:4/129: creat d4/de/d15/d1f/d25/f28 x:0 0 0 2024-04-26T04:52:51.596 INFO:tasks.workunit.client.0.smithi077.stdout:8/103: sync 2024-04-26T04:52:51.598 INFO:tasks.workunit.client.1.smithi078.stdout:1/445: truncate d7/d1e/f7f 640710 0 2024-04-26T04:52:51.600 INFO:tasks.workunit.client.1.smithi078.stdout:7/394: dwrite d9/da/d11/d13/d50/f7c [0,4194304] 0 2024-04-26T04:52:51.600 INFO:tasks.workunit.client.1.smithi078.stdout:9/350: symlink dc/df/d17/l67 0 2024-04-26T04:52:51.601 INFO:tasks.workunit.client.0.smithi077.stdout:0/73: creat d0/d19/d1e/f20 x:0 0 0 2024-04-26T04:52:51.601 INFO:tasks.workunit.client.1.smithi078.stdout:9/351: stat dc/df/f59 0 2024-04-26T04:52:51.609 INFO:tasks.workunit.client.0.smithi077.stdout:6/97: link d8/fe d8/d1c/f24 0 2024-04-26T04:52:51.612 INFO:tasks.workunit.client.0.smithi077.stdout:2/71: mkdir d9/d15 0 2024-04-26T04:52:51.614 INFO:tasks.workunit.client.0.smithi077.stdout:9/89: dwrite d9/f17 [0,4194304] 0 2024-04-26T04:52:51.619 INFO:tasks.workunit.client.0.smithi077.stdout:1/99: getdents d9 0 2024-04-26T04:52:51.620 INFO:tasks.workunit.client.0.smithi077.stdout:1/100: fdatasync f7 0 2024-04-26T04:52:51.622 INFO:tasks.workunit.client.0.smithi077.stdout:0/74: mknod d0/d19/c21 0 2024-04-26T04:52:51.624 INFO:tasks.workunit.client.0.smithi077.stdout:0/75: write d0/db/d11/f14 [1185849,71923] 0 2024-04-26T04:52:51.625 INFO:tasks.workunit.client.1.smithi078.stdout:3/369: dwrite fc [0,4194304] 0 2024-04-26T04:52:51.627 INFO:tasks.workunit.client.0.smithi077.stdout:0/76: write d0/db/d11/f14 [1760791,65192] 0 2024-04-26T04:52:51.629 INFO:tasks.workunit.client.0.smithi077.stdout:5/66: dwrite d1/f5 [0,4194304] 0 2024-04-26T04:52:51.631 INFO:tasks.workunit.client.0.smithi077.stdout:6/98: rename d8/f18 to d8/d1c/f25 0 2024-04-26T04:52:51.635 INFO:tasks.workunit.client.1.smithi078.stdout:1/446: dread d7/f13 [0,4194304] 0 2024-04-26T04:52:51.640 INFO:tasks.workunit.client.0.smithi077.stdout:8/104: mknod d4/df/d22/c27 0 2024-04-26T04:52:51.641 INFO:tasks.workunit.client.0.smithi077.stdout:8/105: readlink d4/df/l11 0 2024-04-26T04:52:51.641 INFO:tasks.workunit.client.1.smithi078.stdout:5/296: dread f5 [0,4194304] 0 2024-04-26T04:52:51.641 INFO:tasks.workunit.client.1.smithi078.stdout:4/411: link d0/d25/d20/d31/d33/d4c/d8e/l97 d0/de/d22/d26/d2e/d85/l9f 0 2024-04-26T04:52:51.642 INFO:tasks.workunit.client.0.smithi077.stdout:8/106: fdatasync d4/df/f25 0 2024-04-26T04:52:51.642 INFO:tasks.workunit.client.1.smithi078.stdout:5/297: stat d7/d21/d2f/l3c 0 2024-04-26T04:52:51.643 INFO:tasks.workunit.client.1.smithi078.stdout:2/357: link d8/d11/d18/d5d/f7e d8/d11/d14/d5b/f81 0 2024-04-26T04:52:51.648 INFO:tasks.workunit.client.0.smithi077.stdout:0/77: creat d0/db/f22 x:0 0 0 2024-04-26T04:52:51.651 INFO:tasks.workunit.client.1.smithi078.stdout:4/412: mknod d0/de/d22/d27/d56/ca0 0 2024-04-26T04:52:51.652 INFO:tasks.workunit.client.1.smithi078.stdout:4/413: dread - d0/de/d22/d27/f96 zero size 2024-04-26T04:52:51.652 INFO:tasks.workunit.client.0.smithi077.stdout:7/52: truncate da/fb 666366 0 2024-04-26T04:52:51.652 INFO:tasks.workunit.client.1.smithi078.stdout:4/414: chown d0/de/d22/l5f 2248448 1 2024-04-26T04:52:51.653 INFO:tasks.workunit.client.0.smithi077.stdout:5/67: fsync d1/fe 0 2024-04-26T04:52:51.653 INFO:tasks.workunit.client.0.smithi077.stdout:5/68: dread - d1/d8/fc zero size 2024-04-26T04:52:51.654 INFO:tasks.workunit.client.1.smithi078.stdout:5/298: fsync d7/da/f24 0 2024-04-26T04:52:51.654 INFO:tasks.workunit.client.0.smithi077.stdout:5/69: write d1/f4 [2321356,10346] 0 2024-04-26T04:52:51.655 INFO:tasks.workunit.client.1.smithi078.stdout:5/299: write d7/d18/f2e [2365141,42890] 0 2024-04-26T04:52:51.656 INFO:tasks.workunit.client.0.smithi077.stdout:9/90: rename d9/l19 to d9/dd/l24 0 2024-04-26T04:52:51.658 INFO:tasks.workunit.client.0.smithi077.stdout:5/70: dread d1/d8/f10 [0,4194304] 0 2024-04-26T04:52:51.659 INFO:tasks.workunit.client.0.smithi077.stdout:6/99: mknod d8/c26 0 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.0.smithi077.stdout:5/71: chown d1/f4 781974696 1 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.0.smithi077.stdout:9/91: dread f3 [4194304,4194304] 0 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.0.smithi077.stdout:9/92: dread - f7 zero size 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.1.smithi078.stdout:8/370: getdents d0/d19/d52/d35/d44/d6b 0 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.1.smithi078.stdout:5/300: dwrite d7/d21/f42 [0,4194304] 0 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.1.smithi078.stdout:3/370: write d16/d26/d2b/d31/f34 [261979,103980] 0 2024-04-26T04:52:51.662 INFO:tasks.workunit.client.1.smithi078.stdout:8/371: write d0/d19/d52/d35/f5f [144856,72260] 0 2024-04-26T04:52:51.664 INFO:tasks.workunit.client.1.smithi078.stdout:1/447: symlink d7/d39/l96 0 2024-04-26T04:52:51.670 INFO:tasks.workunit.client.1.smithi078.stdout:4/415: fsync d0/fc 0 2024-04-26T04:52:51.671 INFO:tasks.workunit.client.1.smithi078.stdout:4/416: chown d0/de/d22/d27/d56/d5a/c9d 3172 1 2024-04-26T04:52:51.672 INFO:tasks.workunit.client.1.smithi078.stdout:1/448: dread d7/d39/dd/d20/f45 [0,4194304] 0 2024-04-26T04:52:51.677 INFO:tasks.workunit.client.1.smithi078.stdout:2/358: symlink d8/d11/d14/d69/l82 0 2024-04-26T04:52:51.678 INFO:tasks.workunit.client.0.smithi077.stdout:2/72: symlink d9/l16 0 2024-04-26T04:52:51.679 INFO:tasks.workunit.client.1.smithi078.stdout:1/449: dwrite d7/d39/db/d37/f6f [0,4194304] 0 2024-04-26T04:52:51.685 INFO:tasks.workunit.client.1.smithi078.stdout:3/371: dread d16/d26/d2e/f51 [0,4194304] 0 2024-04-26T04:52:51.685 INFO:tasks.workunit.client.0.smithi077.stdout:5/72: readlink d1/la 0 2024-04-26T04:52:51.686 INFO:tasks.workunit.client.0.smithi077.stdout:9/93: dwrite d9/f17 [4194304,4194304] 0 2024-04-26T04:52:51.686 INFO:tasks.workunit.client.1.smithi078.stdout:8/372: fdatasync d0/d19/d52/f69 0 2024-04-26T04:52:51.687 INFO:tasks.workunit.client.1.smithi078.stdout:8/373: chown d0 1158 1 2024-04-26T04:52:51.687 INFO:tasks.workunit.client.1.smithi078.stdout:8/374: readlink d0/l6f 0 2024-04-26T04:52:51.692 INFO:tasks.workunit.client.0.smithi077.stdout:4/130: link d4/d17/d18/l1a d4/de/d15/d1f/l29 0 2024-04-26T04:52:51.695 INFO:tasks.workunit.client.1.smithi078.stdout:4/417: dread d0/de/d22/d26/d2e/f52 [0,4194304] 0 2024-04-26T04:52:51.696 INFO:tasks.workunit.client.0.smithi077.stdout:1/101: link d9/f1b d9/db/f1c 0 2024-04-26T04:52:51.696 INFO:tasks.workunit.client.1.smithi078.stdout:4/418: stat d0/d25/d20/f2a 0 2024-04-26T04:52:51.697 INFO:tasks.workunit.client.0.smithi077.stdout:2/73: dread d9/fd [0,4194304] 0 2024-04-26T04:52:51.701 INFO:tasks.workunit.client.1.smithi078.stdout:1/450: symlink d7/d2e/l97 0 2024-04-26T04:52:51.701 INFO:tasks.workunit.client.1.smithi078.stdout:5/301: getdents d7/d21/d2f 0 2024-04-26T04:52:51.701 INFO:tasks.workunit.client.1.smithi078.stdout:1/451: readlink d7/d2e/l97 0 2024-04-26T04:52:51.702 INFO:tasks.workunit.client.1.smithi078.stdout:5/302: chown d7/da/f12 1116517396 1 2024-04-26T04:52:51.702 INFO:tasks.workunit.client.1.smithi078.stdout:1/452: write d7/d2e/f33 [4602600,92221] 0 2024-04-26T04:52:51.702 INFO:tasks.workunit.client.0.smithi077.stdout:1/102: dread d9/db/dd/f11 [0,4194304] 0 2024-04-26T04:52:51.703 INFO:tasks.workunit.client.0.smithi077.stdout:5/73: mkdir d1/d12/d16 0 2024-04-26T04:52:51.704 INFO:tasks.workunit.client.0.smithi077.stdout:1/103: write f7 [679567,73000] 0 2024-04-26T04:52:51.705 INFO:tasks.workunit.client.0.smithi077.stdout:9/94: symlink d9/dd/d1b/l25 0 2024-04-26T04:52:51.705 INFO:tasks.workunit.client.0.smithi077.stdout:2/74: dwrite d9/fd [0,4194304] 0 2024-04-26T04:52:51.706 INFO:tasks.workunit.client.0.smithi077.stdout:4/131: mkdir d4/de/d15/d1f/d2a 0 2024-04-26T04:52:51.707 INFO:tasks.workunit.client.1.smithi078.stdout:8/375: mkdir d0/d19/d52/d35/d73 0 2024-04-26T04:52:51.708 INFO:tasks.workunit.client.0.smithi077.stdout:2/75: write d9/ff [303493,122614] 0 2024-04-26T04:52:51.708 INFO:tasks.workunit.client.0.smithi077.stdout:4/132: write d4/f27 [328552,28984] 0 2024-04-26T04:52:51.709 INFO:tasks.workunit.client.1.smithi078.stdout:1/453: dwrite d7/d2e/f67 [0,4194304] 0 2024-04-26T04:52:51.709 INFO:tasks.workunit.client.1.smithi078.stdout:0/457: sync 2024-04-26T04:52:51.709 INFO:tasks.workunit.client.1.smithi078.stdout:7/395: sync 2024-04-26T04:52:51.709 INFO:tasks.workunit.client.1.smithi078.stdout:6/452: sync 2024-04-26T04:52:51.711 INFO:tasks.workunit.client.1.smithi078.stdout:7/396: chown d9/da/d33/d4b/d4c 975 1 2024-04-26T04:52:51.712 INFO:tasks.workunit.client.1.smithi078.stdout:1/454: chown d7/d39/db/d37/f6f 469304244 1 2024-04-26T04:52:51.713 INFO:tasks.workunit.client.1.smithi078.stdout:9/352: getdents dc/df/d4b/d22/d2a/d57 0 2024-04-26T04:52:51.718 INFO:tasks.workunit.client.0.smithi077.stdout:8/107: symlink d4/d17/l28 0 2024-04-26T04:52:51.718 INFO:tasks.workunit.client.0.smithi077.stdout:4/133: dwrite d4/de/d15/d1f/d25/f28 [0,4194304] 0 2024-04-26T04:52:51.721 INFO:tasks.workunit.client.1.smithi078.stdout:4/419: creat d0/d25/d20/d31/d33/d44/d69/fa1 x:0 0 0 2024-04-26T04:52:51.727 INFO:tasks.workunit.client.0.smithi077.stdout:9/95: stat d9/dd/l24 0 2024-04-26T04:52:51.729 INFO:tasks.workunit.client.1.smithi078.stdout:5/303: rename d7/d29 to d7/d48/d57/d65 0 2024-04-26T04:52:51.731 INFO:tasks.workunit.client.1.smithi078.stdout:5/304: write d7/d48/d57/d65/f59 [824457,82428] 0 2024-04-26T04:52:51.731 INFO:tasks.workunit.client.0.smithi077.stdout:2/76: write d9/f10 [160526,55143] 0 2024-04-26T04:52:51.736 INFO:tasks.workunit.client.0.smithi077.stdout:6/100: getdents d8/d1c 0 2024-04-26T04:52:51.739 INFO:tasks.workunit.client.1.smithi078.stdout:6/453: symlink d2/d1a/d5/d2a/d4d/l88 0 2024-04-26T04:52:51.741 INFO:tasks.workunit.client.0.smithi077.stdout:4/134: creat d4/de/f2b x:0 0 0 2024-04-26T04:52:51.741 INFO:tasks.workunit.client.0.smithi077.stdout:8/108: dread d4/df/f1f [0,4194304] 0 2024-04-26T04:52:51.741 INFO:tasks.workunit.client.1.smithi078.stdout:7/397: readlink d9/da/d11/d13/d1a/d2d/l4a 0 2024-04-26T04:52:51.742 INFO:tasks.workunit.client.0.smithi077.stdout:1/104: write d9/f1b [843274,63483] 0 2024-04-26T04:52:51.744 INFO:tasks.workunit.client.0.smithi077.stdout:5/74: creat d1/d12/d16/f17 x:0 0 0 2024-04-26T04:52:51.746 INFO:tasks.workunit.client.1.smithi078.stdout:4/420: rmdir d0/de/d22/d27/d56 39 2024-04-26T04:52:51.747 INFO:tasks.workunit.client.1.smithi078.stdout:9/353: dwrite dc/df/f2f [0,4194304] 0 2024-04-26T04:52:51.749 INFO:tasks.workunit.client.1.smithi078.stdout:9/354: write dc/df/d4b/f1c [2510485,24657] 0 2024-04-26T04:52:51.752 INFO:tasks.workunit.client.0.smithi077.stdout:6/101: rmdir d8 39 2024-04-26T04:52:51.757 INFO:tasks.workunit.client.0.smithi077.stdout:9/96: write f3 [3512928,102556] 0 2024-04-26T04:52:51.758 INFO:tasks.workunit.client.0.smithi077.stdout:7/53: dwrite da/fb [0,4194304] 0 2024-04-26T04:52:51.758 INFO:tasks.workunit.client.1.smithi078.stdout:3/372: creat d16/d26/f7f x:0 0 0 2024-04-26T04:52:51.760 INFO:tasks.workunit.client.0.smithi077.stdout:1/105: creat d9/f1d x:0 0 0 2024-04-26T04:52:51.761 INFO:tasks.workunit.client.1.smithi078.stdout:0/458: dread d2/d9/f1a [0,4194304] 0 2024-04-26T04:52:51.762 INFO:tasks.workunit.client.0.smithi077.stdout:5/75: rmdir d1/d12/d16 39 2024-04-26T04:52:51.762 INFO:tasks.workunit.client.1.smithi078.stdout:0/459: dread - d2/d9/d78/d84/f88 zero size 2024-04-26T04:52:51.764 INFO:tasks.workunit.client.0.smithi077.stdout:9/97: dread d9/f11 [0,4194304] 0 2024-04-26T04:52:51.765 INFO:tasks.workunit.client.1.smithi078.stdout:6/454: rmdir d2/d1a/d41 39 2024-04-26T04:52:51.765 INFO:tasks.workunit.client.0.smithi077.stdout:7/54: dread f1 [4194304,4194304] 0 2024-04-26T04:52:51.766 INFO:tasks.workunit.client.0.smithi077.stdout:9/98: write d9/dd/d1b/f23 [47257,16854] 0 2024-04-26T04:52:51.767 INFO:tasks.workunit.client.1.smithi078.stdout:7/398: symlink d9/da/d11/d13/d50/l7d 0 2024-04-26T04:52:51.767 INFO:tasks.workunit.client.0.smithi077.stdout:1/106: dread d9/db/f1c [0,4194304] 0 2024-04-26T04:52:51.769 INFO:tasks.workunit.client.1.smithi078.stdout:4/421: mknod d0/d1d/ca2 0 2024-04-26T04:52:51.769 INFO:tasks.workunit.client.1.smithi078.stdout:7/399: read d9/f10 [3627058,13435] 0 2024-04-26T04:52:51.771 INFO:tasks.workunit.client.1.smithi078.stdout:7/400: read d9/da/d11/f45 [206700,1329] 0 2024-04-26T04:52:51.773 INFO:tasks.workunit.client.1.smithi078.stdout:5/305: dwrite f5 [0,4194304] 0 2024-04-26T04:52:51.773 INFO:tasks.workunit.client.1.smithi078.stdout:5/306: stat d7/l17 0 2024-04-26T04:52:51.849 INFO:tasks.workunit.client.1.smithi078.stdout:9/355: sync 2024-04-26T04:52:51.865 INFO:tasks.workunit.client.1.smithi078.stdout:2/359: getdents d8/d11/d18/d5d 0 2024-04-26T04:52:51.868 INFO:tasks.workunit.client.0.smithi077.stdout:5/76: unlink d1/f4 0 2024-04-26T04:52:51.868 INFO:tasks.workunit.client.1.smithi078.stdout:3/373: chown d16/l24 0 1 2024-04-26T04:52:51.871 INFO:tasks.workunit.client.1.smithi078.stdout:0/460: mkdir d2/d39/d96 0 2024-04-26T04:52:51.872 INFO:tasks.workunit.client.0.smithi077.stdout:5/77: dread d1/d8/f10 [0,4194304] 0 2024-04-26T04:52:51.872 INFO:tasks.workunit.client.0.smithi077.stdout:7/55: unlink f1 0 2024-04-26T04:52:51.878 INFO:tasks.workunit.client.0.smithi077.stdout:6/102: creat d8/f27 x:0 0 0 2024-04-26T04:52:51.881 INFO:tasks.workunit.client.1.smithi078.stdout:5/307: creat d7/d11/f66 x:0 0 0 2024-04-26T04:52:51.882 INFO:tasks.workunit.client.0.smithi077.stdout:6/103: dread f3 [0,4194304] 0 2024-04-26T04:52:51.882 INFO:tasks.workunit.client.0.smithi077.stdout:4/135: link d4/cc d4/de/d15/d1f/d25/c2c 0 2024-04-26T04:52:51.882 INFO:tasks.workunit.client.0.smithi077.stdout:4/136: dread - d4/de/f26 zero size 2024-04-26T04:52:51.884 INFO:tasks.workunit.client.1.smithi078.stdout:6/455: sync 2024-04-26T04:52:51.884 INFO:tasks.workunit.client.1.smithi078.stdout:9/356: symlink dc/df/d4b/d22/d2a/l68 0 2024-04-26T04:52:51.886 INFO:tasks.workunit.client.1.smithi078.stdout:4/422: dread d0/de/d22/d27/d56/f60 [0,4194304] 0 2024-04-26T04:52:51.886 INFO:tasks.workunit.client.1.smithi078.stdout:2/360: symlink d8/d3e/l83 0 2024-04-26T04:52:51.887 INFO:tasks.workunit.client.1.smithi078.stdout:5/308: dwrite d7/d18/d1c/d5a/f64 [0,4194304] 0 2024-04-26T04:52:51.888 INFO:tasks.workunit.client.1.smithi078.stdout:1/455: rename d7/d39/d17/c8c to d7/d39/db/c98 0 2024-04-26T04:52:51.888 INFO:tasks.workunit.client.0.smithi077.stdout:2/77: link f5 d9/f17 0 2024-04-26T04:52:51.889 INFO:tasks.workunit.client.1.smithi078.stdout:1/456: write d7/fc [4986580,50257] 0 2024-04-26T04:52:51.891 INFO:tasks.workunit.client.0.smithi077.stdout:7/56: creat da/f11 x:0 0 0 2024-04-26T04:52:51.891 INFO:tasks.workunit.client.1.smithi078.stdout:1/457: truncate d7/d39/db/d47/f90 250653 0 2024-04-26T04:52:51.896 INFO:tasks.workunit.client.0.smithi077.stdout:8/109: rename d4/lb to d4/l29 0 2024-04-26T04:52:51.899 INFO:tasks.workunit.client.0.smithi077.stdout:6/104: creat d8/d1c/f28 x:0 0 0 2024-04-26T04:52:51.902 INFO:tasks.workunit.client.0.smithi077.stdout:4/137: rename d4/de/d15/d1f/d25/f28 to d4/de/d19/f2d 0 2024-04-26T04:52:51.905 INFO:tasks.workunit.client.0.smithi077.stdout:1/107: write f2 [2693644,128510] 0 2024-04-26T04:52:51.905 INFO:tasks.workunit.client.0.smithi077.stdout:1/108: readlink d9/db/dd/l17 0 2024-04-26T04:52:51.906 INFO:tasks.workunit.client.0.smithi077.stdout:1/109: chown d9/db/f1c 13563 1 2024-04-26T04:52:51.907 INFO:tasks.workunit.client.0.smithi077.stdout:7/57: symlink da/l12 0 2024-04-26T04:52:51.909 INFO:tasks.workunit.client.0.smithi077.stdout:9/99: creat d9/f26 x:0 0 0 2024-04-26T04:52:51.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/110: truncate d4/df/f15 627571 0 2024-04-26T04:52:51.913 INFO:tasks.workunit.client.0.smithi077.stdout:4/138: readlink d4/l12 0 2024-04-26T04:52:51.913 INFO:tasks.workunit.client.0.smithi077.stdout:8/111: write d4/df/d22/f24 [161037,44545] 0 2024-04-26T04:52:51.918 INFO:tasks.workunit.client.0.smithi077.stdout:7/58: dwrite da/f11 [0,4194304] 0 2024-04-26T04:52:51.918 INFO:tasks.workunit.client.0.smithi077.stdout:5/78: getdents d1 0 2024-04-26T04:52:51.918 INFO:tasks.workunit.client.0.smithi077.stdout:9/100: dread d9/f11 [0,4194304] 0 2024-04-26T04:52:51.919 INFO:tasks.workunit.client.0.smithi077.stdout:9/101: read f3 [1719525,119138] 0 2024-04-26T04:52:51.920 INFO:tasks.workunit.client.0.smithi077.stdout:9/102: fsync d9/f26 0 2024-04-26T04:52:51.921 INFO:tasks.workunit.client.0.smithi077.stdout:6/105: write d8/d1c/f24 [3015415,66310] 0 2024-04-26T04:52:51.922 INFO:tasks.workunit.client.0.smithi077.stdout:1/110: write d9/db/f1c [1324006,120307] 0 2024-04-26T04:52:51.923 INFO:tasks.workunit.client.0.smithi077.stdout:6/106: write d8/fe [3932708,70616] 0 2024-04-26T04:52:51.923 INFO:tasks.workunit.client.0.smithi077.stdout:6/107: readlink d8/l1b 0 2024-04-26T04:52:51.924 INFO:tasks.workunit.client.0.smithi077.stdout:1/111: write f6 [3057173,106966] 0 2024-04-26T04:52:51.924 INFO:tasks.workunit.client.0.smithi077.stdout:6/108: dread - d8/f17 zero size 2024-04-26T04:52:51.926 INFO:tasks.workunit.client.0.smithi077.stdout:7/59: mkdir da/d13 0 2024-04-26T04:52:51.928 INFO:tasks.workunit.client.0.smithi077.stdout:9/103: fdatasync d9/f11 0 2024-04-26T04:52:51.930 INFO:tasks.workunit.client.0.smithi077.stdout:8/112: symlink d4/l2a 0 2024-04-26T04:52:51.931 INFO:tasks.workunit.client.1.smithi078.stdout:6/456: symlink d2/de/d27/d58/l89 0 2024-04-26T04:52:51.931 INFO:tasks.workunit.client.0.smithi077.stdout:5/79: write d1/fe [690597,23427] 0 2024-04-26T04:52:51.931 INFO:tasks.workunit.client.1.smithi078.stdout:9/357: mkdir dc/df/d4b/d22/d2a/d57/d69 0 2024-04-26T04:52:51.932 INFO:tasks.workunit.client.0.smithi077.stdout:2/78: getdents d9/d11 0 2024-04-26T04:52:51.932 INFO:tasks.workunit.client.1.smithi078.stdout:5/309: fsync d7/da/f10 0 2024-04-26T04:52:51.933 INFO:tasks.workunit.client.0.smithi077.stdout:6/109: fsync d8/f17 0 2024-04-26T04:52:51.935 INFO:tasks.workunit.client.0.smithi077.stdout:1/112: mknod d9/db/dd/c1e 0 2024-04-26T04:52:51.935 INFO:tasks.workunit.client.1.smithi078.stdout:5/310: fdatasync d7/d18/d32/d37/f41 0 2024-04-26T04:52:51.936 INFO:tasks.workunit.client.0.smithi077.stdout:7/60: creat da/f14 x:0 0 0 2024-04-26T04:52:51.936 INFO:tasks.workunit.client.1.smithi078.stdout:1/458: dwrite d7/d39/d4c/d3d/f95 [0,4194304] 0 2024-04-26T04:52:51.937 INFO:tasks.workunit.client.0.smithi077.stdout:7/61: chown da/l12 2816990 1 2024-04-26T04:52:51.938 INFO:tasks.workunit.client.1.smithi078.stdout:8/376: rename d0/d19/d25/d32/f4b to d0/d19/d52/d5c/f74 0 2024-04-26T04:52:51.938 INFO:tasks.workunit.client.0.smithi077.stdout:7/62: write da/f11 [2558413,54284] 0 2024-04-26T04:52:51.939 INFO:tasks.workunit.client.1.smithi078.stdout:8/377: readlink d0/l30 0 2024-04-26T04:52:51.941 INFO:tasks.workunit.client.1.smithi078.stdout:8/378: chown d0/d19/d52/c12 32 1 2024-04-26T04:52:51.942 INFO:tasks.workunit.client.0.smithi077.stdout:1/113: dwrite f2 [0,4194304] 0 2024-04-26T04:52:51.942 INFO:tasks.workunit.client.1.smithi078.stdout:2/361: truncate d8/f2b 3187502 0 2024-04-26T04:52:51.942 INFO:tasks.workunit.client.0.smithi077.stdout:3/93: sync 2024-04-26T04:52:51.943 INFO:tasks.workunit.client.1.smithi078.stdout:2/362: stat d8/d11/d18 0 2024-04-26T04:52:51.943 INFO:tasks.workunit.client.1.smithi078.stdout:1/459: dwrite d7/f57 [0,4194304] 0 2024-04-26T04:52:51.944 INFO:tasks.workunit.client.1.smithi078.stdout:9/358: mknod dc/df/d4b/c6a 0 2024-04-26T04:52:51.947 INFO:tasks.workunit.client.1.smithi078.stdout:6/457: mknod d2/de/d27/d58/d5b/c8a 0 2024-04-26T04:52:51.949 INFO:tasks.workunit.client.0.smithi077.stdout:9/104: mknod d9/dd/c27 0 2024-04-26T04:52:51.949 INFO:tasks.workunit.client.1.smithi078.stdout:6/458: read d2/de/d27/f34 [3008555,34679] 0 2024-04-26T04:52:51.950 INFO:tasks.workunit.client.0.smithi077.stdout:1/114: dwrite f2 [0,4194304] 0 2024-04-26T04:52:51.957 INFO:tasks.workunit.client.0.smithi077.stdout:5/80: read - d1/d12/d16/f17 zero size 2024-04-26T04:52:51.957 INFO:tasks.workunit.client.0.smithi077.stdout:2/79: rename d9/l16 to d9/d11/l18 0 2024-04-26T04:52:51.959 INFO:tasks.workunit.client.0.smithi077.stdout:1/115: dwrite d9/db/fe [0,4194304] 0 2024-04-26T04:52:51.960 INFO:tasks.workunit.client.1.smithi078.stdout:3/374: getdents d16/d26/d2b/d31 0 2024-04-26T04:52:51.961 INFO:tasks.workunit.client.0.smithi077.stdout:5/81: write d1/d8/ff [741594,116004] 0 2024-04-26T04:52:51.962 INFO:tasks.workunit.client.0.smithi077.stdout:6/110: mknod d8/c29 0 2024-04-26T04:52:51.962 INFO:tasks.workunit.client.0.smithi077.stdout:6/111: stat d8/f22 0 2024-04-26T04:52:51.967 INFO:tasks.workunit.client.1.smithi078.stdout:2/363: chown d8/d11/d14/d5b/l63 13 1 2024-04-26T04:52:51.976 INFO:tasks.workunit.client.1.smithi078.stdout:0/461: rename d2/f26 to d2/d9/d31/d73/f97 0 2024-04-26T04:52:51.977 INFO:tasks.workunit.client.0.smithi077.stdout:3/94: creat d0/d7/dd/f21 x:0 0 0 2024-04-26T04:52:51.977 INFO:tasks.workunit.client.0.smithi077.stdout:8/113: dwrite d4/df/f16 [0,4194304] 0 2024-04-26T04:52:51.978 INFO:tasks.workunit.client.0.smithi077.stdout:3/95: dread - d0/f1b zero size 2024-04-26T04:52:51.979 INFO:tasks.workunit.client.0.smithi077.stdout:3/96: dread - d0/f1d zero size 2024-04-26T04:52:51.983 INFO:tasks.workunit.client.0.smithi077.stdout:4/139: getdents d4/de/d15/d1f/d25 0 2024-04-26T04:52:51.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/401: sync 2024-04-26T04:52:51.985 INFO:tasks.workunit.client.1.smithi078.stdout:7/402: readlink d9/da/d11/d13/d1a/d2d/l61 0 2024-04-26T04:52:51.986 INFO:tasks.workunit.client.1.smithi078.stdout:6/459: dread d2/d1a/f6 [0,4194304] 0 2024-04-26T04:52:51.987 INFO:tasks.workunit.client.0.smithi077.stdout:3/97: dwrite d0/f6 [4194304,4194304] 0 2024-04-26T04:52:51.989 INFO:tasks.workunit.client.0.smithi077.stdout:1/116: symlink d9/db/dd/l1f 0 2024-04-26T04:52:51.991 INFO:tasks.workunit.client.1.smithi078.stdout:3/375: creat d16/d26/d2b/d54/d56/f80 x:0 0 0 2024-04-26T04:52:51.991 INFO:tasks.workunit.client.1.smithi078.stdout:3/376: readlink d16/d26/d2b/l53 0 2024-04-26T04:52:51.992 INFO:tasks.workunit.client.1.smithi078.stdout:3/377: chown d16/d26/d2b 4 1 2024-04-26T04:52:51.992 INFO:tasks.workunit.client.1.smithi078.stdout:3/378: fdatasync d16/f22 0 2024-04-26T04:52:51.992 INFO:tasks.workunit.client.0.smithi077.stdout:5/82: unlink d1/c6 0 2024-04-26T04:52:51.993 INFO:tasks.workunit.client.1.smithi078.stdout:3/379: fsync d16/d3c/f5b 0 2024-04-26T04:52:51.994 INFO:tasks.workunit.client.1.smithi078.stdout:0/462: dread d2/f3f [0,4194304] 0 2024-04-26T04:52:51.994 INFO:tasks.workunit.client.0.smithi077.stdout:6/112: mkdir d8/d2a 0 2024-04-26T04:52:51.995 INFO:tasks.workunit.client.1.smithi078.stdout:0/463: truncate d2/d1e/f56 793848 0 2024-04-26T04:52:51.995 INFO:tasks.workunit.client.1.smithi078.stdout:0/464: fdatasync d2/f2e 0 2024-04-26T04:52:51.999 INFO:tasks.workunit.client.1.smithi078.stdout:5/311: getdents d7/d18 0 2024-04-26T04:52:52.001 INFO:tasks.workunit.client.1.smithi078.stdout:6/460: symlink d2/d1a/d5/d3a/d42/l8b 0 2024-04-26T04:52:52.004 INFO:tasks.workunit.client.0.smithi077.stdout:7/63: write da/ff [1655967,53534] 0 2024-04-26T04:52:52.005 INFO:tasks.workunit.client.0.smithi077.stdout:1/117: rmdir d9/db 39 2024-04-26T04:52:52.009 INFO:tasks.workunit.client.0.smithi077.stdout:1/118: dread f6 [0,4194304] 0 2024-04-26T04:52:52.010 INFO:tasks.workunit.client.0.smithi077.stdout:1/119: write f7 [96091,97512] 0 2024-04-26T04:52:52.013 INFO:tasks.workunit.client.0.smithi077.stdout:8/114: mkdir d4/d1a/d2b 0 2024-04-26T04:52:52.013 INFO:tasks.workunit.client.0.smithi077.stdout:8/115: stat f0 0 2024-04-26T04:52:52.014 INFO:tasks.workunit.client.1.smithi078.stdout:0/465: mknod d2/d9/d31/d73/c98 0 2024-04-26T04:52:52.015 INFO:tasks.workunit.client.1.smithi078.stdout:0/466: write d2/d9/d78/f76 [530685,4410] 0 2024-04-26T04:52:52.016 INFO:tasks.workunit.client.1.smithi078.stdout:7/403: mknod d9/da/d11/d13/c7e 0 2024-04-26T04:52:52.016 INFO:tasks.workunit.client.0.smithi077.stdout:2/80: creat d9/dc/f19 x:0 0 0 2024-04-26T04:52:52.017 INFO:tasks.workunit.client.1.smithi078.stdout:7/404: chown d9/da/d20/f6f 669332870 1 2024-04-26T04:52:52.017 INFO:tasks.workunit.client.1.smithi078.stdout:5/312: creat d7/d48/d57/d65/f67 x:0 0 0 2024-04-26T04:52:52.018 INFO:tasks.workunit.client.0.smithi077.stdout:7/64: mkdir da/d15 0 2024-04-26T04:52:52.019 INFO:tasks.workunit.client.1.smithi078.stdout:7/405: stat d9/da/d11/d13/d1a/d2d/l4a 0 2024-04-26T04:52:52.019 INFO:tasks.workunit.client.1.smithi078.stdout:3/380: mkdir d16/d3c/d81 0 2024-04-26T04:52:52.020 INFO:tasks.workunit.client.0.smithi077.stdout:5/83: chown d1/d8/c11 52170924 1 2024-04-26T04:52:52.021 INFO:tasks.workunit.client.1.smithi078.stdout:8/379: link d0/d19/d25/d36/c66 d0/d19/d25/c75 0 2024-04-26T04:52:52.021 INFO:tasks.workunit.client.1.smithi078.stdout:2/364: dwrite d8/d11/d18/d5d/d61/f4a [0,4194304] 0 2024-04-26T04:52:52.021 INFO:tasks.workunit.client.0.smithi077.stdout:6/113: symlink d8/d2a/l2b 0 2024-04-26T04:52:52.021 INFO:tasks.workunit.client.1.smithi078.stdout:8/380: readlink d0/l6f 0 2024-04-26T04:52:52.022 INFO:tasks.workunit.client.1.smithi078.stdout:0/467: symlink d2/d39/l99 0 2024-04-26T04:52:52.022 INFO:tasks.workunit.client.1.smithi078.stdout:7/406: fsync d9/da/d11/d13/d1a/d2d/f3d 0 2024-04-26T04:52:52.022 INFO:tasks.workunit.client.1.smithi078.stdout:2/365: read - d8/d3e/f6a zero size 2024-04-26T04:52:52.024 INFO:tasks.workunit.client.1.smithi078.stdout:0/468: readlink d2/d9/d31/d4a/l5b 0 2024-04-26T04:52:52.025 INFO:tasks.workunit.client.0.smithi077.stdout:6/114: dread d8/f13 [0,4194304] 0 2024-04-26T04:52:52.025 INFO:tasks.workunit.client.1.smithi078.stdout:6/461: mknod d2/d1a/d5/da/d36/d4b/c8c 0 2024-04-26T04:52:52.025 INFO:tasks.workunit.client.0.smithi077.stdout:6/115: readlink d8/l1b 0 2024-04-26T04:52:52.026 INFO:tasks.workunit.client.1.smithi078.stdout:2/366: chown d8/d11/d18/d5d/d61/f4a 3727487 1 2024-04-26T04:52:52.027 INFO:tasks.workunit.client.1.smithi078.stdout:8/381: truncate d0/d19/d52/d3c/f60 758735 0 2024-04-26T04:52:52.028 INFO:tasks.workunit.client.1.smithi078.stdout:5/313: dread d7/d48/d57/d65/d46/f4f [0,4194304] 0 2024-04-26T04:52:52.029 INFO:tasks.workunit.client.0.smithi077.stdout:4/140: rmdir d4/de/d15/d1f/d2a 0 2024-04-26T04:52:52.029 INFO:tasks.workunit.client.0.smithi077.stdout:8/116: dread d4/fd [0,4194304] 0 2024-04-26T04:52:52.029 INFO:tasks.workunit.client.1.smithi078.stdout:6/462: write d2/d1a/d5/d11/f4f [548639,108843] 0 2024-04-26T04:52:52.029 INFO:tasks.workunit.client.0.smithi077.stdout:8/117: chown f0 43537784 1 2024-04-26T04:52:52.030 INFO:tasks.workunit.client.0.smithi077.stdout:4/141: write f2 [4519241,70722] 0 2024-04-26T04:52:52.030 INFO:tasks.workunit.client.1.smithi078.stdout:3/381: dwrite d16/d26/f37 [0,4194304] 0 2024-04-26T04:52:52.032 INFO:tasks.workunit.client.1.smithi078.stdout:3/382: fdatasync d16/d26/d2b/d54/f6e 0 2024-04-26T04:52:52.038 INFO:tasks.workunit.client.1.smithi078.stdout:6/463: unlink d2/c30 0 2024-04-26T04:52:52.039 INFO:tasks.workunit.client.1.smithi078.stdout:5/314: write d7/d48/d57/d65/d46/f4f [1163918,111861] 0 2024-04-26T04:52:52.041 INFO:tasks.workunit.client.1.smithi078.stdout:3/383: rename d16/d26/d2e/d5f/d49/f7c to d16/d26/d2b/d54/d5a/f82 0 2024-04-26T04:52:52.042 INFO:tasks.workunit.client.1.smithi078.stdout:8/382: creat d0/d19/d25/d32/f76 x:0 0 0 2024-04-26T04:52:52.043 INFO:tasks.workunit.client.1.smithi078.stdout:5/315: write d7/f8 [8610808,93432] 0 2024-04-26T04:52:52.043 INFO:tasks.workunit.client.1.smithi078.stdout:8/383: dread - d0/d19/d25/f72 zero size 2024-04-26T04:52:52.044 INFO:tasks.workunit.client.1.smithi078.stdout:2/367: rmdir d8/d53/d5e 0 2024-04-26T04:52:52.044 INFO:tasks.workunit.client.1.smithi078.stdout:5/316: readlink d7/d11/l3f 0 2024-04-26T04:52:52.045 INFO:tasks.workunit.client.1.smithi078.stdout:2/368: write d8/f3a [1495686,128369] 0 2024-04-26T04:52:52.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/317: rename d7/da/f12 to d7/d48/d57/d65/d46/f68 0 2024-04-26T04:52:52.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/318: fdatasync d7/d21/f4e 0 2024-04-26T04:52:52.048 INFO:tasks.workunit.client.1.smithi078.stdout:2/369: creat d8/f84 x:0 0 0 2024-04-26T04:52:52.048 INFO:tasks.workunit.client.1.smithi078.stdout:5/319: readlink d7/d11/l3f 0 2024-04-26T04:52:52.048 INFO:tasks.workunit.client.1.smithi078.stdout:8/384: getdents d0/d19/d52 0 2024-04-26T04:52:52.050 INFO:tasks.workunit.client.1.smithi078.stdout:2/370: truncate d8/d53/f7a 493302 0 2024-04-26T04:52:52.050 INFO:tasks.workunit.client.1.smithi078.stdout:5/320: write d7/d18/d32/d37/f51 [47392,54171] 0 2024-04-26T04:52:52.053 INFO:tasks.workunit.client.1.smithi078.stdout:2/371: read d8/d11/d14/f5f [539513,102676] 0 2024-04-26T04:52:52.057 INFO:tasks.workunit.client.1.smithi078.stdout:8/385: dwrite d0/d19/d52/d35/d44/f59 [0,4194304] 0 2024-04-26T04:52:52.057 INFO:tasks.workunit.client.1.smithi078.stdout:5/321: dread d7/d18/f1a [0,4194304] 0 2024-04-26T04:52:52.059 INFO:tasks.workunit.client.1.smithi078.stdout:8/386: fdatasync d0/d19/d25/f72 0 2024-04-26T04:52:52.061 INFO:tasks.workunit.client.1.smithi078.stdout:8/387: readlink d0/d19/d52/d3c/l40 0 2024-04-26T04:52:52.062 INFO:tasks.workunit.client.1.smithi078.stdout:2/372: dwrite d8/f3a [0,4194304] 0 2024-04-26T04:52:52.064 INFO:tasks.workunit.client.1.smithi078.stdout:3/384: dread d16/d26/f62 [0,4194304] 0 2024-04-26T04:52:52.064 INFO:tasks.workunit.client.1.smithi078.stdout:8/388: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:52.064 INFO:tasks.workunit.client.1.smithi078.stdout:3/385: fdatasync d16/f3f 0 2024-04-26T04:52:52.065 INFO:tasks.workunit.client.1.smithi078.stdout:2/373: mknod d8/d11/d18/d5d/c85 0 2024-04-26T04:52:52.066 INFO:tasks.workunit.client.1.smithi078.stdout:8/389: chown d0/d19/d52/d3c/c45 43 1 2024-04-26T04:52:52.069 INFO:tasks.workunit.client.1.smithi078.stdout:3/386: rename d16/d26/d2b/d54/f6e to d16/d26/d2e/d74/f83 0 2024-04-26T04:52:52.069 INFO:tasks.workunit.client.1.smithi078.stdout:8/390: symlink d0/d19/d52/d35/d73/l77 0 2024-04-26T04:52:52.073 INFO:tasks.workunit.client.1.smithi078.stdout:4/423: sync 2024-04-26T04:52:52.074 INFO:tasks.workunit.client.1.smithi078.stdout:3/387: dwrite d16/d26/d2b/d31/f3d [0,4194304] 0 2024-04-26T04:52:52.075 INFO:tasks.workunit.client.0.smithi077.stdout:7/65: creat da/f16 x:0 0 0 2024-04-26T04:52:52.076 INFO:tasks.workunit.client.0.smithi077.stdout:4/142: write d4/f8 [3075293,74271] 0 2024-04-26T04:52:52.077 INFO:tasks.workunit.client.1.smithi078.stdout:4/424: creat d0/d25/d20/d31/d33/fa3 x:0 0 0 2024-04-26T04:52:52.077 INFO:tasks.workunit.client.0.smithi077.stdout:7/66: dread - da/f14 zero size 2024-04-26T04:52:52.078 INFO:tasks.workunit.client.0.smithi077.stdout:8/118: write d4/df/d22/f24 [421958,7773] 0 2024-04-26T04:52:52.078 INFO:tasks.workunit.client.0.smithi077.stdout:6/116: mkdir d8/d2c 0 2024-04-26T04:52:52.080 INFO:tasks.workunit.client.0.smithi077.stdout:7/67: write da/f14 [536012,71045] 0 2024-04-26T04:52:52.080 INFO:tasks.workunit.client.1.smithi078.stdout:3/388: creat d16/d26/d2b/d31/f84 x:0 0 0 2024-04-26T04:52:52.081 INFO:tasks.workunit.client.0.smithi077.stdout:2/81: fdatasync d9/f12 0 2024-04-26T04:52:52.082 INFO:tasks.workunit.client.0.smithi077.stdout:5/84: mkdir d1/d12/d16/d18 0 2024-04-26T04:52:52.083 INFO:tasks.workunit.client.1.smithi078.stdout:4/425: write d0/d25/d20/d31/f6b [200228,88070] 0 2024-04-26T04:52:52.085 INFO:tasks.workunit.client.0.smithi077.stdout:1/120: creat d9/f20 x:0 0 0 2024-04-26T04:52:52.087 INFO:tasks.workunit.client.0.smithi077.stdout:2/82: dread d9/d11/f13 [0,4194304] 0 2024-04-26T04:52:52.090 INFO:tasks.workunit.client.1.smithi078.stdout:3/389: mknod d16/d7e/c85 0 2024-04-26T04:52:52.090 INFO:tasks.workunit.client.1.smithi078.stdout:3/390: dread - d16/d26/d2b/d54/f6c zero size 2024-04-26T04:52:52.093 INFO:tasks.workunit.client.0.smithi077.stdout:4/143: rename d4/d17/d18/l1e to d4/de/d19/l2e 0 2024-04-26T04:52:52.095 INFO:tasks.workunit.client.0.smithi077.stdout:8/119: dwrite d4/f13 [0,4194304] 0 2024-04-26T04:52:52.095 INFO:tasks.workunit.client.0.smithi077.stdout:4/144: dread - d4/de/f2b zero size 2024-04-26T04:52:52.096 INFO:tasks.workunit.client.1.smithi078.stdout:6/464: dread d2/d1a/d5/d11/f4e [0,4194304] 0 2024-04-26T04:52:52.096 INFO:tasks.workunit.client.0.smithi077.stdout:8/120: chown d4/df/l11 142 1 2024-04-26T04:52:52.097 INFO:tasks.workunit.client.1.smithi078.stdout:3/391: dwrite d16/d26/d2e/d74/f76 [0,4194304] 0 2024-04-26T04:52:52.097 INFO:tasks.workunit.client.0.smithi077.stdout:7/68: rename da/ce to da/c17 0 2024-04-26T04:52:52.098 INFO:tasks.workunit.client.1.smithi078.stdout:6/465: chown d2/d1a/d5/d3a/f62 182666020 1 2024-04-26T04:52:52.098 INFO:tasks.workunit.client.0.smithi077.stdout:7/69: fdatasync da/ff 0 2024-04-26T04:52:52.099 INFO:tasks.workunit.client.1.smithi078.stdout:3/392: fdatasync d16/d26/d2b/d31/f59 0 2024-04-26T04:52:52.101 INFO:tasks.workunit.client.1.smithi078.stdout:4/426: dread d0/fc [0,4194304] 0 2024-04-26T04:52:52.108 INFO:tasks.workunit.client.1.smithi078.stdout:6/466: dwrite d2/d1a/f43 [0,4194304] 0 2024-04-26T04:52:52.110 INFO:tasks.workunit.client.1.smithi078.stdout:0/469: write d2/d9/d60/d6a/f87 [1312646,91348] 0 2024-04-26T04:52:52.110 INFO:tasks.workunit.client.1.smithi078.stdout:3/393: symlink d16/d26/l86 0 2024-04-26T04:52:52.111 INFO:tasks.workunit.client.1.smithi078.stdout:3/394: chown d16/d26/f2a 11621 1 2024-04-26T04:52:52.112 INFO:tasks.workunit.client.1.smithi078.stdout:3/395: chown d16/d26/d2e/d5f/d49/d77/f7b 60 1 2024-04-26T04:52:52.113 INFO:tasks.workunit.client.0.smithi077.stdout:4/145: creat d4/de/f2f x:0 0 0 2024-04-26T04:52:52.117 INFO:tasks.workunit.client.1.smithi078.stdout:4/427: rename d0/de/d22/d26/f57 to d0/d25/d20/d31/d33/d44/d69/fa4 0 2024-04-26T04:52:52.117 INFO:tasks.workunit.client.0.smithi077.stdout:8/121: creat d4/d17/f2c x:0 0 0 2024-04-26T04:52:52.119 INFO:tasks.workunit.client.1.smithi078.stdout:6/467: creat d2/de/f8d x:0 0 0 2024-04-26T04:52:52.120 INFO:tasks.workunit.client.1.smithi078.stdout:6/468: write d2/de/f12 [775833,33436] 0 2024-04-26T04:52:52.123 INFO:tasks.workunit.client.0.smithi077.stdout:7/70: readlink da/l10 0 2024-04-26T04:52:52.123 INFO:tasks.workunit.client.1.smithi078.stdout:3/396: mknod d16/d26/d2e/c87 0 2024-04-26T04:52:52.125 INFO:tasks.workunit.client.1.smithi078.stdout:1/460: sync 2024-04-26T04:52:52.126 INFO:tasks.workunit.client.0.smithi077.stdout:2/83: unlink d9/l14 0 2024-04-26T04:52:52.128 INFO:tasks.workunit.client.1.smithi078.stdout:5/322: dwrite d7/d18/f1a [0,4194304] 0 2024-04-26T04:52:52.131 INFO:tasks.workunit.client.1.smithi078.stdout:6/469: rename d2/d1a/d5/d2a/f3d to d2/d52/f8e 0 2024-04-26T04:52:52.132 INFO:tasks.workunit.client.1.smithi078.stdout:6/470: readlink d2/d1a/l13 0 2024-04-26T04:52:52.135 INFO:tasks.workunit.client.1.smithi078.stdout:0/470: dread d2/ff [0,4194304] 0 2024-04-26T04:52:52.135 INFO:tasks.workunit.client.0.smithi077.stdout:7/71: creat da/f18 x:0 0 0 2024-04-26T04:52:52.136 INFO:tasks.workunit.client.0.smithi077.stdout:7/72: read da/f11 [4134659,27505] 0 2024-04-26T04:52:52.137 INFO:tasks.workunit.client.0.smithi077.stdout:7/73: fdatasync da/ff 0 2024-04-26T04:52:52.137 INFO:tasks.workunit.client.0.smithi077.stdout:5/85: creat d1/d12/f19 x:0 0 0 2024-04-26T04:52:52.138 INFO:tasks.workunit.client.0.smithi077.stdout:7/74: write da/f11 [406052,92153] 0 2024-04-26T04:52:52.139 INFO:tasks.workunit.client.1.smithi078.stdout:8/391: dwrite d0/d19/d52/d35/f4e [0,4194304] 0 2024-04-26T04:52:52.140 INFO:tasks.workunit.client.1.smithi078.stdout:0/471: dread d2/d9/d60/d6a/f89 [0,4194304] 0 2024-04-26T04:52:52.141 INFO:tasks.workunit.client.1.smithi078.stdout:1/461: rmdir d7 39 2024-04-26T04:52:52.143 INFO:tasks.workunit.client.1.smithi078.stdout:2/374: write d8/d11/d18/d5d/d61/f62 [1441850,76181] 0 2024-04-26T04:52:52.145 INFO:tasks.workunit.client.0.smithi077.stdout:1/121: rename f3 to d9/f21 0 2024-04-26T04:52:52.146 INFO:tasks.workunit.client.1.smithi078.stdout:5/323: mknod d7/da/c69 0 2024-04-26T04:52:52.146 INFO:tasks.workunit.client.0.smithi077.stdout:6/117: write d8/fb [1351826,50153] 0 2024-04-26T04:52:52.152 INFO:tasks.workunit.client.1.smithi078.stdout:7/407: sync 2024-04-26T04:52:52.152 INFO:tasks.workunit.client.1.smithi078.stdout:9/359: sync 2024-04-26T04:52:52.152 INFO:tasks.workunit.client.0.smithi077.stdout:6/118: dwrite d8/fe [0,4194304] 0 2024-04-26T04:52:52.152 INFO:tasks.workunit.client.1.smithi078.stdout:3/397: mkdir d16/d3c/d88 0 2024-04-26T04:52:52.153 INFO:tasks.workunit.client.0.smithi077.stdout:5/86: fsync d1/fe 0 2024-04-26T04:52:52.154 INFO:tasks.workunit.client.1.smithi078.stdout:7/408: chown d9/da/d11/d13/d1a/f42 222 1 2024-04-26T04:52:52.155 INFO:tasks.workunit.client.1.smithi078.stdout:4/428: creat d0/d25/d20/fa5 x:0 0 0 2024-04-26T04:52:52.161 INFO:tasks.workunit.client.1.smithi078.stdout:8/392: rmdir d0/d19/d52/d35/d73 39 2024-04-26T04:52:52.166 INFO:tasks.workunit.client.0.smithi077.stdout:8/122: rename d4/d1a to d4/df/d22/d2d 0 2024-04-26T04:52:52.168 INFO:tasks.workunit.client.1.smithi078.stdout:0/472: rename d2/d39/c5e to d2/d9/d60/c9a 0 2024-04-26T04:52:52.168 INFO:tasks.workunit.client.0.smithi077.stdout:1/122: dwrite d9/f21 [4194304,4194304] 0 2024-04-26T04:52:52.172 INFO:tasks.workunit.client.0.smithi077.stdout:4/146: rmdir d4/de/d15/d1f/d20 0 2024-04-26T04:52:52.173 INFO:tasks.workunit.client.0.smithi077.stdout:4/147: write f2 [4846268,73783] 0 2024-04-26T04:52:52.173 INFO:tasks.workunit.client.1.smithi078.stdout:2/375: rmdir d8/d11/d14 39 2024-04-26T04:52:52.174 INFO:tasks.workunit.client.1.smithi078.stdout:2/376: readlink l4 0 2024-04-26T04:52:52.176 INFO:tasks.workunit.client.1.smithi078.stdout:7/409: dread d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e [0,4194304] 0 2024-04-26T04:52:52.177 INFO:tasks.workunit.client.1.smithi078.stdout:5/324: creat d7/d11/f6a x:0 0 0 2024-04-26T04:52:52.178 INFO:tasks.workunit.client.0.smithi077.stdout:6/119: symlink d8/d2a/l2d 0 2024-04-26T04:52:52.179 INFO:tasks.workunit.client.0.smithi077.stdout:4/148: dwrite d4/de/d15/d1f/f23 [0,4194304] 0 2024-04-26T04:52:52.180 INFO:tasks.workunit.client.0.smithi077.stdout:4/149: chown d4/de/d15/d1f/l29 42416 1 2024-04-26T04:52:52.181 INFO:tasks.workunit.client.1.smithi078.stdout:3/398: creat d16/d26/d2b/d31/f89 x:0 0 0 2024-04-26T04:52:52.181 INFO:tasks.workunit.client.0.smithi077.stdout:4/150: fdatasync d4/f8 0 2024-04-26T04:52:52.182 INFO:tasks.workunit.client.1.smithi078.stdout:4/429: dread d0/de/f34 [0,4194304] 0 2024-04-26T04:52:52.183 INFO:tasks.workunit.client.0.smithi077.stdout:7/75: creat da/d13/f19 x:0 0 0 2024-04-26T04:52:52.185 INFO:tasks.workunit.client.0.smithi077.stdout:4/151: truncate d4/de/d19/f1b 5392252 0 2024-04-26T04:52:52.186 INFO:tasks.workunit.client.0.smithi077.stdout:4/152: fsync d4/de/f2f 0 2024-04-26T04:52:52.186 INFO:tasks.workunit.client.0.smithi077.stdout:8/123: symlink d4/df/d22/l2e 0 2024-04-26T04:52:52.187 INFO:tasks.workunit.client.0.smithi077.stdout:6/120: dwrite d8/d1c/f24 [0,4194304] 0 2024-04-26T04:52:52.188 INFO:tasks.workunit.client.1.smithi078.stdout:1/462: dread d7/d39/d17/f66 [0,4194304] 0 2024-04-26T04:52:52.197 INFO:tasks.workunit.client.0.smithi077.stdout:1/123: dwrite d9/db/fe [0,4194304] 0 2024-04-26T04:52:52.202 INFO:tasks.workunit.client.0.smithi077.stdout:5/87: symlink d1/d12/l1a 0 2024-04-26T04:52:52.203 INFO:tasks.workunit.client.0.smithi077.stdout:2/84: getdents d9/dc 0 2024-04-26T04:52:52.208 INFO:tasks.workunit.client.1.smithi078.stdout:9/360: mknod dc/df/d4b/d22/d2a/d57/d69/c6b 0 2024-04-26T04:52:52.209 INFO:tasks.workunit.client.0.smithi077.stdout:4/153: mknod d4/de/c30 0 2024-04-26T04:52:52.213 INFO:tasks.workunit.client.0.smithi077.stdout:8/124: write d4/fd [8985,114333] 0 2024-04-26T04:52:52.213 INFO:tasks.workunit.client.1.smithi078.stdout:9/361: dwrite dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:52:52.213 INFO:tasks.workunit.client.0.smithi077.stdout:4/154: dread d4/de/d15/d1f/f23 [0,4194304] 0 2024-04-26T04:52:52.214 INFO:tasks.workunit.client.0.smithi077.stdout:8/125: fdatasync d4/df/f16 0 2024-04-26T04:52:52.214 INFO:tasks.workunit.client.0.smithi077.stdout:1/124: creat d9/db/dd/f22 x:0 0 0 2024-04-26T04:52:52.215 INFO:tasks.workunit.client.0.smithi077.stdout:8/126: dread - d4/df/d22/d2d/f1c zero size 2024-04-26T04:52:52.215 INFO:tasks.workunit.client.0.smithi077.stdout:1/125: write d9/db/f1c [1875100,86525] 0 2024-04-26T04:52:52.216 INFO:tasks.workunit.client.0.smithi077.stdout:5/88: creat d1/d12/d16/f1b x:0 0 0 2024-04-26T04:52:52.219 INFO:tasks.workunit.client.1.smithi078.stdout:8/393: rename d0/d19/d25/d36/c3e to d0/d19/d52/d35/d44/c78 0 2024-04-26T04:52:52.220 INFO:tasks.workunit.client.0.smithi077.stdout:7/76: mkdir da/d15/d1a 0 2024-04-26T04:52:52.220 INFO:tasks.workunit.client.1.smithi078.stdout:9/362: dwrite dc/f26 [0,4194304] 0 2024-04-26T04:52:52.223 INFO:tasks.workunit.client.1.smithi078.stdout:7/410: fsync d9/d57/f7a 0 2024-04-26T04:52:52.224 INFO:tasks.workunit.client.1.smithi078.stdout:7/411: chown d9/da/d11/c30 0 1 2024-04-26T04:52:52.224 INFO:tasks.workunit.client.0.smithi077.stdout:8/127: dwrite d4/d17/f2c [0,4194304] 0 2024-04-26T04:52:52.225 INFO:tasks.workunit.client.1.smithi078.stdout:5/325: link d7/d18/f44 d7/d18/d32/d37/f6b 0 2024-04-26T04:52:52.228 INFO:tasks.workunit.client.1.smithi078.stdout:6/471: getdents d2/d1a/d5/d11 0 2024-04-26T04:52:52.229 INFO:tasks.workunit.client.0.smithi077.stdout:8/128: dread d4/df/f1f [0,4194304] 0 2024-04-26T04:52:52.233 INFO:tasks.workunit.client.1.smithi078.stdout:3/399: mkdir d16/d26/d2b/d31/d71/d8a 0 2024-04-26T04:52:52.235 INFO:tasks.workunit.client.0.smithi077.stdout:1/126: creat d9/db/dd/f23 x:0 0 0 2024-04-26T04:52:52.236 INFO:tasks.workunit.client.1.smithi078.stdout:1/463: mknod d7/d39/dd/c99 0 2024-04-26T04:52:52.236 INFO:tasks.workunit.client.1.smithi078.stdout:3/400: dread d16/d26/d2b/d31/f3d [0,4194304] 0 2024-04-26T04:52:52.239 INFO:tasks.workunit.client.0.smithi077.stdout:2/85: mkdir d9/d15/d1a 0 2024-04-26T04:52:52.240 INFO:tasks.workunit.client.1.smithi078.stdout:0/473: creat d2/d9/d31/d4a/f9b x:0 0 0 2024-04-26T04:52:52.241 INFO:tasks.workunit.client.0.smithi077.stdout:4/155: write d4/f14 [2330607,130695] 0 2024-04-26T04:52:52.241 INFO:tasks.workunit.client.0.smithi077.stdout:7/77: mknod da/d13/c1b 0 2024-04-26T04:52:52.242 INFO:tasks.workunit.client.0.smithi077.stdout:4/156: truncate d4/f27 630307 0 2024-04-26T04:52:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:51 smithi077 ceph-mon[26311]: pgmap v15: 65 pgs: 65 active+clean; 753 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 78 MiB/s wr, 202 op/s 2024-04-26T04:52:52.244 INFO:tasks.workunit.client.1.smithi078.stdout:8/394: chown d0/d19/d52/d35/d44/c78 1 1 2024-04-26T04:52:52.246 INFO:tasks.workunit.client.1.smithi078.stdout:9/363: mknod dc/df/d37/d64/c6c 0 2024-04-26T04:52:52.246 INFO:tasks.workunit.client.0.smithi077.stdout:4/157: dread d4/de/d15/f16 [0,4194304] 0 2024-04-26T04:52:52.247 INFO:tasks.workunit.client.0.smithi077.stdout:6/121: rmdir d8/d2c 0 2024-04-26T04:52:52.248 INFO:tasks.workunit.client.0.smithi077.stdout:6/122: write d8/f22 [525973,71088] 0 2024-04-26T04:52:52.253 INFO:tasks.workunit.client.1.smithi078.stdout:6/472: mknod d2/d1a/d5/da/c8f 0 2024-04-26T04:52:52.253 INFO:tasks.workunit.client.1.smithi078.stdout:7/412: dwrite d9/da/f23 [4194304,4194304] 0 2024-04-26T04:52:52.254 INFO:tasks.workunit.client.0.smithi077.stdout:5/89: creat d1/d12/d16/d18/f1c x:0 0 0 2024-04-26T04:52:52.255 INFO:tasks.workunit.client.0.smithi077.stdout:5/90: chown d1/d12/l1a 86469 1 2024-04-26T04:52:52.255 INFO:tasks.workunit.client.1.smithi078.stdout:4/430: creat d0/de/d22/d26/fa6 x:0 0 0 2024-04-26T04:52:52.256 INFO:tasks.workunit.client.0.smithi077.stdout:2/86: mknod d9/d11/c1b 0 2024-04-26T04:52:52.257 INFO:tasks.workunit.client.1.smithi078.stdout:4/431: chown d0/d25/d20/d31/d33/d44/d55/f5d 3853 1 2024-04-26T04:52:52.258 INFO:tasks.workunit.client.0.smithi077.stdout:2/87: dread d9/fe [0,4194304] 0 2024-04-26T04:52:52.259 INFO:tasks.workunit.client.1.smithi078.stdout:0/474: chown d2/d9/d78/d84/f8e 0 1 2024-04-26T04:52:52.260 INFO:tasks.workunit.client.0.smithi077.stdout:8/129: creat d4/df/d22/d2d/d2b/f2f x:0 0 0 2024-04-26T04:52:52.262 INFO:tasks.workunit.client.1.smithi078.stdout:2/377: creat d8/d11/d14/f86 x:0 0 0 2024-04-26T04:52:52.263 INFO:tasks.workunit.client.0.smithi077.stdout:6/123: truncate d8/f17 697562 0 2024-04-26T04:52:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:51 smithi078 ceph-mon[33746]: pgmap v15: 65 pgs: 65 active+clean; 753 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 78 MiB/s wr, 202 op/s 2024-04-26T04:52:52.268 INFO:tasks.workunit.client.0.smithi077.stdout:7/78: creat da/d15/d1a/f1c x:0 0 0 2024-04-26T04:52:52.270 INFO:tasks.workunit.client.1.smithi078.stdout:7/413: mknod d9/c7f 0 2024-04-26T04:52:52.275 INFO:tasks.workunit.client.1.smithi078.stdout:3/401: link d16/d26/d2b/f40 d16/d26/d2e/d5f/d49/d77/f8b 0 2024-04-26T04:52:52.277 INFO:tasks.workunit.client.0.smithi077.stdout:6/124: creat d8/d2a/f2e x:0 0 0 2024-04-26T04:52:52.278 INFO:tasks.workunit.client.0.smithi077.stdout:6/125: write d8/d2a/f2e [493079,67405] 0 2024-04-26T04:52:52.278 INFO:tasks.workunit.client.1.smithi078.stdout:4/432: creat d0/de/d22/d27/d56/d5a/fa7 x:0 0 0 2024-04-26T04:52:52.281 INFO:tasks.workunit.client.0.smithi077.stdout:1/127: rename d9/db/dd/f22 to d9/f24 0 2024-04-26T04:52:52.281 INFO:tasks.workunit.client.1.smithi078.stdout:3/402: dwrite d16/d26/d2b/d31/f59 [0,4194304] 0 2024-04-26T04:52:52.282 INFO:tasks.workunit.client.1.smithi078.stdout:3/403: write d16/d26/d2b/d54/f58 [1169113,89984] 0 2024-04-26T04:52:52.285 INFO:tasks.workunit.client.1.smithi078.stdout:0/475: rmdir d2 39 2024-04-26T04:52:52.287 INFO:tasks.workunit.client.0.smithi077.stdout:5/91: mknod d1/c1d 0 2024-04-26T04:52:52.288 INFO:tasks.workunit.client.1.smithi078.stdout:2/378: mknod d8/c87 0 2024-04-26T04:52:52.289 INFO:tasks.workunit.client.1.smithi078.stdout:9/364: symlink dc/df/d4b/d22/l6d 0 2024-04-26T04:52:52.291 INFO:tasks.workunit.client.0.smithi077.stdout:7/79: creat da/d15/f1d x:0 0 0 2024-04-26T04:52:52.291 INFO:tasks.workunit.client.0.smithi077.stdout:7/80: dread - da/f18 zero size 2024-04-26T04:52:52.292 INFO:tasks.workunit.client.1.smithi078.stdout:6/473: mknod d2/d1a/d5/d3a/c90 0 2024-04-26T04:52:52.292 INFO:tasks.workunit.client.0.smithi077.stdout:7/81: dread - da/d13/f19 zero size 2024-04-26T04:52:52.293 INFO:tasks.workunit.client.1.smithi078.stdout:6/474: write d2/d1a/d38/f68 [5075191,104905] 0 2024-04-26T04:52:52.293 INFO:tasks.workunit.client.1.smithi078.stdout:6/475: readlink d2/d1a/d5/d2a/l25 0 2024-04-26T04:52:52.297 INFO:tasks.workunit.client.1.smithi078.stdout:4/433: symlink d0/d25/d20/d31/la8 0 2024-04-26T04:52:52.302 INFO:tasks.workunit.client.1.smithi078.stdout:4/434: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:52:52.302 INFO:tasks.workunit.client.1.smithi078.stdout:3/404: creat d16/d26/d2b/d31/d71/f8c x:0 0 0 2024-04-26T04:52:52.304 INFO:tasks.workunit.client.1.smithi078.stdout:3/405: chown d16/d26/f29 395 1 2024-04-26T04:52:52.304 INFO:tasks.workunit.client.1.smithi078.stdout:4/435: chown d0/d25/d20/fa5 3489 1 2024-04-26T04:52:52.306 INFO:tasks.workunit.client.1.smithi078.stdout:3/406: chown d16/d26/d2b/d31 17114625 1 2024-04-26T04:52:52.306 INFO:tasks.workunit.client.0.smithi077.stdout:5/92: mknod d1/d12/c1e 0 2024-04-26T04:52:52.307 INFO:tasks.workunit.client.0.smithi077.stdout:5/93: chown d1/d12/f13 17176843 1 2024-04-26T04:52:52.308 INFO:tasks.workunit.client.1.smithi078.stdout:9/365: dread dc/df/d4b/d22/d2d/f4d [4194304,4194304] 0 2024-04-26T04:52:52.311 INFO:tasks.workunit.client.1.smithi078.stdout:4/436: dread d0/de/f1a [0,4194304] 0 2024-04-26T04:52:52.311 INFO:tasks.workunit.client.1.smithi078.stdout:1/464: getdents d7/d39/db/d37/d58/d5c/d68 0 2024-04-26T04:52:52.311 INFO:tasks.workunit.client.1.smithi078.stdout:0/476: dwrite d2/d9/d31/f85 [0,4194304] 0 2024-04-26T04:52:52.314 INFO:tasks.workunit.client.1.smithi078.stdout:4/437: chown d0/d25/d20/f2a 7 1 2024-04-26T04:52:52.314 INFO:tasks.workunit.client.1.smithi078.stdout:0/477: truncate d2/d39/f7b 4312524 0 2024-04-26T04:52:52.315 INFO:tasks.workunit.client.0.smithi077.stdout:5/94: creat d1/d12/f1f x:0 0 0 2024-04-26T04:52:52.317 INFO:tasks.workunit.client.1.smithi078.stdout:7/414: link d9/da/d11/d13/d1a/l67 d9/da/d11/d13/d50/l80 0 2024-04-26T04:52:52.317 INFO:tasks.workunit.client.1.smithi078.stdout:3/407: dwrite d16/d26/d2b/d31/f89 [0,4194304] 0 2024-04-26T04:52:52.324 INFO:tasks.workunit.client.0.smithi077.stdout:2/88: dwrite d9/d11/f13 [0,4194304] 0 2024-04-26T04:52:52.324 INFO:tasks.workunit.client.0.smithi077.stdout:5/95: mknod d1/d8/c20 0 2024-04-26T04:52:52.326 INFO:tasks.workunit.client.0.smithi077.stdout:8/130: dwrite d4/df/d22/d2d/f1c [0,4194304] 0 2024-04-26T04:52:52.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/465: unlink d7/d39/db/d37/l7e 0 2024-04-26T04:52:52.327 INFO:tasks.workunit.client.1.smithi078.stdout:1/466: write f3 [7383677,33720] 0 2024-04-26T04:52:52.327 INFO:tasks.workunit.client.0.smithi077.stdout:8/131: fsync d4/df/f25 0 2024-04-26T04:52:52.335 INFO:tasks.workunit.client.1.smithi078.stdout:2/379: creat d8/d11/d18/d5d/d61/d1a/f88 x:0 0 0 2024-04-26T04:52:52.336 INFO:tasks.workunit.client.1.smithi078.stdout:1/467: dwrite d7/f8a [0,4194304] 0 2024-04-26T04:52:52.340 INFO:tasks.workunit.client.1.smithi078.stdout:5/326: sync 2024-04-26T04:52:52.341 INFO:tasks.workunit.client.1.smithi078.stdout:0/478: rename d2/d39/f7b to d2/d1e/d5a/d74/f9c 0 2024-04-26T04:52:52.345 INFO:tasks.workunit.client.0.smithi077.stdout:5/96: rename d1/d12/d16 to d1/d12/d21 0 2024-04-26T04:52:52.345 INFO:tasks.workunit.client.0.smithi077.stdout:1/128: dwrite f6 [0,4194304] 0 2024-04-26T04:52:52.346 INFO:tasks.workunit.client.0.smithi077.stdout:7/82: write da/f11 [1801524,26554] 0 2024-04-26T04:52:52.346 INFO:tasks.workunit.client.0.smithi077.stdout:5/97: dread - d1/d12/f13 zero size 2024-04-26T04:52:52.347 INFO:tasks.workunit.client.1.smithi078.stdout:9/366: mkdir dc/df/d4b/d22/d6e 0 2024-04-26T04:52:52.347 INFO:tasks.workunit.client.0.smithi077.stdout:5/98: dread - d1/d12/f13 zero size 2024-04-26T04:52:52.348 INFO:tasks.workunit.client.1.smithi078.stdout:0/479: dwrite d2/d9/d60/d6a/f87 [0,4194304] 0 2024-04-26T04:52:52.352 INFO:tasks.workunit.client.1.smithi078.stdout:7/415: dread d9/da/d33/d4b/d4c/f55 [0,4194304] 0 2024-04-26T04:52:52.357 INFO:tasks.workunit.client.0.smithi077.stdout:7/83: dwrite da/fc [0,4194304] 0 2024-04-26T04:52:52.363 INFO:tasks.workunit.client.1.smithi078.stdout:6/476: truncate d2/d1a/d5/f31 3496067 0 2024-04-26T04:52:52.364 INFO:tasks.workunit.client.0.smithi077.stdout:1/129: dwrite d9/f24 [0,4194304] 0 2024-04-26T04:52:52.364 INFO:tasks.workunit.client.1.smithi078.stdout:6/477: truncate d2/de/f8d 107861 0 2024-04-26T04:52:52.365 INFO:tasks.workunit.client.1.smithi078.stdout:6/478: write d2/d1a/d5/d2a/d4d/f78 [294410,40712] 0 2024-04-26T04:52:52.370 INFO:tasks.workunit.client.1.smithi078.stdout:9/367: dread dc/f3d [0,4194304] 0 2024-04-26T04:52:52.374 INFO:tasks.workunit.client.0.smithi077.stdout:1/130: dwrite d9/f20 [0,4194304] 0 2024-04-26T04:52:52.375 INFO:tasks.workunit.client.0.smithi077.stdout:7/84: symlink da/d15/d1a/l1e 0 2024-04-26T04:52:52.380 INFO:tasks.workunit.client.1.smithi078.stdout:0/480: creat d2/d39/f9d x:0 0 0 2024-04-26T04:52:52.380 INFO:tasks.workunit.client.1.smithi078.stdout:1/468: mknod d7/c9a 0 2024-04-26T04:52:52.382 INFO:tasks.workunit.client.0.smithi077.stdout:1/131: write d9/db/dd/f11 [2784606,58802] 0 2024-04-26T04:52:52.383 INFO:tasks.workunit.client.0.smithi077.stdout:8/132: dwrite d4/df/d22/d2d/f1b [0,4194304] 0 2024-04-26T04:52:52.384 INFO:tasks.workunit.client.0.smithi077.stdout:1/132: read - d9/db/dd/f23 zero size 2024-04-26T04:52:52.385 INFO:tasks.workunit.client.0.smithi077.stdout:1/133: truncate d9/f1d 234936 0 2024-04-26T04:52:52.386 INFO:tasks.workunit.client.1.smithi078.stdout:3/408: creat d16/f8d x:0 0 0 2024-04-26T04:52:52.390 INFO:tasks.workunit.client.0.smithi077.stdout:5/99: getdents d1/d12/d21/d18 0 2024-04-26T04:52:52.391 INFO:tasks.workunit.client.1.smithi078.stdout:6/479: unlink d2/d1a/f50 0 2024-04-26T04:52:52.392 INFO:tasks.workunit.client.0.smithi077.stdout:8/133: dread d4/df/f16 [0,4194304] 0 2024-04-26T04:52:52.392 INFO:tasks.workunit.client.1.smithi078.stdout:6/480: chown d2/d1a/d5/d3a/d42 14719530 1 2024-04-26T04:52:52.393 INFO:tasks.workunit.client.1.smithi078.stdout:6/481: fdatasync d2/d1a/fd 0 2024-04-26T04:52:52.393 INFO:tasks.workunit.client.1.smithi078.stdout:9/368: unlink dc/df/d4b/d22/d2a/d57/d69/c6b 0 2024-04-26T04:52:52.394 INFO:tasks.workunit.client.1.smithi078.stdout:9/369: stat dc/df/d17 0 2024-04-26T04:52:52.394 INFO:tasks.workunit.client.1.smithi078.stdout:9/370: chown dc/df/d4b/d3e/l65 7606 1 2024-04-26T04:52:52.395 INFO:tasks.workunit.client.0.smithi077.stdout:1/134: dwrite d9/db/f13 [0,4194304] 0 2024-04-26T04:52:52.396 INFO:tasks.workunit.client.0.smithi077.stdout:1/135: chown d9/f1d 0 1 2024-04-26T04:52:52.397 INFO:tasks.workunit.client.0.smithi077.stdout:1/136: write d9/db/dd/f23 [416665,66114] 0 2024-04-26T04:52:52.399 INFO:tasks.workunit.client.1.smithi078.stdout:0/481: creat d2/d1e/f9e x:0 0 0 2024-04-26T04:52:52.401 INFO:tasks.workunit.client.0.smithi077.stdout:8/134: creat d4/df/d22/d2d/f30 x:0 0 0 2024-04-26T04:52:52.402 INFO:tasks.workunit.client.0.smithi077.stdout:8/135: read d4/df/f1f [114517,97229] 0 2024-04-26T04:52:52.404 INFO:tasks.workunit.client.1.smithi078.stdout:4/438: dwrite d0/d25/d20/d31/d33/d44/d69/fa4 [0,4194304] 0 2024-04-26T04:52:52.404 INFO:tasks.workunit.client.0.smithi077.stdout:1/137: rmdir d9/db/dd 39 2024-04-26T04:52:52.405 INFO:tasks.workunit.client.0.smithi077.stdout:3/98: sync 2024-04-26T04:52:52.405 INFO:tasks.workunit.client.0.smithi077.stdout:0/78: sync 2024-04-26T04:52:52.405 INFO:tasks.workunit.client.0.smithi077.stdout:9/105: sync 2024-04-26T04:52:52.406 INFO:tasks.workunit.client.0.smithi077.stdout:1/138: read d9/db/f1c [803789,6791] 0 2024-04-26T04:52:52.406 INFO:tasks.workunit.client.0.smithi077.stdout:3/99: stat d0/f6 0 2024-04-26T04:52:52.406 INFO:tasks.workunit.client.0.smithi077.stdout:9/106: dread - d9/f1c zero size 2024-04-26T04:52:52.407 INFO:tasks.workunit.client.1.smithi078.stdout:2/380: getdents d8/d11/d18/d5d 0 2024-04-26T04:52:52.407 INFO:tasks.workunit.client.0.smithi077.stdout:7/85: rmdir da 39 2024-04-26T04:52:52.407 INFO:tasks.workunit.client.1.smithi078.stdout:4/439: chown d0/de/d22/d27/d56/d5a/fa7 49726154 1 2024-04-26T04:52:52.409 INFO:tasks.workunit.client.1.smithi078.stdout:3/409: creat d16/d26/d2e/d74/f8e x:0 0 0 2024-04-26T04:52:52.410 INFO:tasks.workunit.client.1.smithi078.stdout:0/482: dwrite d2/d9/d31/f50 [4194304,4194304] 0 2024-04-26T04:52:52.410 INFO:tasks.workunit.client.1.smithi078.stdout:4/440: stat d0/d25/d20/d31/d33/d44/d55/f95 0 2024-04-26T04:52:52.411 INFO:tasks.workunit.client.1.smithi078.stdout:1/469: dread d7/f13 [0,4194304] 0 2024-04-26T04:52:52.413 INFO:tasks.workunit.client.0.smithi077.stdout:3/100: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:52:52.417 INFO:tasks.workunit.client.1.smithi078.stdout:4/441: dread d0/de/f78 [0,4194304] 0 2024-04-26T04:52:52.419 INFO:tasks.workunit.client.1.smithi078.stdout:0/483: dwrite d2/d9/d31/d4a/f82 [0,4194304] 0 2024-04-26T04:52:52.420 INFO:tasks.workunit.client.1.smithi078.stdout:9/371: mknod dc/df/d27/c6f 0 2024-04-26T04:52:52.421 INFO:tasks.workunit.client.1.smithi078.stdout:9/372: readlink dc/df/d4b/d22/d2a/l68 0 2024-04-26T04:52:52.426 INFO:tasks.workunit.client.1.smithi078.stdout:9/373: read dc/df/d17/f48 [498754,105326] 0 2024-04-26T04:52:52.428 INFO:tasks.workunit.client.1.smithi078.stdout:9/374: truncate dc/f4e 987401 0 2024-04-26T04:52:52.430 INFO:tasks.workunit.client.1.smithi078.stdout:1/470: dread d7/d39/db/d37/d58/f5e [0,4194304] 0 2024-04-26T04:52:52.431 INFO:tasks.workunit.client.0.smithi077.stdout:3/101: creat d0/d7/d15/f22 x:0 0 0 2024-04-26T04:52:52.432 INFO:tasks.workunit.client.1.smithi078.stdout:8/395: sync 2024-04-26T04:52:52.433 INFO:tasks.workunit.client.0.smithi077.stdout:1/139: mknod d9/db/dd/c25 0 2024-04-26T04:52:52.434 INFO:tasks.workunit.client.1.smithi078.stdout:8/396: dread d0/f2f [0,4194304] 0 2024-04-26T04:52:52.438 INFO:tasks.workunit.client.1.smithi078.stdout:6/482: symlink d2/de/d27/d4c/l91 0 2024-04-26T04:52:52.438 INFO:tasks.workunit.client.0.smithi077.stdout:7/86: mknod da/d13/c1f 0 2024-04-26T04:52:52.439 INFO:tasks.workunit.client.0.smithi077.stdout:7/87: dread - da/d15/d1a/f1c zero size 2024-04-26T04:52:52.440 INFO:tasks.workunit.client.1.smithi078.stdout:0/484: chown d2/d9/f30 8 1 2024-04-26T04:52:52.440 INFO:tasks.workunit.client.0.smithi077.stdout:1/140: symlink d9/db/dd/l26 0 2024-04-26T04:52:52.441 INFO:tasks.workunit.client.0.smithi077.stdout:1/141: fsync f6 0 2024-04-26T04:52:52.444 INFO:tasks.workunit.client.0.smithi077.stdout:3/102: rename d0/f6 to d0/d9/f23 0 2024-04-26T04:52:52.445 INFO:tasks.workunit.client.0.smithi077.stdout:3/103: write d0/f19 [3188935,62751] 0 2024-04-26T04:52:52.445 INFO:tasks.workunit.client.0.smithi077.stdout:7/88: creat da/d15/d1a/f20 x:0 0 0 2024-04-26T04:52:52.446 INFO:tasks.workunit.client.0.smithi077.stdout:7/89: chown da/d15/d1a 254720657 1 2024-04-26T04:52:52.448 INFO:tasks.workunit.client.1.smithi078.stdout:9/375: mknod dc/df/d17/c70 0 2024-04-26T04:52:52.451 INFO:tasks.workunit.client.1.smithi078.stdout:1/471: unlink d7/d2e/f67 0 2024-04-26T04:52:52.453 INFO:tasks.workunit.client.0.smithi077.stdout:7/90: dwrite da/d15/f1d [0,4194304] 0 2024-04-26T04:52:52.453 INFO:tasks.workunit.client.0.smithi077.stdout:0/79: truncate d0/f3 719958 0 2024-04-26T04:52:52.454 INFO:tasks.workunit.client.0.smithi077.stdout:7/91: stat da/d15/f1d 0 2024-04-26T04:52:52.454 INFO:tasks.workunit.client.0.smithi077.stdout:0/80: truncate d0/db/f22 742975 0 2024-04-26T04:52:52.456 INFO:tasks.workunit.client.0.smithi077.stdout:3/104: symlink d0/d7/l24 0 2024-04-26T04:52:52.457 INFO:tasks.workunit.client.0.smithi077.stdout:3/105: stat d0/d7/d15/f1a 0 2024-04-26T04:52:52.457 INFO:tasks.workunit.client.0.smithi077.stdout:1/142: mkdir d9/db/d27 0 2024-04-26T04:52:52.460 INFO:tasks.workunit.client.0.smithi077.stdout:0/81: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:52.460 INFO:tasks.workunit.client.0.smithi077.stdout:0/82: chown d0/d19 20 1 2024-04-26T04:52:52.461 INFO:tasks.workunit.client.1.smithi078.stdout:2/381: truncate d8/d11/f54 3124698 0 2024-04-26T04:52:52.463 INFO:tasks.workunit.client.0.smithi077.stdout:7/92: rmdir da/d13 39 2024-04-26T04:52:52.467 INFO:tasks.workunit.client.0.smithi077.stdout:0/83: mknod d0/c23 0 2024-04-26T04:52:52.468 INFO:tasks.workunit.client.0.smithi077.stdout:0/84: write d0/f2 [308540,25980] 0 2024-04-26T04:52:52.472 INFO:tasks.workunit.client.1.smithi078.stdout:3/410: write d16/f17 [2453779,63224] 0 2024-04-26T04:52:52.474 INFO:tasks.workunit.client.0.smithi077.stdout:1/143: creat d9/db/d27/f28 x:0 0 0 2024-04-26T04:52:52.478 INFO:tasks.workunit.client.1.smithi078.stdout:5/327: sync 2024-04-26T04:52:52.480 INFO:tasks.workunit.client.1.smithi078.stdout:5/328: dread d7/d18/d32/d37/f51 [0,4194304] 0 2024-04-26T04:52:52.481 INFO:tasks.workunit.client.1.smithi078.stdout:1/472: rename d7/d39/d4c/d3d/c6d to d7/d39/db/d49/d93/c9b 0 2024-04-26T04:52:52.482 INFO:tasks.workunit.client.0.smithi077.stdout:7/93: rename da/d13/c1f to da/d13/c21 0 2024-04-26T04:52:52.485 INFO:tasks.workunit.client.1.smithi078.stdout:2/382: mknod d8/d11/d6e/c89 0 2024-04-26T04:52:52.486 INFO:tasks.workunit.client.1.smithi078.stdout:8/397: truncate d0/d19/d52/d5c/f74 177932 0 2024-04-26T04:52:52.489 INFO:tasks.workunit.client.0.smithi077.stdout:7/94: mkdir da/d13/d22 0 2024-04-26T04:52:52.490 INFO:tasks.workunit.client.0.smithi077.stdout:7/95: chown da/l10 74468436 1 2024-04-26T04:52:52.491 INFO:tasks.workunit.client.1.smithi078.stdout:4/442: creat d0/d25/d20/fa9 x:0 0 0 2024-04-26T04:52:52.492 INFO:tasks.workunit.client.1.smithi078.stdout:4/443: truncate d0/de/d22/d26/d2e/f50 357040 0 2024-04-26T04:52:52.492 INFO:tasks.workunit.client.0.smithi077.stdout:1/144: symlink d9/l29 0 2024-04-26T04:52:52.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/485: symlink d2/l9f 0 2024-04-26T04:52:52.493 INFO:tasks.workunit.client.1.smithi078.stdout:3/411: write d16/d26/d2b/d54/d5a/f82 [647099,35499] 0 2024-04-26T04:52:52.494 INFO:tasks.workunit.client.0.smithi077.stdout:1/145: chown d9/db/f1c 1024192168 1 2024-04-26T04:52:52.495 INFO:tasks.workunit.client.1.smithi078.stdout:9/376: creat dc/df/d4b/d22/d6e/f71 x:0 0 0 2024-04-26T04:52:52.495 INFO:tasks.workunit.client.0.smithi077.stdout:0/85: dwrite d0/db/ff [0,4194304] 0 2024-04-26T04:52:52.497 INFO:tasks.workunit.client.1.smithi078.stdout:5/329: symlink d7/d18/d32/l6c 0 2024-04-26T04:52:52.498 INFO:tasks.workunit.client.1.smithi078.stdout:5/330: write d7/f38 [2345786,100370] 0 2024-04-26T04:52:52.499 INFO:tasks.workunit.client.1.smithi078.stdout:1/473: creat d7/d39/db/d47/d4a/f9c x:0 0 0 2024-04-26T04:52:52.501 INFO:tasks.workunit.client.0.smithi077.stdout:7/96: dwrite da/f18 [0,4194304] 0 2024-04-26T04:52:52.502 INFO:tasks.workunit.client.0.smithi077.stdout:1/146: symlink d9/db/d27/l2a 0 2024-04-26T04:52:52.503 INFO:tasks.workunit.client.0.smithi077.stdout:7/97: write da/fb [2716525,117890] 0 2024-04-26T04:52:52.509 INFO:tasks.workunit.client.0.smithi077.stdout:0/86: dread d0/db/d11/f14 [0,4194304] 0 2024-04-26T04:52:52.511 INFO:tasks.workunit.client.1.smithi078.stdout:2/383: dwrite d8/d11/d18/d5d/d61/d1a/d34/f3d [0,4194304] 0 2024-04-26T04:52:52.516 INFO:tasks.workunit.client.1.smithi078.stdout:9/377: symlink dc/df/d4b/l72 0 2024-04-26T04:52:52.522 INFO:tasks.workunit.client.0.smithi077.stdout:1/147: mkdir d9/db/d2b 0 2024-04-26T04:52:52.524 INFO:tasks.workunit.client.0.smithi077.stdout:7/98: link da/f18 da/d13/f23 0 2024-04-26T04:52:52.529 INFO:tasks.workunit.client.0.smithi077.stdout:1/148: rename d9/db/d27/l2a to d9/db/d2b/l2c 0 2024-04-26T04:52:52.530 INFO:tasks.workunit.client.0.smithi077.stdout:1/149: write d9/f1d [166510,38786] 0 2024-04-26T04:52:52.530 INFO:tasks.workunit.client.1.smithi078.stdout:6/483: getdents d2/d1a/d5/d2a/d4d 0 2024-04-26T04:52:52.531 INFO:tasks.workunit.client.0.smithi077.stdout:1/150: readlink d9/db/dd/lf 0 2024-04-26T04:52:52.531 INFO:tasks.workunit.client.1.smithi078.stdout:6/484: chown d2/de/f33 1 1 2024-04-26T04:52:52.531 INFO:tasks.workunit.client.1.smithi078.stdout:6/485: chown d2/d1a/c1f 1122711 1 2024-04-26T04:52:52.532 INFO:tasks.workunit.client.0.smithi077.stdout:1/151: chown d9/db/dd/f11 330619602 1 2024-04-26T04:52:52.532 INFO:tasks.workunit.client.1.smithi078.stdout:6/486: fdatasync d2/d1a/d5c/f6d 0 2024-04-26T04:52:52.533 INFO:tasks.workunit.client.0.smithi077.stdout:1/152: write d9/db/d27/f28 [779804,23519] 0 2024-04-26T04:52:52.533 INFO:tasks.workunit.client.1.smithi078.stdout:0/486: creat d2/d9/d18/d7d/fa0 x:0 0 0 2024-04-26T04:52:52.534 INFO:tasks.workunit.client.0.smithi077.stdout:1/153: write f2 [2645922,99711] 0 2024-04-26T04:52:52.538 INFO:tasks.workunit.client.0.smithi077.stdout:0/87: truncate d0/db/d11/f14 239983 0 2024-04-26T04:52:52.538 INFO:tasks.workunit.client.1.smithi078.stdout:3/412: dwrite f13 [0,4194304] 0 2024-04-26T04:52:52.539 INFO:tasks.workunit.client.1.smithi078.stdout:2/384: dwrite d8/d11/d6e/f80 [0,4194304] 0 2024-04-26T04:52:52.540 INFO:tasks.workunit.client.1.smithi078.stdout:3/413: fsync d16/d26/d2e/d5f/d49/f4b 0 2024-04-26T04:52:52.543 INFO:tasks.workunit.client.0.smithi077.stdout:7/99: truncate da/d15/f1d 2939760 0 2024-04-26T04:52:52.544 INFO:tasks.workunit.client.1.smithi078.stdout:1/474: symlink d7/d39/dd/l9d 0 2024-04-26T04:52:52.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/398: link d0/d19/d52/c12 d0/d19/d52/d35/d44/d6b/c79 0 2024-04-26T04:52:52.545 INFO:tasks.workunit.client.0.smithi077.stdout:1/154: mknod d9/c2d 0 2024-04-26T04:52:52.546 INFO:tasks.workunit.client.0.smithi077.stdout:1/155: chown d9/db/d27/f28 6960161 1 2024-04-26T04:52:52.549 INFO:tasks.workunit.client.0.smithi077.stdout:7/100: write da/f18 [309264,74024] 0 2024-04-26T04:52:52.549 INFO:tasks.workunit.client.0.smithi077.stdout:7/101: readlink da/d15/d1a/l1e 0 2024-04-26T04:52:52.550 INFO:tasks.workunit.client.1.smithi078.stdout:6/487: symlink d2/de/d27/d4c/d5f/l92 0 2024-04-26T04:52:52.552 INFO:tasks.workunit.client.1.smithi078.stdout:6/488: fdatasync d2/d1a/d5/da/f4a 0 2024-04-26T04:52:52.552 INFO:tasks.workunit.client.0.smithi077.stdout:7/102: write da/d15/d1a/f1c [372916,24469] 0 2024-04-26T04:52:52.552 INFO:tasks.workunit.client.0.smithi077.stdout:1/156: symlink d9/db/dd/l2e 0 2024-04-26T04:52:52.553 INFO:tasks.workunit.client.0.smithi077.stdout:7/103: fdatasync da/f11 0 2024-04-26T04:52:52.553 INFO:tasks.workunit.client.1.smithi078.stdout:6/489: write d2/de/f12 [2148937,113878] 0 2024-04-26T04:52:52.554 INFO:tasks.workunit.client.1.smithi078.stdout:9/378: truncate f1 563581 0 2024-04-26T04:52:52.555 INFO:tasks.workunit.client.1.smithi078.stdout:0/487: mknod d2/d9/d31/ca1 0 2024-04-26T04:52:52.557 INFO:tasks.workunit.client.1.smithi078.stdout:2/385: dread d8/f55 [0,4194304] 0 2024-04-26T04:52:52.557 INFO:tasks.workunit.client.0.smithi077.stdout:1/157: unlink d9/db/dd/f10 0 2024-04-26T04:52:52.562 INFO:tasks.workunit.client.1.smithi078.stdout:3/414: creat d16/d26/d2e/d5f/d49/f8f x:0 0 0 2024-04-26T04:52:52.563 INFO:tasks.workunit.client.0.smithi077.stdout:1/158: dwrite d9/f20 [0,4194304] 0 2024-04-26T04:52:52.567 INFO:tasks.workunit.client.0.smithi077.stdout:7/104: rename da/d15/f1d to da/d15/f24 0 2024-04-26T04:52:52.571 INFO:tasks.workunit.client.1.smithi078.stdout:1/475: rename d7/d39 to d7/d1e/d9e 0 2024-04-26T04:52:52.572 INFO:tasks.workunit.client.1.smithi078.stdout:1/476: chown d7/d2e/l52 1928306745 1 2024-04-26T04:52:52.572 INFO:tasks.workunit.client.0.smithi077.stdout:7/105: dread da/d15/d1a/f1c [0,4194304] 0 2024-04-26T04:52:52.572 INFO:tasks.workunit.client.0.smithi077.stdout:1/159: rename d9/db/fe to d9/db/dd/f2f 0 2024-04-26T04:52:52.576 INFO:tasks.workunit.client.1.smithi078.stdout:1/477: read d7/d1e/d9e/db/f15 [3436154,74872] 0 2024-04-26T04:52:52.577 INFO:tasks.workunit.client.1.smithi078.stdout:6/490: dread d2/d1a/d5/f6b [0,4194304] 0 2024-04-26T04:52:52.578 INFO:tasks.workunit.client.1.smithi078.stdout:6/491: chown d2/d1a/d5/c9 311640834 1 2024-04-26T04:52:52.583 INFO:tasks.workunit.client.1.smithi078.stdout:8/399: dread d0/d19/d52/d5c/f74 [0,4194304] 0 2024-04-26T04:52:52.585 INFO:tasks.workunit.client.1.smithi078.stdout:2/386: unlink d8/c26 0 2024-04-26T04:52:52.586 INFO:tasks.workunit.client.1.smithi078.stdout:2/387: read - d8/d11/d18/d5d/d61/d1a/f88 zero size 2024-04-26T04:52:52.587 INFO:tasks.workunit.client.0.smithi077.stdout:7/106: creat da/d13/d22/f25 x:0 0 0 2024-04-26T04:52:52.587 INFO:tasks.workunit.client.1.smithi078.stdout:2/388: read - d8/d11/d14/f86 zero size 2024-04-26T04:52:52.587 INFO:tasks.workunit.client.1.smithi078.stdout:7/416: sync 2024-04-26T04:52:52.588 INFO:tasks.workunit.client.0.smithi077.stdout:7/107: rename da/d15/d1a to da/d15/d1a/d26 22 2024-04-26T04:52:52.591 INFO:tasks.workunit.client.0.smithi077.stdout:7/108: dread da/fc [0,4194304] 0 2024-04-26T04:52:52.593 INFO:tasks.workunit.client.1.smithi078.stdout:1/478: write d7/d2e/f5f [1576784,35725] 0 2024-04-26T04:52:52.596 INFO:tasks.workunit.client.1.smithi078.stdout:6/492: mknod d2/d1a/d5/da/c93 0 2024-04-26T04:52:52.597 INFO:tasks.workunit.client.0.smithi077.stdout:7/109: creat da/d13/d22/f27 x:0 0 0 2024-04-26T04:52:52.598 INFO:tasks.workunit.client.1.smithi078.stdout:6/493: write d2/d1a/d5/da/f2c [509837,75101] 0 2024-04-26T04:52:52.598 INFO:tasks.workunit.client.1.smithi078.stdout:6/494: readlink d2/d52/l85 0 2024-04-26T04:52:52.600 INFO:tasks.workunit.client.0.smithi077.stdout:7/110: fsync da/fc 0 2024-04-26T04:52:52.601 INFO:tasks.workunit.client.0.smithi077.stdout:7/111: write da/fc [659422,69383] 0 2024-04-26T04:52:52.603 INFO:tasks.workunit.client.1.smithi078.stdout:6/495: dwrite d2/d1a/d5/d11/f1c [0,4194304] 0 2024-04-26T04:52:52.604 INFO:tasks.workunit.client.0.smithi077.stdout:1/160: dwrite d9/f1b [0,4194304] 0 2024-04-26T04:52:52.606 INFO:tasks.workunit.client.1.smithi078.stdout:2/389: rmdir d8/d11/d18/d5d/d61/d1a 39 2024-04-26T04:52:52.607 INFO:tasks.workunit.client.1.smithi078.stdout:7/417: write d9/da/d11/d13/d1a/f40 [906144,108473] 0 2024-04-26T04:52:52.609 INFO:tasks.workunit.client.1.smithi078.stdout:3/415: symlink d16/d26/d2b/d31/d78/l90 0 2024-04-26T04:52:52.613 INFO:tasks.workunit.client.0.smithi077.stdout:1/161: fdatasync d9/db/dd/f15 0 2024-04-26T04:52:52.615 INFO:tasks.workunit.client.0.smithi077.stdout:7/112: mknod da/c28 0 2024-04-26T04:52:52.618 INFO:tasks.workunit.client.0.smithi077.stdout:1/162: creat d9/db/d27/f30 x:0 0 0 2024-04-26T04:52:52.619 INFO:tasks.workunit.client.1.smithi078.stdout:0/488: link d2/d39/f52 d2/d9/d31/d4a/fa2 0 2024-04-26T04:52:52.620 INFO:tasks.workunit.client.0.smithi077.stdout:7/113: dread da/f18 [0,4194304] 0 2024-04-26T04:52:52.620 INFO:tasks.workunit.client.0.smithi077.stdout:7/114: fsync da/fb 0 2024-04-26T04:52:52.620 INFO:tasks.workunit.client.0.smithi077.stdout:7/115: readlink da/l10 0 2024-04-26T04:52:52.622 INFO:tasks.workunit.client.0.smithi077.stdout:1/163: rename d9/ca to d9/db/d2b/c31 0 2024-04-26T04:52:52.628 INFO:tasks.workunit.client.0.smithi077.stdout:7/116: dwrite da/fc [0,4194304] 0 2024-04-26T04:52:52.631 INFO:tasks.workunit.client.0.smithi077.stdout:1/164: write d9/db/dd/f2f [2918405,105553] 0 2024-04-26T04:52:52.631 INFO:tasks.workunit.client.0.smithi077.stdout:1/165: chown f6 76512723 1 2024-04-26T04:52:52.633 INFO:tasks.workunit.client.0.smithi077.stdout:7/117: rename da/d13/c1b to da/d15/c29 0 2024-04-26T04:52:52.634 INFO:tasks.workunit.client.1.smithi078.stdout:2/390: dwrite d8/d11/d18/d5d/d61/d1a/f6c [0,4194304] 0 2024-04-26T04:52:52.635 INFO:tasks.workunit.client.1.smithi078.stdout:9/379: getdents dc/df 0 2024-04-26T04:52:52.636 INFO:tasks.workunit.client.1.smithi078.stdout:0/489: mkdir d2/d1e/d5a/da3 0 2024-04-26T04:52:52.638 INFO:tasks.workunit.client.1.smithi078.stdout:0/490: write d2/f6 [8835476,108173] 0 2024-04-26T04:52:52.639 INFO:tasks.workunit.client.0.smithi077.stdout:7/118: creat da/d15/f2a x:0 0 0 2024-04-26T04:52:52.645 INFO:tasks.workunit.client.1.smithi078.stdout:2/391: dread d8/d11/d18/d5d/d61/d1a/f49 [0,4194304] 0 2024-04-26T04:52:52.645 INFO:tasks.workunit.client.1.smithi078.stdout:0/491: dwrite d2/d9/d18/f62 [0,4194304] 0 2024-04-26T04:52:52.645 INFO:tasks.workunit.client.1.smithi078.stdout:7/418: creat d9/da/d33/d4b/d4c/d6c/f81 x:0 0 0 2024-04-26T04:52:52.646 INFO:tasks.workunit.client.0.smithi077.stdout:7/119: creat da/d13/d22/f2b x:0 0 0 2024-04-26T04:52:52.647 INFO:tasks.workunit.client.1.smithi078.stdout:0/492: chown d2/d1e/d5a/d74 156636 1 2024-04-26T04:52:52.648 INFO:tasks.workunit.client.0.smithi077.stdout:7/120: truncate da/d13/d22/f2b 531845 0 2024-04-26T04:52:52.649 INFO:tasks.workunit.client.1.smithi078.stdout:6/496: dwrite d2/d49/f57 [4194304,4194304] 0 2024-04-26T04:52:52.649 INFO:tasks.workunit.client.1.smithi078.stdout:0/493: readlink d2/d9/d31/d4a/l51 0 2024-04-26T04:52:52.651 INFO:tasks.workunit.client.0.smithi077.stdout:1/166: truncate d9/f1b 2182257 0 2024-04-26T04:52:52.651 INFO:tasks.workunit.client.0.smithi077.stdout:7/121: rename da to da/d2c 22 2024-04-26T04:52:52.653 INFO:tasks.workunit.client.1.smithi078.stdout:8/400: getdents d0/d19/d25/d36 0 2024-04-26T04:52:52.654 INFO:tasks.workunit.client.1.smithi078.stdout:8/401: fsync d0/d19/d52/d35/d44/f5d 0 2024-04-26T04:52:52.654 INFO:tasks.workunit.client.0.smithi077.stdout:1/167: dread d9/f1d [0,4194304] 0 2024-04-26T04:52:52.659 INFO:tasks.workunit.client.1.smithi078.stdout:2/392: creat d8/d11/d18/d5d/f8a x:0 0 0 2024-04-26T04:52:52.661 INFO:tasks.workunit.client.0.smithi077.stdout:1/168: rmdir d9 39 2024-04-26T04:52:52.663 INFO:tasks.workunit.client.1.smithi078.stdout:1/479: getdents d7/d2e 0 2024-04-26T04:52:52.666 INFO:tasks.workunit.client.1.smithi078.stdout:0/494: dread d2/d9/d18/f8a [0,4194304] 0 2024-04-26T04:52:52.668 INFO:tasks.workunit.client.1.smithi078.stdout:5/331: sync 2024-04-26T04:52:52.668 INFO:tasks.workunit.client.1.smithi078.stdout:4/444: sync 2024-04-26T04:52:52.670 INFO:tasks.workunit.client.1.smithi078.stdout:8/402: mkdir d0/d7a 0 2024-04-26T04:52:52.671 INFO:tasks.workunit.client.1.smithi078.stdout:8/403: chown d0/d19/d52/d35/c55 13 1 2024-04-26T04:52:52.671 INFO:tasks.workunit.client.0.smithi077.stdout:1/169: dwrite d9/db/d27/f28 [0,4194304] 0 2024-04-26T04:52:52.672 INFO:tasks.workunit.client.1.smithi078.stdout:8/404: chown d0/d19/d52/d35/d44/c1d 2 1 2024-04-26T04:52:52.672 INFO:tasks.workunit.client.1.smithi078.stdout:9/380: dread dc/df/d17/f48 [0,4194304] 0 2024-04-26T04:52:52.673 INFO:tasks.workunit.client.1.smithi078.stdout:7/419: mknod d9/da/d11/c82 0 2024-04-26T04:52:52.675 INFO:tasks.workunit.client.0.smithi077.stdout:7/122: link da/d15/c29 da/c2d 0 2024-04-26T04:52:52.678 INFO:tasks.workunit.client.1.smithi078.stdout:6/497: write d2/d1a/d5/da/d35/f3e [1326613,102604] 0 2024-04-26T04:52:52.681 INFO:tasks.workunit.client.0.smithi077.stdout:7/123: rename da/d15/d1a/f1c to da/d13/d22/f2e 0 2024-04-26T04:52:52.682 INFO:tasks.workunit.client.0.smithi077.stdout:7/124: truncate da/d15/d1a/f20 213251 0 2024-04-26T04:52:52.684 INFO:tasks.workunit.client.1.smithi078.stdout:2/393: write d8/d11/d14/d5b/f81 [740775,79493] 0 2024-04-26T04:52:52.684 INFO:tasks.workunit.client.1.smithi078.stdout:4/445: rmdir d0/d25/d20/d31/d33/d44/d55/d5e 39 2024-04-26T04:52:52.685 INFO:tasks.workunit.client.1.smithi078.stdout:6/498: dread d2/d1a/d5/d3a/f62 [0,4194304] 0 2024-04-26T04:52:52.686 INFO:tasks.workunit.client.1.smithi078.stdout:8/405: symlink d0/d19/d52/d35/d3f/l7b 0 2024-04-26T04:52:52.686 INFO:tasks.workunit.client.0.smithi077.stdout:7/125: creat da/d15/f2f x:0 0 0 2024-04-26T04:52:52.687 INFO:tasks.workunit.client.1.smithi078.stdout:7/420: mkdir d9/da/d33/d4b/d4c/d5b/d83 0 2024-04-26T04:52:52.688 INFO:tasks.workunit.client.1.smithi078.stdout:8/406: write d0/d19/d52/d35/d44/f4d [4685635,125839] 0 2024-04-26T04:52:52.690 INFO:tasks.workunit.client.0.smithi077.stdout:1/170: getdents d9/db 0 2024-04-26T04:52:52.691 INFO:tasks.workunit.client.1.smithi078.stdout:5/332: link d7/d18/f34 d7/d18/d32/d37/f6d 0 2024-04-26T04:52:52.692 INFO:tasks.workunit.client.0.smithi077.stdout:1/171: readlink d9/db/d2b/l2c 0 2024-04-26T04:52:52.693 INFO:tasks.workunit.client.1.smithi078.stdout:2/394: chown d8/d11/d14/f67 1989845671 1 2024-04-26T04:52:52.693 INFO:tasks.workunit.client.1.smithi078.stdout:4/446: dwrite d0/de/d22/d27/d56/f5c [0,4194304] 0 2024-04-26T04:52:52.693 INFO:tasks.workunit.client.0.smithi077.stdout:7/126: dwrite da/f11 [0,4194304] 0 2024-04-26T04:52:52.693 INFO:tasks.workunit.client.1.smithi078.stdout:6/499: dwrite d2/d1a/d38/f68 [0,4194304] 0 2024-04-26T04:52:52.694 INFO:tasks.workunit.client.1.smithi078.stdout:5/333: truncate d7/d18/f26 5651661 0 2024-04-26T04:52:52.694 INFO:tasks.workunit.client.0.smithi077.stdout:1/172: write d9/db/dd/f2f [4736024,98046] 0 2024-04-26T04:52:52.701 INFO:tasks.workunit.client.0.smithi077.stdout:7/127: dwrite da/f11 [0,4194304] 0 2024-04-26T04:52:52.743 INFO:tasks.workunit.client.1.smithi078.stdout:3/416: sync 2024-04-26T04:52:52.744 INFO:tasks.workunit.client.1.smithi078.stdout:3/417: write d16/d26/d2b/d31/d71/f8c [909147,84142] 0 2024-04-26T04:52:52.745 INFO:tasks.workunit.client.1.smithi078.stdout:3/418: chown d16/d26/d2e/d74/f4f 1138430 1 2024-04-26T04:52:52.759 INFO:tasks.workunit.client.0.smithi077.stdout:4/158: sync 2024-04-26T04:52:52.760 INFO:tasks.workunit.client.0.smithi077.stdout:4/159: read d4/f27 [484372,122170] 0 2024-04-26T04:52:52.773 INFO:tasks.workunit.client.1.smithi078.stdout:1/480: rename d7/d2e/f5b to d7/d1e/d9e/f9f 0 2024-04-26T04:52:52.776 INFO:tasks.workunit.client.1.smithi078.stdout:8/407: fsync d0/d19/d52/f29 0 2024-04-26T04:52:52.778 INFO:tasks.workunit.client.1.smithi078.stdout:8/408: read d0/d19/d25/d36/d58/f70 [3473778,28816] 0 2024-04-26T04:52:52.778 INFO:tasks.workunit.client.1.smithi078.stdout:8/409: chown d0/l6f 158805 1 2024-04-26T04:52:52.779 INFO:tasks.workunit.client.1.smithi078.stdout:7/421: sync 2024-04-26T04:52:52.786 INFO:tasks.workunit.client.1.smithi078.stdout:5/334: creat d7/da/d1e/d4d/f6e x:0 0 0 2024-04-26T04:52:52.786 INFO:tasks.workunit.client.1.smithi078.stdout:5/335: chown d7/l17 3960 1 2024-04-26T04:52:52.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/160: rmdir d4/de/d15/d1f 39 2024-04-26T04:52:52.789 INFO:tasks.workunit.client.1.smithi078.stdout:1/481: rmdir d7 39 2024-04-26T04:52:52.791 INFO:tasks.workunit.client.1.smithi078.stdout:0/495: rename d2/d9/d60/d6a/f6d to d2/d39/d96/fa4 0 2024-04-26T04:52:52.792 INFO:tasks.workunit.client.0.smithi077.stdout:7/128: link da/d13/f19 da/d13/f30 0 2024-04-26T04:52:52.793 INFO:tasks.workunit.client.0.smithi077.stdout:1/173: write d9/db/dd/f15 [596323,89660] 0 2024-04-26T04:52:52.795 INFO:tasks.workunit.client.0.smithi077.stdout:4/161: symlink d4/d17/d18/l31 0 2024-04-26T04:52:52.797 INFO:tasks.workunit.client.0.smithi077.stdout:7/129: chown da/d15/f24 927 1 2024-04-26T04:52:52.798 INFO:tasks.workunit.client.0.smithi077.stdout:1/174: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:52.798 INFO:tasks.workunit.client.1.smithi078.stdout:8/410: mknod d0/d19/c7c 0 2024-04-26T04:52:52.799 INFO:tasks.workunit.client.0.smithi077.stdout:7/130: write da/d13/d22/f2b [1173919,81369] 0 2024-04-26T04:52:52.800 INFO:tasks.workunit.client.0.smithi077.stdout:7/131: dread da/d15/d1a/f20 [0,4194304] 0 2024-04-26T04:52:52.801 INFO:tasks.workunit.client.0.smithi077.stdout:7/132: dread - da/d13/d22/f25 zero size 2024-04-26T04:52:52.804 INFO:tasks.workunit.client.0.smithi077.stdout:1/175: mknod d9/db/dd/c32 0 2024-04-26T04:52:52.806 INFO:tasks.workunit.client.1.smithi078.stdout:5/336: creat d7/d48/f6f x:0 0 0 2024-04-26T04:52:52.810 INFO:tasks.workunit.client.0.smithi077.stdout:4/162: rename d4/de/d15/d1f/f23 to d4/de/d15/d1f/f32 0 2024-04-26T04:52:52.810 INFO:tasks.workunit.client.1.smithi078.stdout:5/337: dwrite d7/d18/d32/f4a [0,4194304] 0 2024-04-26T04:52:52.811 INFO:tasks.workunit.client.1.smithi078.stdout:5/338: chown d7/d18/d32/l6c 1003579 1 2024-04-26T04:52:52.812 INFO:tasks.workunit.client.1.smithi078.stdout:9/381: rename dc/df/d17/f48 to dc/df/d4b/d22/d2a/d57/f73 0 2024-04-26T04:52:52.812 INFO:tasks.workunit.client.1.smithi078.stdout:9/382: chown dc/df/f1e 334 1 2024-04-26T04:52:52.813 INFO:tasks.workunit.client.1.smithi078.stdout:9/383: stat dc/df/d27/l34 0 2024-04-26T04:52:52.813 INFO:tasks.workunit.client.0.smithi077.stdout:1/176: rename d9/db/c19 to d9/db/d27/c33 0 2024-04-26T04:52:52.820 INFO:tasks.workunit.client.0.smithi077.stdout:4/163: symlink d4/d17/d18/l33 0 2024-04-26T04:52:52.821 INFO:tasks.workunit.client.0.smithi077.stdout:1/177: dwrite d9/f21 [8388608,4194304] 0 2024-04-26T04:52:52.821 INFO:tasks.workunit.client.1.smithi078.stdout:7/422: mknod d9/da/d33/d4b/d4c/d5b/d83/c84 0 2024-04-26T04:52:52.825 INFO:tasks.workunit.client.1.smithi078.stdout:2/395: mkdir d8/d11/d18/d5d/d8b 0 2024-04-26T04:52:52.829 INFO:tasks.workunit.client.0.smithi077.stdout:4/164: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:52:52.831 INFO:tasks.workunit.client.0.smithi077.stdout:7/133: dwrite da/d15/f24 [0,4194304] 0 2024-04-26T04:52:52.831 INFO:tasks.workunit.client.0.smithi077.stdout:4/165: chown d4/de/f2b 1307 1 2024-04-26T04:52:52.835 INFO:tasks.workunit.client.1.smithi078.stdout:5/339: mkdir d7/d48/d57/d65/d70 0 2024-04-26T04:52:52.838 INFO:tasks.workunit.client.1.smithi078.stdout:3/419: rename f13 to d16/d3c/d81/f91 0 2024-04-26T04:52:52.840 INFO:tasks.workunit.client.0.smithi077.stdout:7/134: creat da/d13/f31 x:0 0 0 2024-04-26T04:52:52.842 INFO:tasks.workunit.client.1.smithi078.stdout:9/384: mknod dc/df/d37/c74 0 2024-04-26T04:52:52.843 INFO:tasks.workunit.client.1.smithi078.stdout:0/496: write d2/d9/d31/f4f [4551348,102937] 0 2024-04-26T04:52:52.843 INFO:tasks.workunit.client.0.smithi077.stdout:4/166: rmdir d4/de 39 2024-04-26T04:52:52.852 INFO:tasks.workunit.client.1.smithi078.stdout:4/447: rmdir d0/d25/d20/d31/d33/d44/d55/d5e/d9e 0 2024-04-26T04:52:52.853 INFO:tasks.workunit.client.0.smithi077.stdout:1/178: dwrite d9/db/f13 [0,4194304] 0 2024-04-26T04:52:52.854 INFO:tasks.workunit.client.1.smithi078.stdout:6/500: getdents d2/de/d27/d4c/d5f 0 2024-04-26T04:52:52.858 INFO:tasks.workunit.client.1.smithi078.stdout:5/340: fdatasync f3 0 2024-04-26T04:52:52.859 INFO:tasks.workunit.client.1.smithi078.stdout:5/341: dread - d7/d48/d57/d65/f67 zero size 2024-04-26T04:52:52.860 INFO:tasks.workunit.client.1.smithi078.stdout:0/497: mkdir d2/d9/d60/d6a/da5 0 2024-04-26T04:52:52.862 INFO:tasks.workunit.client.0.smithi077.stdout:1/179: mkdir d9/db/dd/d34 0 2024-04-26T04:52:52.863 INFO:tasks.workunit.client.1.smithi078.stdout:2/396: dread d8/f25 [0,4194304] 0 2024-04-26T04:52:52.863 INFO:tasks.workunit.client.0.smithi077.stdout:1/180: write d9/db/dd/f23 [1500842,3880] 0 2024-04-26T04:52:52.864 INFO:tasks.workunit.client.1.smithi078.stdout:3/420: dread f6 [0,4194304] 0 2024-04-26T04:52:52.865 INFO:tasks.workunit.client.1.smithi078.stdout:8/411: getdents d0/d19/d25/d36 0 2024-04-26T04:52:52.868 INFO:tasks.workunit.client.0.smithi077.stdout:7/135: write da/d13/d22/f2e [1375686,85090] 0 2024-04-26T04:52:52.870 INFO:tasks.workunit.client.1.smithi078.stdout:9/385: dread dc/df/f1e [0,4194304] 0 2024-04-26T04:52:52.871 INFO:tasks.workunit.client.0.smithi077.stdout:1/181: chown d9/db/d27/f28 486 1 2024-04-26T04:52:52.873 INFO:tasks.workunit.client.1.smithi078.stdout:5/342: creat d7/d18/d1c/f71 x:0 0 0 2024-04-26T04:52:52.873 INFO:tasks.workunit.client.0.smithi077.stdout:1/182: write d9/db/dd/f15 [1770916,75323] 0 2024-04-26T04:52:52.873 INFO:tasks.workunit.client.1.smithi078.stdout:0/498: truncate d2/ff 1626391 0 2024-04-26T04:52:52.876 INFO:tasks.workunit.client.1.smithi078.stdout:7/423: truncate d9/da/d11/d13/d50/f59 2255834 0 2024-04-26T04:52:52.876 INFO:tasks.workunit.client.1.smithi078.stdout:4/448: dwrite d0/f13 [4194304,4194304] 0 2024-04-26T04:52:52.876 INFO:tasks.workunit.client.0.smithi077.stdout:7/136: dwrite da/d15/f2a [0,4194304] 0 2024-04-26T04:52:52.879 INFO:tasks.workunit.client.1.smithi078.stdout:0/499: dread d2/d9/d18/f62 [0,4194304] 0 2024-04-26T04:52:52.880 INFO:tasks.workunit.client.1.smithi078.stdout:4/449: dread d0/de/f34 [4194304,4194304] 0 2024-04-26T04:52:52.881 INFO:tasks.workunit.client.1.smithi078.stdout:3/421: mknod d16/d26/d2b/d54/d5a/c92 0 2024-04-26T04:52:52.883 INFO:tasks.workunit.client.0.smithi077.stdout:7/137: creat da/d15/f32 x:0 0 0 2024-04-26T04:52:52.886 INFO:tasks.workunit.client.0.smithi077.stdout:1/183: symlink d9/db/dd/d34/l35 0 2024-04-26T04:52:52.887 INFO:tasks.workunit.client.0.smithi077.stdout:1/184: readlink d9/db/dd/l17 0 2024-04-26T04:52:52.887 INFO:tasks.workunit.client.0.smithi077.stdout:7/138: mknod da/d13/c33 0 2024-04-26T04:52:52.887 INFO:tasks.workunit.client.1.smithi078.stdout:3/422: dwrite d16/d26/d2b/d54/d5a/f5d [0,4194304] 0 2024-04-26T04:52:52.900 INFO:tasks.workunit.client.1.smithi078.stdout:2/397: dwrite d8/d11/d14/f67 [0,4194304] 0 2024-04-26T04:52:52.904 INFO:tasks.workunit.client.1.smithi078.stdout:6/501: creat d2/d1a/d5/d2a/f94 x:0 0 0 2024-04-26T04:52:52.906 INFO:tasks.workunit.client.1.smithi078.stdout:1/482: link d7/d1e/d9e/dd/d20/d53/c6b d7/ca0 0 2024-04-26T04:52:52.908 INFO:tasks.workunit.client.0.smithi077.stdout:1/185: write d9/f1b [2042925,42662] 0 2024-04-26T04:52:52.909 INFO:tasks.workunit.client.1.smithi078.stdout:8/412: write d0/f1a [8039125,834] 0 2024-04-26T04:52:52.910 INFO:tasks.workunit.client.1.smithi078.stdout:8/413: chown d0/c11 2179 1 2024-04-26T04:52:52.911 INFO:tasks.workunit.client.1.smithi078.stdout:5/343: mkdir d7/d48/d57/d65/d70/d72 0 2024-04-26T04:52:52.912 INFO:tasks.workunit.client.1.smithi078.stdout:5/344: dread - d7/d11/f6a zero size 2024-04-26T04:52:52.933 INFO:tasks.workunit.client.0.smithi077.stdout:2/89: sync 2024-04-26T04:52:52.934 INFO:tasks.workunit.client.0.smithi077.stdout:1/186: creat d9/db/d2b/f36 x:0 0 0 2024-04-26T04:52:52.934 INFO:tasks.workunit.client.0.smithi077.stdout:2/90: chown d9/fe 0 1 2024-04-26T04:52:52.935 INFO:tasks.workunit.client.1.smithi078.stdout:7/424: symlink d9/da/d11/l85 0 2024-04-26T04:52:52.938 INFO:tasks.workunit.client.0.smithi077.stdout:2/91: rename d9/dc/f19 to d9/d15/d1a/f1c 0 2024-04-26T04:52:52.938 INFO:tasks.workunit.client.1.smithi078.stdout:4/450: dread d0/de/d22/d27/d7b/f82 [0,4194304] 0 2024-04-26T04:52:52.939 INFO:tasks.workunit.client.1.smithi078.stdout:2/398: mknod d8/d11/d6e/c8c 0 2024-04-26T04:52:52.940 INFO:tasks.workunit.client.1.smithi078.stdout:6/502: mknod d2/d1a/d38/c95 0 2024-04-26T04:52:52.940 INFO:tasks.workunit.client.1.smithi078.stdout:7/425: dwrite d9/da/d11/f3a [0,4194304] 0 2024-04-26T04:52:52.940 INFO:tasks.workunit.client.1.smithi078.stdout:6/503: fsync d2/f53 0 2024-04-26T04:52:52.941 INFO:tasks.workunit.client.1.smithi078.stdout:6/504: chown d2/de 208 1 2024-04-26T04:52:52.943 INFO:tasks.workunit.client.1.smithi078.stdout:1/483: creat d7/d1e/d9e/d17/fa1 x:0 0 0 2024-04-26T04:52:52.944 INFO:tasks.workunit.client.0.smithi077.stdout:2/92: mkdir d9/dc/d1d 0 2024-04-26T04:52:52.945 INFO:tasks.workunit.client.1.smithi078.stdout:9/386: truncate dc/f26 892364 0 2024-04-26T04:52:52.951 INFO:tasks.workunit.client.1.smithi078.stdout:0/500: creat d2/d9/d31/d4a/fa6 x:0 0 0 2024-04-26T04:52:52.952 INFO:tasks.workunit.client.1.smithi078.stdout:0/501: fdatasync d2/d9/d18/d7d/fa0 0 2024-04-26T04:52:52.955 INFO:tasks.workunit.client.1.smithi078.stdout:4/451: rename d0/de/d22/d26 to d0/d25/d20/d31/d33/d4c/d62/daa 0 2024-04-26T04:52:52.955 INFO:tasks.workunit.client.1.smithi078.stdout:1/484: rename d7 to d7/d1e/d9e/d92/da2 22 2024-04-26T04:52:52.956 INFO:tasks.workunit.client.1.smithi078.stdout:7/426: creat d9/da/d11/d13/d50/f86 x:0 0 0 2024-04-26T04:52:52.956 INFO:tasks.workunit.client.1.smithi078.stdout:4/452: write d0/d25/d20/f2a [4397838,118868] 0 2024-04-26T04:52:52.957 INFO:tasks.workunit.client.1.smithi078.stdout:9/387: fsync dc/df/f1e 0 2024-04-26T04:52:52.959 INFO:tasks.workunit.client.1.smithi078.stdout:3/423: link d16/d26/d2b/f40 d16/d26/d2b/d31/f93 0 2024-04-26T04:52:52.962 INFO:tasks.workunit.client.1.smithi078.stdout:5/345: link f5 d7/da/f73 0 2024-04-26T04:52:52.963 INFO:tasks.workunit.client.1.smithi078.stdout:4/453: dwrite d0/d25/d20/d31/d33/d44/f68 [0,4194304] 0 2024-04-26T04:52:52.963 INFO:tasks.workunit.client.1.smithi078.stdout:8/414: mknod d0/d19/d25/c7d 0 2024-04-26T04:52:52.964 INFO:tasks.workunit.client.1.smithi078.stdout:9/388: dread dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:52:52.964 INFO:tasks.workunit.client.0.smithi077.stdout:2/93: dread d9/f17 [0,4194304] 0 2024-04-26T04:52:52.970 INFO:tasks.workunit.client.1.smithi078.stdout:4/454: dwrite d0/d25/d20/d31/d33/d4c/d62/daa/fa6 [0,4194304] 0 2024-04-26T04:52:52.970 INFO:tasks.workunit.client.1.smithi078.stdout:7/427: unlink d9/l79 0 2024-04-26T04:52:52.971 INFO:tasks.workunit.client.1.smithi078.stdout:4/455: write d0/d25/d20/fa9 [713391,4092] 0 2024-04-26T04:52:52.973 INFO:tasks.workunit.client.0.smithi077.stdout:2/94: rename d9/d11 to d9/dc/d1d/d1e 0 2024-04-26T04:52:52.974 INFO:tasks.workunit.client.0.smithi077.stdout:1/187: truncate d9/db/f1c 1406232 0 2024-04-26T04:52:52.981 INFO:tasks.workunit.client.1.smithi078.stdout:2/399: write d8/d11/d14/f40 [4599872,13478] 0 2024-04-26T04:52:52.982 INFO:tasks.workunit.client.0.smithi077.stdout:1/188: readlink d9/db/l1a 0 2024-04-26T04:52:52.982 INFO:tasks.workunit.client.0.smithi077.stdout:1/189: chown d9/db/dd 2950 1 2024-04-26T04:52:52.986 INFO:tasks.workunit.client.1.smithi078.stdout:3/424: symlink d16/d26/d2b/d54/l94 0 2024-04-26T04:52:52.986 INFO:tasks.workunit.client.1.smithi078.stdout:3/425: write d16/d26/d2b/d54/f68 [358009,28018] 0 2024-04-26T04:52:52.992 INFO:tasks.workunit.client.1.smithi078.stdout:3/426: dwrite d16/f8d [0,4194304] 0 2024-04-26T04:52:52.994 INFO:tasks.workunit.client.1.smithi078.stdout:3/427: read d16/d26/d2e/d74/f4f [770567,72343] 0 2024-04-26T04:52:52.997 INFO:tasks.workunit.client.0.smithi077.stdout:1/190: dread d9/db/f1c [0,4194304] 0 2024-04-26T04:52:52.998 INFO:tasks.workunit.client.1.smithi078.stdout:4/456: fsync d0/d25/d20/f1b 0 2024-04-26T04:52:53.000 INFO:tasks.workunit.client.0.smithi077.stdout:2/95: write f5 [2569593,38064] 0 2024-04-26T04:52:53.001 INFO:tasks.workunit.client.1.smithi078.stdout:6/505: link d2/d49/l83 d2/d49/l96 0 2024-04-26T04:52:53.003 INFO:tasks.workunit.client.0.smithi077.stdout:1/191: rmdir d9/db 39 2024-04-26T04:52:53.003 INFO:tasks.workunit.client.0.smithi077.stdout:2/96: dread d9/f17 [0,4194304] 0 2024-04-26T04:52:53.007 INFO:tasks.workunit.client.1.smithi078.stdout:2/400: fdatasync d8/d11/d18/d5d/d61/f32 0 2024-04-26T04:52:53.010 INFO:tasks.workunit.client.1.smithi078.stdout:9/389: fdatasync f1 0 2024-04-26T04:52:53.011 INFO:tasks.workunit.client.0.smithi077.stdout:2/97: creat d9/dc/d1d/f1f x:0 0 0 2024-04-26T04:52:53.012 INFO:tasks.workunit.client.0.smithi077.stdout:2/98: dread - d9/d15/d1a/f1c zero size 2024-04-26T04:52:53.012 INFO:tasks.workunit.client.0.smithi077.stdout:2/99: readlink l6 0 2024-04-26T04:52:53.013 INFO:tasks.workunit.client.1.smithi078.stdout:5/346: getdents d7/d48/d57/d65/d70/d72 0 2024-04-26T04:52:53.013 INFO:tasks.workunit.client.0.smithi077.stdout:2/100: chown d9/d15/d1a/f1c 0 1 2024-04-26T04:52:53.014 INFO:tasks.workunit.client.0.smithi077.stdout:2/101: read d9/fd [1191803,21244] 0 2024-04-26T04:52:53.014 INFO:tasks.workunit.client.1.smithi078.stdout:0/502: link d2/d9/f3e d2/d9/d31/d73/fa7 0 2024-04-26T04:52:53.015 INFO:tasks.workunit.client.1.smithi078.stdout:0/503: fdatasync d2/d9/d60/f68 0 2024-04-26T04:52:53.016 INFO:tasks.workunit.client.1.smithi078.stdout:0/504: write d2/d39/f9d [254670,22274] 0 2024-04-26T04:52:53.016 INFO:tasks.workunit.client.0.smithi077.stdout:1/192: symlink d9/db/l37 0 2024-04-26T04:52:53.017 INFO:tasks.workunit.client.1.smithi078.stdout:0/505: write d2/d9/d31/d4a/fa6 [943395,74643] 0 2024-04-26T04:52:53.018 INFO:tasks.workunit.client.0.smithi077.stdout:2/102: dread d9/fd [0,4194304] 0 2024-04-26T04:52:53.019 INFO:tasks.workunit.client.1.smithi078.stdout:3/428: creat d16/d26/d2e/d5f/d49/d77/f95 x:0 0 0 2024-04-26T04:52:53.020 INFO:tasks.workunit.client.1.smithi078.stdout:3/429: chown d16/d26/d2e/l44 94 1 2024-04-26T04:52:53.020 INFO:tasks.workunit.client.0.smithi077.stdout:1/193: truncate d9/f20 552676 0 2024-04-26T04:52:53.021 INFO:tasks.workunit.client.1.smithi078.stdout:3/430: write d16/d26/d2e/d74/f76 [2097078,69807] 0 2024-04-26T04:52:53.022 INFO:tasks.workunit.client.0.smithi077.stdout:2/103: mknod d9/d15/d1a/c20 0 2024-04-26T04:52:53.022 INFO:tasks.workunit.client.1.smithi078.stdout:3/431: chown f9 394745745 1 2024-04-26T04:52:53.022 INFO:tasks.workunit.client.1.smithi078.stdout:0/506: dwrite d2/f1d [0,4194304] 0 2024-04-26T04:52:53.023 INFO:tasks.workunit.client.1.smithi078.stdout:0/507: fdatasync d2/d9/d18/f22 0 2024-04-26T04:52:53.024 INFO:tasks.workunit.client.1.smithi078.stdout:4/457: mkdir d0/d25/d20/d31/d33/d4c/d62/daa/d2e/d85/dab 0 2024-04-26T04:52:53.088 INFO:tasks.workunit.client.0.smithi077.stdout:2/104: creat d9/dc/d1d/d1e/f21 x:0 0 0 2024-04-26T04:52:53.088 INFO:tasks.workunit.client.1.smithi078.stdout:7/428: truncate d9/da/d33/d4b/d4c/f55 1506017 0 2024-04-26T04:52:53.089 INFO:tasks.workunit.client.1.smithi078.stdout:7/429: write d9/f10 [5552447,83057] 0 2024-04-26T04:52:53.090 INFO:tasks.workunit.client.1.smithi078.stdout:7/430: chown d9/da/d20/c32 29 1 2024-04-26T04:52:53.091 INFO:tasks.workunit.client.0.smithi077.stdout:2/105: creat d9/d15/d1a/f22 x:0 0 0 2024-04-26T04:52:53.092 INFO:tasks.workunit.client.0.smithi077.stdout:2/106: write d9/d15/d1a/f22 [211302,123394] 0 2024-04-26T04:52:53.094 INFO:tasks.workunit.client.1.smithi078.stdout:9/390: mknod dc/df/c75 0 2024-04-26T04:52:53.099 INFO:tasks.workunit.client.0.smithi077.stdout:2/107: fsync d9/fd 0 2024-04-26T04:52:53.099 INFO:tasks.workunit.client.0.smithi077.stdout:2/108: rename d9/dc to d9/dc/d1d/d1e/d23 22 2024-04-26T04:52:53.100 INFO:tasks.workunit.client.1.smithi078.stdout:1/485: getdents d7/d2e 0 2024-04-26T04:52:53.101 INFO:tasks.workunit.client.0.smithi077.stdout:1/194: getdents d9/db/d2b 0 2024-04-26T04:52:53.102 INFO:tasks.workunit.client.1.smithi078.stdout:3/432: mknod d16/d3c/d81/c96 0 2024-04-26T04:52:53.103 INFO:tasks.workunit.client.0.smithi077.stdout:1/195: write d9/db/d2b/f36 [905152,105334] 0 2024-04-26T04:52:53.103 INFO:tasks.workunit.client.1.smithi078.stdout:3/433: stat d16/d26/d2e/f73 0 2024-04-26T04:52:53.104 INFO:tasks.workunit.client.1.smithi078.stdout:0/508: mkdir d2/d9/d60/d6a/d8f/da8 0 2024-04-26T04:52:53.105 INFO:tasks.workunit.client.1.smithi078.stdout:0/509: write d2/d1e/f9e [290178,86253] 0 2024-04-26T04:52:53.106 INFO:tasks.workunit.client.1.smithi078.stdout:7/431: symlink d9/da/d11/d13/d50/l87 0 2024-04-26T04:52:53.108 INFO:tasks.workunit.client.1.smithi078.stdout:7/432: write d9/f10 [3596581,79721] 0 2024-04-26T04:52:53.108 INFO:tasks.workunit.client.1.smithi078.stdout:5/347: unlink d7/d18/d32/c5e 0 2024-04-26T04:52:53.109 INFO:tasks.workunit.client.1.smithi078.stdout:8/415: getdents d0/d19/d52/d3c 0 2024-04-26T04:52:53.111 INFO:tasks.workunit.client.1.smithi078.stdout:2/401: creat d8/d11/d14/f8d x:0 0 0 2024-04-26T04:52:53.111 INFO:tasks.workunit.client.1.smithi078.stdout:0/510: mkdir d2/d9/d18/d7d/da9 0 2024-04-26T04:52:53.112 INFO:tasks.workunit.client.0.smithi077.stdout:6/126: sync 2024-04-26T04:52:53.113 INFO:tasks.workunit.client.1.smithi078.stdout:6/506: getdents d2/de 0 2024-04-26T04:52:53.114 INFO:tasks.workunit.client.1.smithi078.stdout:6/507: chown d2/f14 251 1 2024-04-26T04:52:53.114 INFO:tasks.workunit.client.0.smithi077.stdout:2/109: creat d9/dc/d1d/f24 x:0 0 0 2024-04-26T04:52:53.115 INFO:tasks.workunit.client.1.smithi078.stdout:6/508: write d2/d1a/d5c/f6d [786160,31873] 0 2024-04-26T04:52:53.115 INFO:tasks.workunit.client.1.smithi078.stdout:6/509: fsync d2/d1a/f43 0 2024-04-26T04:52:53.117 INFO:tasks.workunit.client.1.smithi078.stdout:4/458: rename d0/d25/d20/d31/d33/d44/l94 to d0/de/d22/lac 0 2024-04-26T04:52:53.117 INFO:tasks.workunit.client.1.smithi078.stdout:3/434: dread d16/d26/d2e/d5f/d49/d77/f7b [0,4194304] 0 2024-04-26T04:52:53.117 INFO:tasks.workunit.client.1.smithi078.stdout:6/510: truncate d2/de/f67 964214 0 2024-04-26T04:52:53.118 INFO:tasks.workunit.client.0.smithi077.stdout:1/196: dread d9/f21 [4194304,4194304] 0 2024-04-26T04:52:53.118 INFO:tasks.workunit.client.1.smithi078.stdout:7/433: mknod d9/da/d11/d13/d1a/c88 0 2024-04-26T04:52:53.120 INFO:tasks.workunit.client.1.smithi078.stdout:1/486: dread d7/d1e/d9e/db/d37/f65 [0,4194304] 0 2024-04-26T04:52:53.121 INFO:tasks.workunit.client.0.smithi077.stdout:6/127: creat d8/d1c/f2f x:0 0 0 2024-04-26T04:52:53.122 INFO:tasks.workunit.client.1.smithi078.stdout:5/348: mknod d7/da/d1e/d4d/c74 0 2024-04-26T04:52:53.123 INFO:tasks.workunit.client.0.smithi077.stdout:2/110: creat d9/dc/d1d/f25 x:0 0 0 2024-04-26T04:52:53.125 INFO:tasks.workunit.client.0.smithi077.stdout:2/111: truncate d9/d15/d1a/f22 1191526 0 2024-04-26T04:52:53.126 INFO:tasks.workunit.client.1.smithi078.stdout:2/402: dread d8/d11/d14/f3b [4194304,4194304] 0 2024-04-26T04:52:53.127 INFO:tasks.workunit.client.1.smithi078.stdout:9/391: rmdir dc/df/d4b/d3e 39 2024-04-26T04:52:53.128 INFO:tasks.workunit.client.0.smithi077.stdout:1/197: dread d9/db/f13 [0,4194304] 0 2024-04-26T04:52:53.129 INFO:tasks.workunit.client.0.smithi077.stdout:1/198: chown d9/f24 124958 1 2024-04-26T04:52:53.129 INFO:tasks.workunit.client.1.smithi078.stdout:0/511: rmdir d2/d9/d60 39 2024-04-26T04:52:53.131 INFO:tasks.workunit.client.0.smithi077.stdout:6/128: symlink d8/d2a/l30 0 2024-04-26T04:52:53.134 INFO:tasks.workunit.client.0.smithi077.stdout:6/129: dread d8/d1c/f24 [0,4194304] 0 2024-04-26T04:52:53.137 INFO:tasks.workunit.client.0.smithi077.stdout:1/199: stat d9/db/d27/f30 0 2024-04-26T04:52:53.138 INFO:tasks.workunit.client.0.smithi077.stdout:2/112: dread d9/f17 [0,4194304] 0 2024-04-26T04:52:53.142 INFO:tasks.workunit.client.0.smithi077.stdout:6/130: rmdir d8/d1c 39 2024-04-26T04:52:53.144 INFO:tasks.workunit.client.1.smithi078.stdout:5/349: unlink d7/d11/f2c 0 2024-04-26T04:52:53.145 INFO:tasks.workunit.client.1.smithi078.stdout:5/350: chown d7/f8 479110579 1 2024-04-26T04:52:53.146 INFO:tasks.workunit.client.0.smithi077.stdout:1/200: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:53.147 INFO:tasks.workunit.client.0.smithi077.stdout:2/113: fsync d9/f12 0 2024-04-26T04:52:53.148 INFO:tasks.workunit.client.0.smithi077.stdout:1/201: write d9/f24 [1833423,48518] 0 2024-04-26T04:52:53.148 INFO:tasks.workunit.client.1.smithi078.stdout:2/403: mkdir d8/d3e/d8e 0 2024-04-26T04:52:53.150 INFO:tasks.workunit.client.1.smithi078.stdout:9/392: read dc/df/f1e [940427,108427] 0 2024-04-26T04:52:53.151 INFO:tasks.workunit.client.1.smithi078.stdout:3/435: dread d16/d26/f2c [0,4194304] 0 2024-04-26T04:52:53.158 INFO:tasks.workunit.client.0.smithi077.stdout:1/202: readlink d9/db/l1a 0 2024-04-26T04:52:53.160 INFO:tasks.workunit.client.0.smithi077.stdout:6/131: unlink d8/d1c/f2f 0 2024-04-26T04:52:53.162 INFO:tasks.workunit.client.1.smithi078.stdout:7/434: truncate d9/da/d11/d13/d1a/f1e 2177598 0 2024-04-26T04:52:53.165 INFO:tasks.workunit.client.1.smithi078.stdout:1/487: truncate d7/d1e/d9e/dd/d20/f40 153312 0 2024-04-26T04:52:53.166 INFO:tasks.workunit.client.1.smithi078.stdout:5/351: creat d7/da/f75 x:0 0 0 2024-04-26T04:52:53.166 INFO:tasks.workunit.client.1.smithi078.stdout:5/352: dread - d7/d48/d57/d65/f67 zero size 2024-04-26T04:52:53.167 INFO:tasks.workunit.client.0.smithi077.stdout:1/203: chown d9/db/dd/l1f 36611303 1 2024-04-26T04:52:53.167 INFO:tasks.workunit.client.0.smithi077.stdout:1/204: fsync d9/db/d27/f28 0 2024-04-26T04:52:53.168 INFO:tasks.workunit.client.1.smithi078.stdout:9/393: write dc/df/d37/d64/f66 [686175,70673] 0 2024-04-26T04:52:53.168 INFO:tasks.workunit.client.0.smithi077.stdout:6/132: rename d8/d1c/l1d to d8/d1c/l31 0 2024-04-26T04:52:53.169 INFO:tasks.workunit.client.0.smithi077.stdout:1/205: write d9/db/dd/f2f [2484714,101393] 0 2024-04-26T04:52:53.169 INFO:tasks.workunit.client.0.smithi077.stdout:1/206: stat d9 0 2024-04-26T04:52:53.170 INFO:tasks.workunit.client.1.smithi078.stdout:0/512: mkdir d2/d9/d60/d6a/daa 0 2024-04-26T04:52:53.172 INFO:tasks.workunit.client.1.smithi078.stdout:4/459: rename d0/de/d22/d27/d56/d5a/fa7 to d0/de/fad 0 2024-04-26T04:52:53.175 INFO:tasks.workunit.client.1.smithi078.stdout:8/416: link d0/d19/d25/d36/l54 d0/d19/d52/d35/d44/l7e 0 2024-04-26T04:52:53.176 INFO:tasks.workunit.client.0.smithi077.stdout:6/133: fdatasync f3 0 2024-04-26T04:52:53.179 INFO:tasks.workunit.client.0.smithi077.stdout:2/114: getdents d9/d15 0 2024-04-26T04:52:53.181 INFO:tasks.workunit.client.0.smithi077.stdout:1/207: chown d9/db/f1c 27006 1 2024-04-26T04:52:53.183 INFO:tasks.workunit.client.0.smithi077.stdout:1/208: dread - d9/db/d27/f30 zero size 2024-04-26T04:52:53.183 INFO:tasks.workunit.client.0.smithi077.stdout:6/134: dwrite d8/d2a/f2e [0,4194304] 0 2024-04-26T04:52:53.185 INFO:tasks.workunit.client.0.smithi077.stdout:6/135: chown c1 52582 1 2024-04-26T04:52:53.185 INFO:tasks.workunit.client.0.smithi077.stdout:1/209: chown d9/db/d2b/f36 25076 1 2024-04-26T04:52:53.186 INFO:tasks.workunit.client.0.smithi077.stdout:1/210: write f2 [3957583,44891] 0 2024-04-26T04:52:53.187 INFO:tasks.workunit.client.1.smithi078.stdout:5/353: rmdir d7/d48/d57 39 2024-04-26T04:52:53.187 INFO:tasks.workunit.client.1.smithi078.stdout:7/435: dread d9/da/d11/d13/d1a/d52/d5a/d41/d68/f7b [4194304,4194304] 0 2024-04-26T04:52:53.188 INFO:tasks.workunit.client.0.smithi077.stdout:5/100: sync 2024-04-26T04:52:53.188 INFO:tasks.workunit.client.0.smithi077.stdout:8/136: sync 2024-04-26T04:52:53.188 INFO:tasks.workunit.client.1.smithi078.stdout:7/436: fsync d9/f10 0 2024-04-26T04:52:53.188 INFO:tasks.workunit.client.0.smithi077.stdout:2/115: mknod d9/d15/d1a/c26 0 2024-04-26T04:52:53.190 INFO:tasks.workunit.client.1.smithi078.stdout:7/437: write d9/da/f23 [4889000,13768] 0 2024-04-26T04:52:53.191 INFO:tasks.workunit.client.1.smithi078.stdout:7/438: fdatasync d9/da/d11/d13/d1a/d2d/f3d 0 2024-04-26T04:52:53.192 INFO:tasks.workunit.client.1.smithi078.stdout:3/436: link d16/d26/d2b/d54/f68 d16/d26/d2e/d74/f97 0 2024-04-26T04:52:53.193 INFO:tasks.workunit.client.1.smithi078.stdout:2/404: truncate d8/d11/d14/f19 1029181 0 2024-04-26T04:52:53.195 INFO:tasks.workunit.client.0.smithi077.stdout:8/137: stat d4/c14 0 2024-04-26T04:52:53.196 INFO:tasks.workunit.client.1.smithi078.stdout:3/437: read d16/d26/d2b/d31/f34 [239587,19612] 0 2024-04-26T04:52:53.197 INFO:tasks.workunit.client.1.smithi078.stdout:6/511: rename d2/d1a/d5/da/d36/d4b to d2/d49/d97 0 2024-04-26T04:52:53.198 INFO:tasks.workunit.client.1.smithi078.stdout:3/438: chown d16/d26/d2e/d5f/d49/c63 1182 1 2024-04-26T04:52:53.198 INFO:tasks.workunit.client.0.smithi077.stdout:5/101: creat d1/d8/f22 x:0 0 0 2024-04-26T04:52:53.199 INFO:tasks.workunit.client.1.smithi078.stdout:4/460: mkdir d0/de/d22/d71/dae 0 2024-04-26T04:52:53.199 INFO:tasks.workunit.client.0.smithi077.stdout:2/116: mknod d9/dc/d1d/c27 0 2024-04-26T04:52:53.199 INFO:tasks.workunit.client.0.smithi077.stdout:5/102: write d1/d12/d21/d18/f1c [455639,78239] 0 2024-04-26T04:52:53.201 INFO:tasks.workunit.client.0.smithi077.stdout:6/136: write d8/d1c/f25 [2849,15335] 0 2024-04-26T04:52:53.201 INFO:tasks.workunit.client.1.smithi078.stdout:8/417: unlink d0/d19/d52/d35/d44/c50 0 2024-04-26T04:52:53.202 INFO:tasks.workunit.client.1.smithi078.stdout:6/512: dread d2/d49/f57 [4194304,4194304] 0 2024-04-26T04:52:53.203 INFO:tasks.workunit.client.1.smithi078.stdout:1/488: fdatasync d7/f8e 0 2024-04-26T04:52:53.205 INFO:tasks.workunit.client.1.smithi078.stdout:5/354: creat d7/d11/f76 x:0 0 0 2024-04-26T04:52:53.206 INFO:tasks.workunit.client.0.smithi077.stdout:5/103: mknod d1/d12/d21/c23 0 2024-04-26T04:52:53.210 INFO:tasks.workunit.client.0.smithi077.stdout:5/104: dread d1/d8/f10 [0,4194304] 0 2024-04-26T04:52:53.211 INFO:tasks.workunit.client.1.smithi078.stdout:9/394: dread dc/df/f2f [0,4194304] 0 2024-04-26T04:52:53.217 INFO:tasks.workunit.client.1.smithi078.stdout:0/513: dwrite d2/d9/d31/d4a/fa2 [0,4194304] 0 2024-04-26T04:52:53.218 INFO:tasks.workunit.client.0.smithi077.stdout:5/105: dwrite d1/d12/d21/d18/f1c [0,4194304] 0 2024-04-26T04:52:53.220 INFO:tasks.workunit.client.0.smithi077.stdout:1/211: getdents d9/db/dd 0 2024-04-26T04:52:53.224 INFO:tasks.workunit.client.1.smithi078.stdout:2/405: chown d8/d11/f31 45 1 2024-04-26T04:52:53.225 INFO:tasks.workunit.client.0.smithi077.stdout:2/117: link d9/d15/d1a/c20 d9/dc/d1d/d1e/c28 0 2024-04-26T04:52:53.226 INFO:tasks.workunit.client.1.smithi078.stdout:2/406: truncate d8/d11/d6e/f7b 939571 0 2024-04-26T04:52:53.228 INFO:tasks.workunit.client.0.smithi077.stdout:5/106: mknod d1/d8/c24 0 2024-04-26T04:52:53.228 INFO:tasks.workunit.client.0.smithi077.stdout:5/107: read - d1/d12/f13 zero size 2024-04-26T04:52:53.229 INFO:tasks.workunit.client.0.smithi077.stdout:6/137: write d8/f11 [5019117,103824] 0 2024-04-26T04:52:53.231 INFO:tasks.workunit.client.0.smithi077.stdout:6/138: chown d8/f22 27561 1 2024-04-26T04:52:53.231 INFO:tasks.workunit.client.0.smithi077.stdout:1/212: creat d9/db/dd/d34/f38 x:0 0 0 2024-04-26T04:52:53.234 INFO:tasks.workunit.client.1.smithi078.stdout:8/418: rename d0/f18 to d0/d19/d25/d36/f7f 0 2024-04-26T04:52:53.236 INFO:tasks.workunit.client.0.smithi077.stdout:8/138: dread f0 [0,4194304] 0 2024-04-26T04:52:53.236 INFO:tasks.workunit.client.0.smithi077.stdout:5/108: dread d1/d12/d21/d18/f1c [0,4194304] 0 2024-04-26T04:52:53.236 INFO:tasks.workunit.client.1.smithi078.stdout:6/513: fsync d2/d1a/d5/d11/f4e 0 2024-04-26T04:52:53.238 INFO:tasks.workunit.client.1.smithi078.stdout:1/489: unlink d7/d1e/d9e/c4d 0 2024-04-26T04:52:53.239 INFO:tasks.workunit.client.0.smithi077.stdout:8/139: write d4/df/d22/d2d/f1c [744999,71418] 0 2024-04-26T04:52:53.240 INFO:tasks.workunit.client.1.smithi078.stdout:5/355: mknod d7/d18/d1c/d5a/c77 0 2024-04-26T04:52:53.240 INFO:tasks.workunit.client.0.smithi077.stdout:1/213: dwrite f6 [0,4194304] 0 2024-04-26T04:52:53.242 INFO:tasks.workunit.client.0.smithi077.stdout:6/139: link d8/d1c/f23 d8/d2a/f32 0 2024-04-26T04:52:53.242 INFO:tasks.workunit.client.0.smithi077.stdout:5/109: dread d1/d8/f10 [0,4194304] 0 2024-04-26T04:52:53.250 INFO:tasks.workunit.client.0.smithi077.stdout:8/140: symlink d4/df/d22/d2d/d2b/l31 0 2024-04-26T04:52:53.251 INFO:tasks.workunit.client.0.smithi077.stdout:2/118: write f5 [890355,75272] 0 2024-04-26T04:52:53.253 INFO:tasks.workunit.client.1.smithi078.stdout:3/439: dwrite d16/f69 [0,4194304] 0 2024-04-26T04:52:53.256 INFO:tasks.workunit.client.0.smithi077.stdout:5/110: mkdir d1/d12/d21/d25 0 2024-04-26T04:52:53.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/141: creat d4/df/f32 x:0 0 0 2024-04-26T04:52:53.261 INFO:tasks.workunit.client.0.smithi077.stdout:9/107: sync 2024-04-26T04:52:53.261 INFO:tasks.workunit.client.0.smithi077.stdout:3/106: sync 2024-04-26T04:52:53.262 INFO:tasks.workunit.client.0.smithi077.stdout:0/88: sync 2024-04-26T04:52:53.262 INFO:tasks.workunit.client.0.smithi077.stdout:4/167: sync 2024-04-26T04:52:53.262 INFO:tasks.workunit.client.0.smithi077.stdout:7/139: sync 2024-04-26T04:52:53.262 INFO:tasks.workunit.client.0.smithi077.stdout:0/89: fsync d0/f4 0 2024-04-26T04:52:53.264 INFO:tasks.workunit.client.0.smithi077.stdout:2/119: mknod d9/dc/d1d/c29 0 2024-04-26T04:52:53.267 INFO:tasks.workunit.client.0.smithi077.stdout:0/90: dread d0/f12 [0,4194304] 0 2024-04-26T04:52:53.268 INFO:tasks.workunit.client.0.smithi077.stdout:0/91: readlink d0/l6 0 2024-04-26T04:52:53.269 INFO:tasks.workunit.client.0.smithi077.stdout:7/140: dread da/ff [0,4194304] 0 2024-04-26T04:52:53.269 INFO:tasks.workunit.client.1.smithi078.stdout:2/407: rename d8/d11/d18/d5d/d61/d1a to d8/d11/d14/d69/d8f 0 2024-04-26T04:52:53.270 INFO:tasks.workunit.client.0.smithi077.stdout:1/214: creat d9/db/f39 x:0 0 0 2024-04-26T04:52:53.270 INFO:tasks.workunit.client.0.smithi077.stdout:1/215: readlink d9/db/l37 0 2024-04-26T04:52:53.271 INFO:tasks.workunit.client.1.smithi078.stdout:8/419: unlink d0/cc 0 2024-04-26T04:52:53.276 INFO:tasks.workunit.client.0.smithi077.stdout:4/168: rmdir d4 39 2024-04-26T04:52:53.282 INFO:tasks.workunit.client.0.smithi077.stdout:9/108: mkdir d9/dd/d28 0 2024-04-26T04:52:53.282 INFO:tasks.workunit.client.0.smithi077.stdout:9/109: chown f7 3652991 1 2024-04-26T04:52:53.283 INFO:tasks.workunit.client.0.smithi077.stdout:9/110: chown d9/la 6615 1 2024-04-26T04:52:53.284 INFO:tasks.workunit.client.1.smithi078.stdout:9/395: unlink dc/df/d17/c61 0 2024-04-26T04:52:53.284 INFO:tasks.workunit.client.0.smithi077.stdout:3/107: creat d0/d9/f25 x:0 0 0 2024-04-26T04:52:53.285 INFO:tasks.workunit.client.1.smithi078.stdout:0/514: mkdir d2/d9/d18/d7d/da9/dab 0 2024-04-26T04:52:53.285 INFO:tasks.workunit.client.1.smithi078.stdout:8/420: dread d0/d19/d52/d35/d3f/f57 [0,4194304] 0 2024-04-26T04:52:53.288 INFO:tasks.workunit.client.0.smithi077.stdout:6/140: creat d8/f33 x:0 0 0 2024-04-26T04:52:53.289 INFO:tasks.workunit.client.1.smithi078.stdout:7/439: link d9/da/d11/l69 d9/l89 0 2024-04-26T04:52:53.289 INFO:tasks.workunit.client.0.smithi077.stdout:6/141: dread - d8/d2a/f32 zero size 2024-04-26T04:52:53.290 INFO:tasks.workunit.client.1.smithi078.stdout:7/440: dread - d9/f58 zero size 2024-04-26T04:52:53.290 INFO:tasks.workunit.client.0.smithi077.stdout:5/111: symlink d1/d12/d21/d25/l26 0 2024-04-26T04:52:53.291 INFO:tasks.workunit.client.1.smithi078.stdout:8/421: dwrite d0/d19/d52/d35/d3f/f5e [0,4194304] 0 2024-04-26T04:52:53.291 INFO:tasks.workunit.client.1.smithi078.stdout:7/441: write d9/da/d11/d13/f25 [504157,76795] 0 2024-04-26T04:52:53.293 INFO:tasks.workunit.client.1.smithi078.stdout:7/442: chown d9/da/d11/d13/c48 5 1 2024-04-26T04:52:53.293 INFO:tasks.workunit.client.0.smithi077.stdout:0/92: rmdir d0/db 39 2024-04-26T04:52:53.295 INFO:tasks.workunit.client.0.smithi077.stdout:7/141: fdatasync da/d13/f19 0 2024-04-26T04:52:53.300 INFO:tasks.workunit.client.0.smithi077.stdout:5/112: dwrite d1/d12/f19 [0,4194304] 0 2024-04-26T04:52:53.300 INFO:tasks.workunit.client.0.smithi077.stdout:5/113: dread - d1/d8/fc zero size 2024-04-26T04:52:53.302 INFO:tasks.workunit.client.1.smithi078.stdout:6/514: unlink d2/d52/c6a 0 2024-04-26T04:52:53.304 INFO:tasks.workunit.client.1.smithi078.stdout:6/515: read d2/de/f33 [639914,51322] 0 2024-04-26T04:52:53.305 INFO:tasks.workunit.client.1.smithi078.stdout:6/516: chown d2/d1a/d5/d3a/d69/c86 946978 1 2024-04-26T04:52:53.306 INFO:tasks.workunit.client.1.smithi078.stdout:6/517: truncate d2/d52/f7a 1516535 0 2024-04-26T04:52:53.309 INFO:tasks.workunit.client.0.smithi077.stdout:9/111: rename d9/la to d9/dd/l29 0 2024-04-26T04:52:53.311 INFO:tasks.workunit.client.0.smithi077.stdout:3/108: creat d0/d9/f26 x:0 0 0 2024-04-26T04:52:53.311 INFO:tasks.workunit.client.0.smithi077.stdout:3/109: read - d0/f1d zero size 2024-04-26T04:52:53.312 INFO:tasks.workunit.client.0.smithi077.stdout:3/110: chown d0/f1b 22 1 2024-04-26T04:52:53.314 INFO:tasks.workunit.client.1.smithi078.stdout:3/440: mkdir d16/d26/d2b/d31/d71/d8a/d98 0 2024-04-26T04:52:53.315 INFO:tasks.workunit.client.0.smithi077.stdout:2/120: mknod d9/c2a 0 2024-04-26T04:52:53.316 INFO:tasks.workunit.client.1.smithi078.stdout:4/461: getdents d0/d25/d20 0 2024-04-26T04:52:53.319 INFO:tasks.workunit.client.0.smithi077.stdout:8/142: write d4/f9 [4988157,4661] 0 2024-04-26T04:52:53.320 INFO:tasks.workunit.client.1.smithi078.stdout:7/443: mkdir d9/da/d11/d13/d50/d8a 0 2024-04-26T04:52:53.322 INFO:tasks.workunit.client.0.smithi077.stdout:6/142: creat d8/d2a/f34 x:0 0 0 2024-04-26T04:52:53.322 INFO:tasks.workunit.client.0.smithi077.stdout:1/216: dwrite d9/f20 [0,4194304] 0 2024-04-26T04:52:53.328 INFO:tasks.workunit.client.1.smithi078.stdout:0/515: dwrite d2/d9/f3e [0,4194304] 0 2024-04-26T04:52:53.328 INFO:tasks.workunit.client.1.smithi078.stdout:5/356: link d7/d18/d1c/d5a/c77 d7/d21/c78 0 2024-04-26T04:52:53.329 INFO:tasks.workunit.client.0.smithi077.stdout:1/217: dread d9/db/dd/f15 [0,4194304] 0 2024-04-26T04:52:53.329 INFO:tasks.workunit.client.0.smithi077.stdout:7/142: rmdir da 39 2024-04-26T04:52:53.333 INFO:tasks.workunit.client.1.smithi078.stdout:9/396: creat dc/df/d27/f76 x:0 0 0 2024-04-26T04:52:53.341 INFO:tasks.workunit.client.1.smithi078.stdout:4/462: creat d0/d1d/faf x:0 0 0 2024-04-26T04:52:53.342 INFO:tasks.workunit.client.0.smithi077.stdout:3/111: fsync d0/fb 0 2024-04-26T04:52:53.343 INFO:tasks.workunit.client.0.smithi077.stdout:2/121: fsync d9/f10 0 2024-04-26T04:52:53.345 INFO:tasks.workunit.client.1.smithi078.stdout:7/444: symlink d9/da/d11/d13/d1a/d52/d5a/d41/d68/l8b 0 2024-04-26T04:52:53.347 INFO:tasks.workunit.client.1.smithi078.stdout:1/490: rename d7/d1e/d9e/d4c/d3d/c77 to d7/d1e/d9e/dd/ca3 0 2024-04-26T04:52:53.347 INFO:tasks.workunit.client.0.smithi077.stdout:8/143: mkdir d4/df/d22/d2d/d2b/d33 0 2024-04-26T04:52:53.349 INFO:tasks.workunit.client.0.smithi077.stdout:6/143: fdatasync d8/fe 0 2024-04-26T04:52:53.351 INFO:tasks.workunit.client.1.smithi078.stdout:0/516: symlink d2/d9/d60/lac 0 2024-04-26T04:52:53.351 INFO:tasks.workunit.client.1.smithi078.stdout:0/517: chown d2/f21 902825 1 2024-04-26T04:52:53.352 INFO:tasks.workunit.client.1.smithi078.stdout:0/518: read d2/d9/d18/f1c [4161768,11765] 0 2024-04-26T04:52:53.353 INFO:tasks.workunit.client.1.smithi078.stdout:5/357: mkdir d7/d18/d32/d37/d79 0 2024-04-26T04:52:53.353 INFO:tasks.workunit.client.1.smithi078.stdout:0/519: write d2/d9/d31/d4a/f82 [3629322,99835] 0 2024-04-26T04:52:53.358 INFO:tasks.workunit.client.0.smithi077.stdout:1/218: rename d9/db/d2b/l2c to d9/db/dd/l3a 0 2024-04-26T04:52:53.360 INFO:tasks.workunit.client.1.smithi078.stdout:3/441: mknod d16/d26/d2e/d5f/c99 0 2024-04-26T04:52:53.360 INFO:tasks.workunit.client.1.smithi078.stdout:3/442: stat d16/d26/d2b/d31/d71/f8c 0 2024-04-26T04:52:53.360 INFO:tasks.workunit.client.0.smithi077.stdout:5/114: mknod d1/d12/c27 0 2024-04-26T04:52:53.361 INFO:tasks.workunit.client.0.smithi077.stdout:5/115: readlink d1/d12/d21/d25/l26 0 2024-04-26T04:52:53.361 INFO:tasks.workunit.client.0.smithi077.stdout:5/116: dread - d1/d8/fc zero size 2024-04-26T04:52:53.361 INFO:tasks.workunit.client.1.smithi078.stdout:4/463: unlink d0/d25/d20/d31/d33/d44/f59 0 2024-04-26T04:52:53.363 INFO:tasks.workunit.client.1.smithi078.stdout:8/422: link d0/d19/d25/c75 d0/d19/d25/d36/d58/d5b/d61/c80 0 2024-04-26T04:52:53.365 INFO:tasks.workunit.client.1.smithi078.stdout:7/445: mknod d9/da/d11/d13/d1a/c8c 0 2024-04-26T04:52:53.366 INFO:tasks.workunit.client.1.smithi078.stdout:7/446: fdatasync d9/da/f23 0 2024-04-26T04:52:53.366 INFO:tasks.workunit.client.1.smithi078.stdout:7/447: fdatasync d9/da/d11/d13/d50/f86 0 2024-04-26T04:52:53.369 INFO:tasks.workunit.client.0.smithi077.stdout:3/112: mkdir d0/d7/d27 0 2024-04-26T04:52:53.371 INFO:tasks.workunit.client.1.smithi078.stdout:1/491: dwrite d7/d1e/d9e/db/d37/f65 [0,4194304] 0 2024-04-26T04:52:53.375 INFO:tasks.workunit.client.1.smithi078.stdout:1/492: dread d7/d1e/d9e/db/d37/d58/f5e [0,4194304] 0 2024-04-26T04:52:53.376 INFO:tasks.workunit.client.0.smithi077.stdout:9/112: truncate d9/f10 5519949 0 2024-04-26T04:52:53.377 INFO:tasks.workunit.client.1.smithi078.stdout:9/397: chown dc/df/d4b/d3e/c56 67846201 1 2024-04-26T04:52:53.378 INFO:tasks.workunit.client.1.smithi078.stdout:9/398: write dc/df/d17/f2e [1213014,66667] 0 2024-04-26T04:52:53.381 INFO:tasks.workunit.client.0.smithi077.stdout:6/144: symlink d8/d1c/l35 0 2024-04-26T04:52:53.383 INFO:tasks.workunit.client.1.smithi078.stdout:4/464: symlink d0/d1d/lb0 0 2024-04-26T04:52:53.383 INFO:tasks.workunit.client.0.smithi077.stdout:0/93: creat d0/f24 x:0 0 0 2024-04-26T04:52:53.385 INFO:tasks.workunit.client.0.smithi077.stdout:7/143: mkdir da/d13/d22/d34 0 2024-04-26T04:52:53.386 INFO:tasks.workunit.client.0.smithi077.stdout:1/219: mknod d9/db/dd/d34/c3b 0 2024-04-26T04:52:53.388 INFO:tasks.workunit.client.1.smithi078.stdout:7/448: symlink d9/da/d20/l8d 0 2024-04-26T04:52:53.388 INFO:tasks.workunit.client.1.smithi078.stdout:7/449: stat d9/da 0 2024-04-26T04:52:53.388 INFO:tasks.workunit.client.0.smithi077.stdout:6/145: dwrite d8/f33 [0,4194304] 0 2024-04-26T04:52:53.390 INFO:tasks.workunit.client.0.smithi077.stdout:4/169: link d4/f14 d4/de/d15/d1f/f34 0 2024-04-26T04:52:53.391 INFO:tasks.workunit.client.0.smithi077.stdout:6/146: write d8/f22 [1407419,22895] 0 2024-04-26T04:52:53.391 INFO:tasks.workunit.client.1.smithi078.stdout:5/358: mkdir d7/d48/d57/d65/d7a 0 2024-04-26T04:52:53.392 INFO:tasks.workunit.client.0.smithi077.stdout:6/147: fdatasync f3 0 2024-04-26T04:52:53.392 INFO:tasks.workunit.client.1.smithi078.stdout:5/359: write d7/da/d1e/f4c [561185,64635] 0 2024-04-26T04:52:53.393 INFO:tasks.workunit.client.1.smithi078.stdout:0/520: creat d2/d9/d60/d6a/d8f/d92/fad x:0 0 0 2024-04-26T04:52:53.393 INFO:tasks.workunit.client.0.smithi077.stdout:6/148: readlink l6 0 2024-04-26T04:52:53.395 INFO:tasks.workunit.client.0.smithi077.stdout:3/113: fsync d0/f3 0 2024-04-26T04:52:53.396 INFO:tasks.workunit.client.0.smithi077.stdout:5/117: dwrite d1/d12/d21/d18/f1c [4194304,4194304] 0 2024-04-26T04:52:53.396 INFO:tasks.workunit.client.0.smithi077.stdout:3/114: dread - d0/d9/f26 zero size 2024-04-26T04:52:53.396 INFO:tasks.workunit.client.1.smithi078.stdout:1/493: unlink d7/d1e/d9e/dd/c99 0 2024-04-26T04:52:53.397 INFO:tasks.workunit.client.0.smithi077.stdout:3/115: write d0/f1d [193224,36349] 0 2024-04-26T04:52:53.398 INFO:tasks.workunit.client.1.smithi078.stdout:1/494: fdatasync d7/d1e/f31 0 2024-04-26T04:52:53.399 INFO:tasks.workunit.client.1.smithi078.stdout:5/360: dwrite d7/d11/f47 [0,4194304] 0 2024-04-26T04:52:53.400 INFO:tasks.workunit.client.1.smithi078.stdout:1/495: dread - d7/d1e/d9e/dd/d20/d53/f76 zero size 2024-04-26T04:52:53.401 INFO:tasks.workunit.client.0.smithi077.stdout:9/113: fdatasync f7 0 2024-04-26T04:52:53.401 INFO:tasks.workunit.client.1.smithi078.stdout:1/496: write d7/d1e/d9e/d17/fa1 [323272,43361] 0 2024-04-26T04:52:53.402 INFO:tasks.workunit.client.1.smithi078.stdout:1/497: chown d7/d1e/f36 1266 1 2024-04-26T04:52:53.405 INFO:tasks.workunit.client.1.smithi078.stdout:9/399: read dc/df/f1d [2793463,54010] 0 2024-04-26T04:52:53.406 INFO:tasks.workunit.client.0.smithi077.stdout:0/94: dread - d0/db/f16 zero size 2024-04-26T04:52:53.407 INFO:tasks.workunit.client.1.smithi078.stdout:0/521: mkdir d2/d9/d60/dae 0 2024-04-26T04:52:53.407 INFO:tasks.workunit.client.0.smithi077.stdout:5/118: dwrite d1/d12/d21/f17 [0,4194304] 0 2024-04-26T04:52:53.408 INFO:tasks.workunit.client.0.smithi077.stdout:0/95: read - d0/f24 zero size 2024-04-26T04:52:53.408 INFO:tasks.workunit.client.0.smithi077.stdout:7/144: mknod da/d15/c35 0 2024-04-26T04:52:53.410 INFO:tasks.workunit.client.0.smithi077.stdout:5/119: truncate d1/d12/f13 974688 0 2024-04-26T04:52:53.410 INFO:tasks.workunit.client.0.smithi077.stdout:0/96: read d0/db/ff [1606878,99135] 0 2024-04-26T04:52:53.411 INFO:tasks.workunit.client.1.smithi078.stdout:9/400: dwrite dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:53.414 INFO:tasks.workunit.client.0.smithi077.stdout:0/97: write d0/f4 [1337053,93624] 0 2024-04-26T04:52:53.415 INFO:tasks.workunit.client.0.smithi077.stdout:1/220: rename d9/db/dd/c32 to d9/db/d27/c3c 0 2024-04-26T04:52:53.415 INFO:tasks.workunit.client.1.smithi078.stdout:3/443: creat d16/d26/d2e/d5f/f9a x:0 0 0 2024-04-26T04:52:53.416 INFO:tasks.workunit.client.1.smithi078.stdout:3/444: dread - d16/d26/d2e/d74/f83 zero size 2024-04-26T04:52:53.417 INFO:tasks.workunit.client.0.smithi077.stdout:4/170: fsync d4/de/d15/f16 0 2024-04-26T04:52:53.418 INFO:tasks.workunit.client.1.smithi078.stdout:8/423: link d0/c11 d0/c81 0 2024-04-26T04:52:53.422 INFO:tasks.workunit.client.0.smithi077.stdout:5/120: dwrite d1/d8/f22 [0,4194304] 0 2024-04-26T04:52:53.425 INFO:tasks.workunit.client.0.smithi077.stdout:5/121: write d1/d8/fc [679502,56712] 0 2024-04-26T04:52:53.427 INFO:tasks.workunit.client.0.smithi077.stdout:2/122: getdents d9/d15 0 2024-04-26T04:52:53.428 INFO:tasks.workunit.client.0.smithi077.stdout:0/98: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:53.428 INFO:tasks.workunit.client.0.smithi077.stdout:2/123: chown f7 120 1 2024-04-26T04:52:53.429 INFO:tasks.workunit.client.0.smithi077.stdout:2/124: fsync f5 0 2024-04-26T04:52:53.429 INFO:tasks.workunit.client.1.smithi078.stdout:2/408: rename d8/d11/d18/d5d/f7e to d8/d11/d14/f90 0 2024-04-26T04:52:53.431 INFO:tasks.workunit.client.1.smithi078.stdout:2/409: truncate d8/d11/d14/f86 477603 0 2024-04-26T04:52:53.433 INFO:tasks.workunit.client.1.smithi078.stdout:2/410: write d8/d11/d14/d69/d8f/f6c [3162367,116104] 0 2024-04-26T04:52:53.434 INFO:tasks.workunit.client.0.smithi077.stdout:3/116: creat d0/d7/dd/f28 x:0 0 0 2024-04-26T04:52:53.442 INFO:tasks.workunit.client.1.smithi078.stdout:3/445: rmdir d16/d26/d2b/d31/d71/d8a 39 2024-04-26T04:52:53.448 INFO:tasks.workunit.client.0.smithi077.stdout:1/221: truncate d9/f21 12512428 0 2024-04-26T04:52:53.451 INFO:tasks.workunit.client.1.smithi078.stdout:6/518: rename d2/d52/f8e to d2/d1a/d5c/f98 0 2024-04-26T04:52:53.452 INFO:tasks.workunit.client.1.smithi078.stdout:6/519: fdatasync d2/d1a/d5/d2a/d4d/f61 0 2024-04-26T04:52:53.454 INFO:tasks.workunit.client.0.smithi077.stdout:4/171: rmdir d4/d17/d18 39 2024-04-26T04:52:53.456 INFO:tasks.workunit.client.1.smithi078.stdout:0/522: creat d2/d9/d60/d6a/daa/faf x:0 0 0 2024-04-26T04:52:53.457 INFO:tasks.workunit.client.1.smithi078.stdout:4/465: truncate d0/de/d22/d27/d56/f5c 2360639 0 2024-04-26T04:52:53.457 INFO:tasks.workunit.client.1.smithi078.stdout:0/523: fdatasync d2/d1e/d5a/d74/f79 0 2024-04-26T04:52:53.461 INFO:tasks.workunit.client.0.smithi077.stdout:2/125: mkdir d9/d15/d1a/d2b 0 2024-04-26T04:52:53.461 INFO:tasks.workunit.client.0.smithi077.stdout:2/126: fdatasync d9/f17 0 2024-04-26T04:52:53.462 INFO:tasks.workunit.client.0.smithi077.stdout:8/144: link d4/fe d4/df/d22/f34 0 2024-04-26T04:52:53.463 INFO:tasks.workunit.client.1.smithi078.stdout:1/498: link d7/d1e/d9e/c4b d7/d2e/ca4 0 2024-04-26T04:52:53.463 INFO:tasks.workunit.client.1.smithi078.stdout:0/524: dwrite d2/d9/d60/d6a/d8f/f95 [0,4194304] 0 2024-04-26T04:52:53.464 INFO:tasks.workunit.client.0.smithi077.stdout:3/117: mkdir d0/d7/dd/d29 0 2024-04-26T04:52:53.465 INFO:tasks.workunit.client.0.smithi077.stdout:2/127: dread d9/f17 [0,4194304] 0 2024-04-26T04:52:53.465 INFO:tasks.workunit.client.0.smithi077.stdout:2/128: chown d9/d15 13224 1 2024-04-26T04:52:53.467 INFO:tasks.workunit.client.0.smithi077.stdout:7/145: write da/d15/d1a/f20 [965925,12314] 0 2024-04-26T04:52:53.467 INFO:tasks.workunit.client.1.smithi078.stdout:7/450: rename d9/da/d11/d13/d50/l87 to d9/da/d33/d4b/d4c/d5b/l8e 0 2024-04-26T04:52:53.468 INFO:tasks.workunit.client.1.smithi078.stdout:6/520: creat d2/d1a/d5/d3a/d69/f99 x:0 0 0 2024-04-26T04:52:53.469 INFO:tasks.workunit.client.1.smithi078.stdout:0/525: dwrite d2/d9/f3e [0,4194304] 0 2024-04-26T04:52:53.470 INFO:tasks.workunit.client.1.smithi078.stdout:2/411: mkdir d8/d3e/d8e/d91 0 2024-04-26T04:52:53.472 INFO:tasks.workunit.client.1.smithi078.stdout:2/412: read d8/d11/f31 [2025521,35831] 0 2024-04-26T04:52:53.473 INFO:tasks.workunit.client.1.smithi078.stdout:2/413: chown d8/d11/d6e/f7b 3966437 1 2024-04-26T04:52:53.474 INFO:tasks.workunit.client.0.smithi077.stdout:1/222: creat d9/db/d2b/f3d x:0 0 0 2024-04-26T04:52:53.478 INFO:tasks.workunit.client.0.smithi077.stdout:4/172: unlink d4/de/f2f 0 2024-04-26T04:52:53.479 INFO:tasks.workunit.client.0.smithi077.stdout:1/223: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:53.480 INFO:tasks.workunit.client.1.smithi078.stdout:4/466: creat d0/d25/d20/d31/d33/d44/fb1 x:0 0 0 2024-04-26T04:52:53.481 INFO:tasks.workunit.client.1.smithi078.stdout:9/401: link dc/df/d4b/l1f dc/df/d4b/d3e/l77 0 2024-04-26T04:52:53.481 INFO:tasks.workunit.client.1.smithi078.stdout:9/402: stat dc 0 2024-04-26T04:52:53.482 INFO:tasks.workunit.client.1.smithi078.stdout:9/403: chown dc/f3d 274219 1 2024-04-26T04:52:53.484 INFO:tasks.workunit.client.1.smithi078.stdout:9/404: write dc/df/d27/f76 [398254,35497] 0 2024-04-26T04:52:53.486 INFO:tasks.workunit.client.1.smithi078.stdout:4/467: dwrite d0/d25/d20/d31/d33/d44/d55/f5d [0,4194304] 0 2024-04-26T04:52:53.488 INFO:tasks.workunit.client.0.smithi077.stdout:8/145: mkdir d4/d17/d35 0 2024-04-26T04:52:53.490 INFO:tasks.workunit.client.0.smithi077.stdout:3/118: mkdir d0/d7/dd/d2a 0 2024-04-26T04:52:53.495 INFO:tasks.workunit.client.1.smithi078.stdout:5/361: rename d7/d11/c25 to d7/d18/d32/d37/d79/c7b 0 2024-04-26T04:52:53.497 INFO:tasks.workunit.client.1.smithi078.stdout:6/521: mknod d2/d52/c9a 0 2024-04-26T04:52:53.498 INFO:tasks.workunit.client.1.smithi078.stdout:6/522: chown d2/d1a/d5/da/f22 3992657 1 2024-04-26T04:52:53.502 INFO:tasks.workunit.client.1.smithi078.stdout:5/362: dwrite f3 [0,4194304] 0 2024-04-26T04:52:53.502 INFO:tasks.workunit.client.0.smithi077.stdout:4/173: mkdir d4/de/d15/d1f/d35 0 2024-04-26T04:52:53.504 INFO:tasks.workunit.client.0.smithi077.stdout:1/224: rename d9/db/dd/f15 to d9/db/dd/f3e 0 2024-04-26T04:52:53.506 INFO:tasks.workunit.client.1.smithi078.stdout:2/414: write d8/d11/d14/f3b [112310,41500] 0 2024-04-26T04:52:53.506 INFO:tasks.workunit.client.1.smithi078.stdout:2/415: chown d8 2162060 1 2024-04-26T04:52:53.508 INFO:tasks.workunit.client.0.smithi077.stdout:6/149: getdents d8 0 2024-04-26T04:52:53.511 INFO:tasks.workunit.client.0.smithi077.stdout:5/122: rename d1/d12/d21/d18 to d1/d12/d28 0 2024-04-26T04:52:53.511 INFO:tasks.workunit.client.0.smithi077.stdout:5/123: chown d1/d8/c20 6 1 2024-04-26T04:52:53.513 INFO:tasks.workunit.client.1.smithi078.stdout:2/416: dwrite d8/d11/f54 [4194304,4194304] 0 2024-04-26T04:52:53.514 INFO:tasks.workunit.client.0.smithi077.stdout:8/146: chown d4/df/f15 850534 1 2024-04-26T04:52:53.514 INFO:tasks.workunit.client.1.smithi078.stdout:6/523: dread d2/d1a/d5/d3a/d42/f7d [0,4194304] 0 2024-04-26T04:52:53.515 INFO:tasks.workunit.client.1.smithi078.stdout:9/405: rmdir dc/df/d37/d64 39 2024-04-26T04:52:53.515 INFO:tasks.workunit.client.1.smithi078.stdout:6/524: fdatasync d2/f53 0 2024-04-26T04:52:53.516 INFO:tasks.workunit.client.0.smithi077.stdout:3/119: creat d0/d9/f2b x:0 0 0 2024-04-26T04:52:53.518 INFO:tasks.workunit.client.1.smithi078.stdout:4/468: write d0/de/d22/d27/f96 [60354,99056] 0 2024-04-26T04:52:53.526 INFO:tasks.workunit.client.0.smithi077.stdout:1/225: rmdir d9 39 2024-04-26T04:52:53.526 INFO:tasks.workunit.client.1.smithi078.stdout:1/499: truncate d7/d1e/f31 2653855 0 2024-04-26T04:52:53.526 INFO:tasks.workunit.client.1.smithi078.stdout:8/424: rename d0/d19/d52/d35/d44/c1d to d0/d19/d52/d35/d44/d6b/c82 0 2024-04-26T04:52:53.527 INFO:tasks.workunit.client.0.smithi077.stdout:1/226: read f7 [72914,50180] 0 2024-04-26T04:52:53.529 INFO:tasks.workunit.client.1.smithi078.stdout:8/425: truncate d0/d19/d52/d35/d44/f5d 634370 0 2024-04-26T04:52:53.529 INFO:tasks.workunit.client.0.smithi077.stdout:5/124: rmdir d1/d8 39 2024-04-26T04:52:53.529 INFO:tasks.workunit.client.1.smithi078.stdout:8/426: stat d0 0 2024-04-26T04:52:53.530 INFO:tasks.workunit.client.1.smithi078.stdout:4/469: dwrite d0/d25/d20/f1b [0,4194304] 0 2024-04-26T04:52:53.530 INFO:tasks.workunit.client.1.smithi078.stdout:5/363: creat d7/d18/d1c/d5a/f7c x:0 0 0 2024-04-26T04:52:53.532 INFO:tasks.workunit.client.0.smithi077.stdout:8/147: unlink d4/df/d22/f24 0 2024-04-26T04:52:53.534 INFO:tasks.workunit.client.0.smithi077.stdout:3/120: creat d0/d7/dd/f2c x:0 0 0 2024-04-26T04:52:53.535 INFO:tasks.workunit.client.0.smithi077.stdout:3/121: readlink d0/d9/lc 0 2024-04-26T04:52:53.536 INFO:tasks.workunit.client.0.smithi077.stdout:7/146: truncate da/f14 419892 0 2024-04-26T04:52:53.538 INFO:tasks.workunit.client.0.smithi077.stdout:7/147: dread - da/d15/f2f zero size 2024-04-26T04:52:53.538 INFO:tasks.workunit.client.0.smithi077.stdout:8/148: dread d4/df/d22/d2d/f1b [0,4194304] 0 2024-04-26T04:52:53.541 INFO:tasks.workunit.client.1.smithi078.stdout:5/364: dread f5 [0,4194304] 0 2024-04-26T04:52:53.541 INFO:tasks.workunit.client.1.smithi078.stdout:7/451: truncate d9/da/d11/f2f 1451672 0 2024-04-26T04:52:53.542 INFO:tasks.workunit.client.1.smithi078.stdout:5/365: dread - d7/da/f5f zero size 2024-04-26T04:52:53.543 INFO:tasks.workunit.client.1.smithi078.stdout:3/446: getdents d16/d26/d2b 0 2024-04-26T04:52:53.545 INFO:tasks.workunit.client.1.smithi078.stdout:2/417: symlink d8/d3e/d8e/l92 0 2024-04-26T04:52:53.545 INFO:tasks.workunit.client.0.smithi077.stdout:6/150: truncate f4 3548093 0 2024-04-26T04:52:53.548 INFO:tasks.workunit.client.0.smithi077.stdout:5/125: symlink d1/d12/d21/l29 0 2024-04-26T04:52:53.548 INFO:tasks.workunit.client.1.smithi078.stdout:2/418: dread d8/d11/d14/f90 [0,4194304] 0 2024-04-26T04:52:53.550 INFO:tasks.workunit.client.1.smithi078.stdout:2/419: truncate d8/d11/d14/f3b 13371056 0 2024-04-26T04:52:53.553 INFO:tasks.workunit.client.0.smithi077.stdout:2/129: getdents d9/d15/d1a 0 2024-04-26T04:52:53.553 INFO:tasks.workunit.client.0.smithi077.stdout:2/130: readlink l6 0 2024-04-26T04:52:53.554 INFO:tasks.workunit.client.0.smithi077.stdout:2/131: read d9/f17 [155056,4965] 0 2024-04-26T04:52:53.557 INFO:tasks.workunit.client.0.smithi077.stdout:3/122: dwrite d0/d9/f23 [8388608,4194304] 0 2024-04-26T04:52:53.560 INFO:tasks.workunit.client.0.smithi077.stdout:6/151: symlink d8/d2a/l36 0 2024-04-26T04:52:53.561 INFO:tasks.workunit.client.0.smithi077.stdout:6/152: write d8/f27 [954160,59707] 0 2024-04-26T04:52:53.562 INFO:tasks.workunit.client.0.smithi077.stdout:1/227: mkdir d9/db/d2b/d3f 0 2024-04-26T04:52:53.564 INFO:tasks.workunit.client.0.smithi077.stdout:0/99: rename d0/d19/f1a to d0/f25 0 2024-04-26T04:52:53.569 INFO:tasks.workunit.client.0.smithi077.stdout:8/149: mkdir d4/df/d22/d2d/d2b/d33/d36 0 2024-04-26T04:52:53.570 INFO:tasks.workunit.client.0.smithi077.stdout:2/132: rmdir d9/dc/d1d/d1e 39 2024-04-26T04:52:53.576 INFO:tasks.workunit.client.0.smithi077.stdout:1/228: creat d9/db/d27/f40 x:0 0 0 2024-04-26T04:52:53.577 INFO:tasks.workunit.client.0.smithi077.stdout:1/229: write d9/db/f39 [924337,38513] 0 2024-04-26T04:52:53.578 INFO:tasks.workunit.client.0.smithi077.stdout:0/100: creat d0/d19/d1e/f26 x:0 0 0 2024-04-26T04:52:53.579 INFO:tasks.workunit.client.0.smithi077.stdout:3/123: dread d0/d7/d15/f1a [0,4194304] 0 2024-04-26T04:52:53.580 INFO:tasks.workunit.client.1.smithi078.stdout:1/500: chown d7/d2e/ca4 241852 1 2024-04-26T04:52:53.581 INFO:tasks.workunit.client.0.smithi077.stdout:7/148: creat da/f36 x:0 0 0 2024-04-26T04:52:53.583 INFO:tasks.workunit.client.1.smithi078.stdout:0/526: rename d2/d9/c35 to d2/d9/cb0 0 2024-04-26T04:52:53.585 INFO:tasks.workunit.client.1.smithi078.stdout:4/470: write d0/d25/d20/f4f [766814,66106] 0 2024-04-26T04:52:53.587 INFO:tasks.workunit.client.0.smithi077.stdout:5/126: creat d1/d12/f2a x:0 0 0 2024-04-26T04:52:53.588 INFO:tasks.workunit.client.0.smithi077.stdout:0/101: symlink d0/db/l27 0 2024-04-26T04:52:53.589 INFO:tasks.workunit.client.0.smithi077.stdout:0/102: dread - d0/d19/d1e/f26 zero size 2024-04-26T04:52:53.589 INFO:tasks.workunit.client.1.smithi078.stdout:5/366: creat d7/d18/d1c/f7d x:0 0 0 2024-04-26T04:52:53.593 INFO:tasks.workunit.client.0.smithi077.stdout:2/133: dread d9/ff [0,4194304] 0 2024-04-26T04:52:53.593 INFO:tasks.workunit.client.0.smithi077.stdout:7/149: symlink da/d15/d1a/l37 0 2024-04-26T04:52:53.595 INFO:tasks.workunit.client.1.smithi078.stdout:2/420: creat d8/d3e/f93 x:0 0 0 2024-04-26T04:52:53.595 INFO:tasks.workunit.client.0.smithi077.stdout:6/153: link d8/d2a/f32 d8/f37 0 2024-04-26T04:52:53.595 INFO:tasks.workunit.client.1.smithi078.stdout:3/447: dwrite d16/d26/f2a [0,4194304] 0 2024-04-26T04:52:53.597 INFO:tasks.workunit.client.1.smithi078.stdout:1/501: mkdir d7/d1e/d9e/db/d47/da5 0 2024-04-26T04:52:53.597 INFO:tasks.workunit.client.1.smithi078.stdout:1/502: chown d7/d2e/c30 22595614 1 2024-04-26T04:52:53.598 INFO:tasks.workunit.client.0.smithi077.stdout:1/230: creat d9/db/d2b/d3f/f41 x:0 0 0 2024-04-26T04:52:53.602 INFO:tasks.workunit.client.0.smithi077.stdout:5/127: readlink d1/l9 0 2024-04-26T04:52:53.605 INFO:tasks.workunit.client.0.smithi077.stdout:3/124: symlink d0/d7/dd/d2a/l2d 0 2024-04-26T04:52:53.607 INFO:tasks.workunit.client.0.smithi077.stdout:3/125: fdatasync d0/f1d 0 2024-04-26T04:52:53.607 INFO:tasks.workunit.client.0.smithi077.stdout:1/231: dwrite d9/db/d27/f28 [0,4194304] 0 2024-04-26T04:52:53.610 INFO:tasks.workunit.client.1.smithi078.stdout:4/471: rename d0/d25/d20/d31/d33 to d0/d25/d20/d31/d47/db2 0 2024-04-26T04:52:53.612 INFO:tasks.workunit.client.1.smithi078.stdout:2/421: dread d8/f4e [4194304,4194304] 0 2024-04-26T04:52:53.613 INFO:tasks.workunit.client.0.smithi077.stdout:0/103: unlink d0/f12 0 2024-04-26T04:52:53.615 INFO:tasks.workunit.client.0.smithi077.stdout:3/126: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:53.615 INFO:tasks.workunit.client.0.smithi077.stdout:0/104: chown d0/c8 77822 1 2024-04-26T04:52:53.615 INFO:tasks.workunit.client.1.smithi078.stdout:7/452: mkdir d9/da/d11/d13/d50/d8a/d8f 0 2024-04-26T04:52:53.616 INFO:tasks.workunit.client.0.smithi077.stdout:3/127: fdatasync d0/f1b 0 2024-04-26T04:52:53.616 INFO:tasks.workunit.client.0.smithi077.stdout:0/105: fdatasync d0/f24 0 2024-04-26T04:52:53.616 INFO:tasks.workunit.client.1.smithi078.stdout:7/453: stat d9/da/f23 0 2024-04-26T04:52:53.616 INFO:tasks.workunit.client.1.smithi078.stdout:7/454: dread - d9/f58 zero size 2024-04-26T04:52:53.617 INFO:tasks.workunit.client.0.smithi077.stdout:8/150: rmdir d4/df/d22/d2d/d2b/d33/d36 0 2024-04-26T04:52:53.617 INFO:tasks.workunit.client.0.smithi077.stdout:3/128: stat d0/d7/dd/f21 0 2024-04-26T04:52:53.617 INFO:tasks.workunit.client.1.smithi078.stdout:4/472: dread d0/d25/d20/d31/d47/db2/d44/f68 [0,4194304] 0 2024-04-26T04:52:53.619 INFO:tasks.workunit.client.1.smithi078.stdout:5/367: symlink d7/d48/d57/d65/d70/l7e 0 2024-04-26T04:52:53.620 INFO:tasks.workunit.client.1.smithi078.stdout:5/368: chown d7/d18/f40 588509107 1 2024-04-26T04:52:53.620 INFO:tasks.workunit.client.1.smithi078.stdout:9/406: link dc/df/d4b/d22/l31 dc/df/d27/l78 0 2024-04-26T04:52:53.621 INFO:tasks.workunit.client.1.smithi078.stdout:5/369: chown d7/da/d1e/l20 952 1 2024-04-26T04:52:53.622 INFO:tasks.workunit.client.0.smithi077.stdout:3/129: dread d0/f1 [0,4194304] 0 2024-04-26T04:52:53.622 INFO:tasks.workunit.client.0.smithi077.stdout:7/150: mknod da/d13/c38 0 2024-04-26T04:52:53.623 INFO:tasks.workunit.client.0.smithi077.stdout:3/130: chown d0/c12 15588746 1 2024-04-26T04:52:53.625 INFO:tasks.workunit.client.0.smithi077.stdout:3/131: fsync d0/d9/f25 0 2024-04-26T04:52:53.627 INFO:tasks.workunit.client.0.smithi077.stdout:2/134: dwrite d9/ff [0,4194304] 0 2024-04-26T04:52:53.628 INFO:tasks.workunit.client.0.smithi077.stdout:2/135: chown d9/f10 1550479 1 2024-04-26T04:52:53.635 INFO:tasks.workunit.client.1.smithi078.stdout:2/422: rename d8/d11/d18/c57 to d8/d3e/d8e/d91/c94 0 2024-04-26T04:52:53.636 INFO:tasks.workunit.client.1.smithi078.stdout:1/503: dread d7/d2e/f5f [0,4194304] 0 2024-04-26T04:52:53.641 INFO:tasks.workunit.client.0.smithi077.stdout:3/132: mknod d0/d7/d15/c2e 0 2024-04-26T04:52:53.643 INFO:tasks.workunit.client.1.smithi078.stdout:3/448: mkdir d16/d9b 0 2024-04-26T04:52:53.646 INFO:tasks.workunit.client.0.smithi077.stdout:0/106: mkdir d0/d28 0 2024-04-26T04:52:53.646 INFO:tasks.workunit.client.0.smithi077.stdout:0/107: chown d0 7855003 1 2024-04-26T04:52:53.650 INFO:tasks.workunit.client.1.smithi078.stdout:2/423: rename d8/d3e/d8e/l92 to d8/d3e/l95 0 2024-04-26T04:52:53.651 INFO:tasks.workunit.client.1.smithi078.stdout:2/424: chown d8/d11/d6e 13 1 2024-04-26T04:52:53.654 INFO:tasks.workunit.client.0.smithi077.stdout:3/133: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:52:53.656 INFO:tasks.workunit.client.0.smithi077.stdout:3/134: truncate d0/f1d 1185692 0 2024-04-26T04:52:53.656 INFO:tasks.workunit.client.0.smithi077.stdout:1/232: creat d9/f42 x:0 0 0 2024-04-26T04:52:53.657 INFO:tasks.workunit.client.0.smithi077.stdout:1/233: chown d9/db/dd/lf 45649208 1 2024-04-26T04:52:53.657 INFO:tasks.workunit.client.0.smithi077.stdout:0/108: symlink d0/d19/l29 0 2024-04-26T04:52:53.658 INFO:tasks.workunit.client.0.smithi077.stdout:1/234: read - d9/db/d2b/d3f/f41 zero size 2024-04-26T04:52:53.658 INFO:tasks.workunit.client.0.smithi077.stdout:0/109: fdatasync d0/d19/d1e/f26 0 2024-04-26T04:52:53.660 INFO:tasks.workunit.client.0.smithi077.stdout:6/154: truncate d8/fb 1762129 0 2024-04-26T04:52:53.660 INFO:tasks.workunit.client.0.smithi077.stdout:3/135: dread d0/d9/f23 [0,4194304] 0 2024-04-26T04:52:53.662 INFO:tasks.workunit.client.1.smithi078.stdout:6/525: truncate d2/de/d27/d58/f76 841033 0 2024-04-26T04:52:53.662 INFO:tasks.workunit.client.0.smithi077.stdout:5/128: getdents d1/d12/d21 0 2024-04-26T04:52:53.663 INFO:tasks.workunit.client.1.smithi078.stdout:6/526: dread - d2/d1a/d5/d3a/d42/f5a zero size 2024-04-26T04:52:53.664 INFO:tasks.workunit.client.1.smithi078.stdout:9/407: creat dc/df/d4b/f79 x:0 0 0 2024-04-26T04:52:53.665 INFO:tasks.workunit.client.0.smithi077.stdout:2/136: creat d9/f2c x:0 0 0 2024-04-26T04:52:53.666 INFO:tasks.workunit.client.0.smithi077.stdout:2/137: write d9/f17 [2763213,98452] 0 2024-04-26T04:52:53.671 INFO:tasks.workunit.client.0.smithi077.stdout:0/110: mknod d0/d19/d1e/c2a 0 2024-04-26T04:52:53.671 INFO:tasks.workunit.client.0.smithi077.stdout:0/111: chown d0/d19 96032052 1 2024-04-26T04:52:53.675 INFO:tasks.workunit.client.0.smithi077.stdout:3/136: creat d0/d7/dd/f2f x:0 0 0 2024-04-26T04:52:53.677 INFO:tasks.workunit.client.0.smithi077.stdout:5/129: rmdir d1/d12/d28 39 2024-04-26T04:52:53.679 INFO:tasks.workunit.client.0.smithi077.stdout:2/138: symlink d9/d15/d1a/l2d 0 2024-04-26T04:52:53.680 INFO:tasks.workunit.client.1.smithi078.stdout:0/527: write d2/ff [979469,9418] 0 2024-04-26T04:52:53.680 INFO:tasks.workunit.client.1.smithi078.stdout:2/425: link d8/d11/d14/c50 d8/d53/c96 0 2024-04-26T04:52:53.682 INFO:tasks.workunit.client.1.smithi078.stdout:0/528: mknod d2/d9/d60/cb1 0 2024-04-26T04:52:53.696 INFO:tasks.workunit.client.0.smithi077.stdout:6/155: unlink d8/l14 0 2024-04-26T04:52:53.698 INFO:tasks.workunit.client.0.smithi077.stdout:3/137: unlink d0/d9/lc 0 2024-04-26T04:52:53.700 INFO:tasks.workunit.client.0.smithi077.stdout:2/139: creat d9/dc/f2e x:0 0 0 2024-04-26T04:52:53.701 INFO:tasks.workunit.client.0.smithi077.stdout:1/235: creat d9/f43 x:0 0 0 2024-04-26T04:52:53.702 INFO:tasks.workunit.client.0.smithi077.stdout:0/112: symlink d0/d28/l2b 0 2024-04-26T04:52:53.704 INFO:tasks.workunit.client.0.smithi077.stdout:3/138: read d0/d9/f23 [8004723,120453] 0 2024-04-26T04:52:53.705 INFO:tasks.workunit.client.1.smithi078.stdout:3/449: getdents d16 0 2024-04-26T04:52:53.706 INFO:tasks.workunit.client.1.smithi078.stdout:4/473: truncate d0/de/d22/d27/d7b/f8f 2156673 0 2024-04-26T04:52:53.707 INFO:tasks.workunit.client.1.smithi078.stdout:6/527: dread d2/d1a/d5/f31 [0,4194304] 0 2024-04-26T04:52:53.708 INFO:tasks.workunit.client.1.smithi078.stdout:3/450: chown d16/d26/d2e/f6d 7109246 1 2024-04-26T04:52:53.708 INFO:tasks.workunit.client.0.smithi077.stdout:2/140: dwrite d9/dc/f2e [0,4194304] 0 2024-04-26T04:52:53.708 INFO:tasks.workunit.client.1.smithi078.stdout:1/504: truncate d7/d1e/f36 1743125 0 2024-04-26T04:52:53.709 INFO:tasks.workunit.client.1.smithi078.stdout:5/370: dwrite d7/da/f10 [4194304,4194304] 0 2024-04-26T04:52:53.710 INFO:tasks.workunit.client.1.smithi078.stdout:6/528: chown d2/de/f48 590041625 1 2024-04-26T04:52:53.710 INFO:tasks.workunit.client.0.smithi077.stdout:2/141: read d9/d15/d1a/f22 [125401,69706] 0 2024-04-26T04:52:53.711 INFO:tasks.workunit.client.1.smithi078.stdout:5/371: stat d7/d18/d1c/f7d 0 2024-04-26T04:52:53.712 INFO:tasks.workunit.client.1.smithi078.stdout:1/505: read - d7/d1e/d9e/dd/d20/d53/f76 zero size 2024-04-26T04:52:53.712 INFO:tasks.workunit.client.1.smithi078.stdout:5/372: chown d7/d48 43055 1 2024-04-26T04:52:53.712 INFO:tasks.workunit.client.1.smithi078.stdout:2/426: rename d8/le to d8/d11/l97 0 2024-04-26T04:52:53.713 INFO:tasks.workunit.client.1.smithi078.stdout:1/506: read - d7/d1e/d9e/dd/d20/f7b zero size 2024-04-26T04:52:53.715 INFO:tasks.workunit.client.1.smithi078.stdout:0/529: truncate d2/d1e/f2a 4061796 0 2024-04-26T04:52:53.716 INFO:tasks.workunit.client.1.smithi078.stdout:2/427: chown d8/d11/d14/d69/d8f/c4b 3 1 2024-04-26T04:52:53.718 INFO:tasks.workunit.client.1.smithi078.stdout:3/451: truncate f6 2220459 0 2024-04-26T04:52:53.719 INFO:tasks.workunit.client.0.smithi077.stdout:3/139: creat d0/d9/f30 x:0 0 0 2024-04-26T04:52:53.720 INFO:tasks.workunit.client.0.smithi077.stdout:3/140: dread - d0/f1c zero size 2024-04-26T04:52:53.721 INFO:tasks.workunit.client.0.smithi077.stdout:5/130: dread d1/fe [0,4194304] 0 2024-04-26T04:52:53.721 INFO:tasks.workunit.client.0.smithi077.stdout:3/141: stat d0/d7/dd/f2c 0 2024-04-26T04:52:53.722 INFO:tasks.workunit.client.0.smithi077.stdout:0/113: mkdir d0/d2c 0 2024-04-26T04:52:53.723 INFO:tasks.workunit.client.0.smithi077.stdout:3/142: write d0/d9/f25 [927287,55407] 0 2024-04-26T04:52:53.724 INFO:tasks.workunit.client.0.smithi077.stdout:3/143: write d0/f1d [863980,108306] 0 2024-04-26T04:52:53.724 INFO:tasks.workunit.client.1.smithi078.stdout:4/474: symlink d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/lb3 0 2024-04-26T04:52:53.725 INFO:tasks.workunit.client.0.smithi077.stdout:3/144: fsync d0/d9/f14 0 2024-04-26T04:52:53.729 INFO:tasks.workunit.client.1.smithi078.stdout:5/373: rmdir d7/d48/d57/d65/d46 39 2024-04-26T04:52:53.731 INFO:tasks.workunit.client.1.smithi078.stdout:5/374: read - d7/d18/d1c/f7d zero size 2024-04-26T04:52:53.732 INFO:tasks.workunit.client.1.smithi078.stdout:7/455: sync 2024-04-26T04:52:53.732 INFO:tasks.workunit.client.1.smithi078.stdout:8/427: sync 2024-04-26T04:52:53.734 INFO:tasks.workunit.client.0.smithi077.stdout:1/236: creat d9/f44 x:0 0 0 2024-04-26T04:52:53.736 INFO:tasks.workunit.client.1.smithi078.stdout:2/428: dread d8/d11/f45 [0,4194304] 0 2024-04-26T04:52:53.737 INFO:tasks.workunit.client.1.smithi078.stdout:2/429: write d8/d11/d14/f3b [5498578,78609] 0 2024-04-26T04:52:53.738 INFO:tasks.workunit.client.0.smithi077.stdout:6/156: write d8/f17 [984712,18609] 0 2024-04-26T04:52:53.738 INFO:tasks.workunit.client.1.smithi078.stdout:0/530: truncate d2/f57 3805835 0 2024-04-26T04:52:53.740 INFO:tasks.workunit.client.1.smithi078.stdout:0/531: truncate d2/d9/d18/d7d/fa0 765584 0 2024-04-26T04:52:53.741 INFO:tasks.workunit.client.1.smithi078.stdout:0/532: read - d2/d9/d31/d4a/f64 zero size 2024-04-26T04:52:53.742 INFO:tasks.workunit.client.0.smithi077.stdout:1/237: dwrite d9/db/dd/f23 [0,4194304] 0 2024-04-26T04:52:53.743 INFO:tasks.workunit.client.1.smithi078.stdout:3/452: dread d16/f25 [0,4194304] 0 2024-04-26T04:52:53.744 INFO:tasks.workunit.client.1.smithi078.stdout:4/475: mkdir d0/d1d/db4 0 2024-04-26T04:52:53.746 INFO:tasks.workunit.client.0.smithi077.stdout:5/131: dwrite d1/d12/d28/f1c [8388608,4194304] 0 2024-04-26T04:52:53.750 INFO:tasks.workunit.client.0.smithi077.stdout:3/145: rename d0/f3 to d0/d7/dd/d29/f31 0 2024-04-26T04:52:53.752 INFO:tasks.workunit.client.1.smithi078.stdout:5/375: rmdir d7/d11 39 2024-04-26T04:52:53.753 INFO:tasks.workunit.client.1.smithi078.stdout:5/376: read d7/f39 [72886,113557] 0 2024-04-26T04:52:53.754 INFO:tasks.workunit.client.0.smithi077.stdout:2/142: unlink d9/dc/d1d/d1e/c1b 0 2024-04-26T04:52:53.754 INFO:tasks.workunit.client.0.smithi077.stdout:2/143: chown d9/dc/d1d/f24 10957 1 2024-04-26T04:52:53.756 INFO:tasks.workunit.client.1.smithi078.stdout:7/456: creat d9/f90 x:0 0 0 2024-04-26T04:52:53.760 INFO:tasks.workunit.client.0.smithi077.stdout:6/157: write d8/f11 [5356772,6082] 0 2024-04-26T04:52:53.761 INFO:tasks.workunit.client.0.smithi077.stdout:0/114: getdents d0/d2c 0 2024-04-26T04:52:53.761 INFO:tasks.workunit.client.0.smithi077.stdout:0/115: chown d0/d2c 582 1 2024-04-26T04:52:53.761 INFO:tasks.workunit.client.1.smithi078.stdout:1/507: dwrite d7/d1e/d9e/d4c/d3d/f91 [0,4194304] 0 2024-04-26T04:52:53.767 INFO:tasks.workunit.client.1.smithi078.stdout:2/430: creat d8/d11/d18/f98 x:0 0 0 2024-04-26T04:52:53.769 INFO:tasks.workunit.client.0.smithi077.stdout:1/238: rename d9/db/dd/d34/c3b to d9/db/d2b/c45 0 2024-04-26T04:52:53.773 INFO:tasks.workunit.client.0.smithi077.stdout:2/144: creat d9/d15/f2f x:0 0 0 2024-04-26T04:52:53.773 INFO:tasks.workunit.client.1.smithi078.stdout:0/533: rmdir d2/d9/d78/d84 39 2024-04-26T04:52:53.776 INFO:tasks.workunit.client.0.smithi077.stdout:6/158: write d8/ff [52375,90606] 0 2024-04-26T04:52:53.777 INFO:tasks.workunit.client.1.smithi078.stdout:3/453: unlink fc 0 2024-04-26T04:52:53.777 INFO:tasks.workunit.client.0.smithi077.stdout:1/239: dwrite d9/db/d2b/f3d [0,4194304] 0 2024-04-26T04:52:53.779 INFO:tasks.workunit.client.0.smithi077.stdout:0/116: rmdir d0 39 2024-04-26T04:52:53.781 INFO:tasks.workunit.client.1.smithi078.stdout:4/476: unlink d0/d1d/d46/f84 0 2024-04-26T04:52:53.782 INFO:tasks.workunit.client.1.smithi078.stdout:4/477: chown d0/d25/d20/d31/d47/db2/d4c/d62 13 1 2024-04-26T04:52:53.783 INFO:tasks.workunit.client.0.smithi077.stdout:1/240: dwrite d9/db/dd/f2f [0,4194304] 0 2024-04-26T04:52:53.785 INFO:tasks.workunit.client.1.smithi078.stdout:6/529: rename d2/de/d27/d58/d5b/f7c to d2/d1a/f9b 0 2024-04-26T04:52:53.786 INFO:tasks.workunit.client.0.smithi077.stdout:2/145: rmdir d9/dc/d1d 39 2024-04-26T04:52:53.786 INFO:tasks.workunit.client.1.smithi078.stdout:6/530: write d2/d1a/fd [4120703,99169] 0 2024-04-26T04:52:53.787 INFO:tasks.workunit.client.0.smithi077.stdout:2/146: chown d9/f2c 7285182 1 2024-04-26T04:52:53.790 INFO:tasks.workunit.client.1.smithi078.stdout:5/377: readlink d7/d48/d57/d65/l3e 0 2024-04-26T04:52:53.791 INFO:tasks.workunit.client.1.smithi078.stdout:5/378: fdatasync d7/d21/f4e 0 2024-04-26T04:52:53.794 INFO:tasks.workunit.client.1.smithi078.stdout:8/428: mkdir d0/d7a/d83 0 2024-04-26T04:52:53.795 INFO:tasks.workunit.client.1.smithi078.stdout:8/429: chown d0/cf 187 1 2024-04-26T04:52:53.800 INFO:tasks.workunit.client.0.smithi077.stdout:5/132: creat d1/d12/f2b x:0 0 0 2024-04-26T04:52:53.803 INFO:tasks.workunit.client.1.smithi078.stdout:2/431: mknod d8/d11/d14/d69/d8f/d34/c99 0 2024-04-26T04:52:53.803 INFO:tasks.workunit.client.1.smithi078.stdout:2/432: readlink d8/l42 0 2024-04-26T04:52:53.804 INFO:tasks.workunit.client.0.smithi077.stdout:5/133: dread d1/fe [0,4194304] 0 2024-04-26T04:52:53.804 INFO:tasks.workunit.client.0.smithi077.stdout:6/159: mkdir d8/d2a/d38 0 2024-04-26T04:52:53.807 INFO:tasks.workunit.client.0.smithi077.stdout:2/147: unlink d9/cb 0 2024-04-26T04:52:53.808 INFO:tasks.workunit.client.1.smithi078.stdout:2/433: dwrite d8/d3e/f43 [0,4194304] 0 2024-04-26T04:52:53.809 INFO:tasks.workunit.client.1.smithi078.stdout:4/478: rmdir d0/de/d22/d71 39 2024-04-26T04:52:53.811 INFO:tasks.workunit.client.1.smithi078.stdout:2/434: truncate d8/d11/d14/d69/d8f/d34/d70/f75 422140 0 2024-04-26T04:52:53.811 INFO:tasks.workunit.client.1.smithi078.stdout:9/408: sync 2024-04-26T04:52:53.811 INFO:tasks.workunit.client.1.smithi078.stdout:2/435: chown d8/d3e/f6a 0 1 2024-04-26T04:52:53.813 INFO:tasks.workunit.client.1.smithi078.stdout:0/534: rename d2/d9/d18/f22 to d2/d1e/d5a/d74/fb2 0 2024-04-26T04:52:53.815 INFO:tasks.workunit.client.1.smithi078.stdout:0/535: write d2/f6c [797046,93526] 0 2024-04-26T04:52:53.818 INFO:tasks.workunit.client.1.smithi078.stdout:9/409: dwrite dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:52:53.818 INFO:tasks.workunit.client.1.smithi078.stdout:1/508: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:53.820 INFO:tasks.workunit.client.1.smithi078.stdout:9/410: write dc/df/d4b/f79 [262052,81223] 0 2024-04-26T04:52:53.822 INFO:tasks.workunit.client.0.smithi077.stdout:5/134: mknod d1/c2c 0 2024-04-26T04:52:53.824 INFO:tasks.workunit.client.0.smithi077.stdout:5/135: read d1/d12/d28/f1c [7577943,101980] 0 2024-04-26T04:52:53.825 INFO:tasks.workunit.client.0.smithi077.stdout:2/148: dread d9/dc/d1d/d1e/f13 [0,4194304] 0 2024-04-26T04:52:53.826 INFO:tasks.workunit.client.1.smithi078.stdout:7/457: sync 2024-04-26T04:52:53.826 INFO:tasks.workunit.client.0.smithi077.stdout:1/241: getdents d9 0 2024-04-26T04:52:53.827 INFO:tasks.workunit.client.0.smithi077.stdout:1/242: write d9/f24 [1304128,63500] 0 2024-04-26T04:52:53.827 INFO:tasks.workunit.client.1.smithi078.stdout:3/454: dwrite d16/d26/f2c [4194304,4194304] 0 2024-04-26T04:52:53.829 INFO:tasks.workunit.client.0.smithi077.stdout:1/243: truncate d9/db/dd/d34/f38 700804 0 2024-04-26T04:52:53.830 INFO:tasks.workunit.client.0.smithi077.stdout:0/117: truncate d0/db/ff 1596401 0 2024-04-26T04:52:53.834 INFO:tasks.workunit.client.0.smithi077.stdout:5/136: fsync d1/d8/f22 0 2024-04-26T04:52:53.837 INFO:tasks.workunit.client.1.smithi078.stdout:0/536: dread d2/d9/d31/f85 [0,4194304] 0 2024-04-26T04:52:53.839 INFO:tasks.workunit.client.0.smithi077.stdout:5/137: dread d1/fe [0,4194304] 0 2024-04-26T04:52:53.839 INFO:tasks.workunit.client.1.smithi078.stdout:5/379: dwrite d7/d48/d57/d65/d46/f4f [0,4194304] 0 2024-04-26T04:52:53.840 INFO:tasks.workunit.client.0.smithi077.stdout:1/244: symlink d9/l46 0 2024-04-26T04:52:53.841 INFO:tasks.workunit.client.1.smithi078.stdout:6/531: dwrite d2/de/d27/f64 [4194304,4194304] 0 2024-04-26T04:52:53.841 INFO:tasks.workunit.client.0.smithi077.stdout:0/118: creat d0/d19/f2d x:0 0 0 2024-04-26T04:52:53.853 INFO:tasks.workunit.client.1.smithi078.stdout:3/455: dread d16/d26/f37 [0,4194304] 0 2024-04-26T04:52:53.905 INFO:tasks.workunit.client.1.smithi078.stdout:1/509: rmdir d7/d1e 39 2024-04-26T04:52:53.909 INFO:tasks.workunit.client.1.smithi078.stdout:9/411: mknod dc/df/d17/c7a 0 2024-04-26T04:52:53.909 INFO:tasks.workunit.client.1.smithi078.stdout:9/412: write dc/df/d27/f5c [27376,33080] 0 2024-04-26T04:52:53.912 INFO:tasks.workunit.client.0.smithi077.stdout:6/160: link d8/d1c/f24 d8/f39 0 2024-04-26T04:52:53.916 INFO:tasks.workunit.client.1.smithi078.stdout:4/479: mkdir d0/de/d22/d27/db5 0 2024-04-26T04:52:53.919 INFO:tasks.workunit.client.0.smithi077.stdout:1/245: fsync d9/db/f1c 0 2024-04-26T04:52:53.921 INFO:tasks.workunit.client.1.smithi078.stdout:2/436: unlink d8/d11/f22 0 2024-04-26T04:52:53.921 INFO:tasks.workunit.client.1.smithi078.stdout:2/437: read - d8/d3e/f58 zero size 2024-04-26T04:52:53.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/380: mknod d7/d48/d57/d65/d46/c7f 0 2024-04-26T04:52:53.922 INFO:tasks.workunit.client.0.smithi077.stdout:6/161: mknod d8/d2a/d38/c3a 0 2024-04-26T04:52:53.923 INFO:tasks.workunit.client.1.smithi078.stdout:5/381: readlink d7/da/d1e/l63 0 2024-04-26T04:52:53.924 INFO:tasks.workunit.client.0.smithi077.stdout:5/138: getdents d1/d12 0 2024-04-26T04:52:53.926 INFO:tasks.workunit.client.1.smithi078.stdout:1/510: unlink d7/d2e/f63 0 2024-04-26T04:52:53.927 INFO:tasks.workunit.client.0.smithi077.stdout:1/246: symlink d9/db/d2b/d3f/l47 0 2024-04-26T04:52:53.928 INFO:tasks.workunit.client.1.smithi078.stdout:5/382: dwrite d7/da/d1e/f4c [0,4194304] 0 2024-04-26T04:52:53.929 INFO:tasks.workunit.client.1.smithi078.stdout:9/413: mkdir dc/df/d27/d7b 0 2024-04-26T04:52:53.929 INFO:tasks.workunit.client.1.smithi078.stdout:5/383: write d7/d48/d57/d65/d46/f4f [2135486,76064] 0 2024-04-26T04:52:53.931 INFO:tasks.workunit.client.1.smithi078.stdout:5/384: write d7/d18/d32/f3b [2077600,35621] 0 2024-04-26T04:52:53.934 INFO:tasks.workunit.client.1.smithi078.stdout:8/430: symlink d0/d19/d52/d5c/d68/l84 0 2024-04-26T04:52:53.937 INFO:tasks.workunit.client.1.smithi078.stdout:4/480: creat d0/de/d22/d27/d56/d5a/fb6 x:0 0 0 2024-04-26T04:52:53.938 INFO:tasks.workunit.client.0.smithi077.stdout:5/139: dwrite d1/d8/ff [0,4194304] 0 2024-04-26T04:52:53.938 INFO:tasks.workunit.client.0.smithi077.stdout:0/119: dwrite d0/db/ff [0,4194304] 0 2024-04-26T04:52:53.938 INFO:tasks.workunit.client.0.smithi077.stdout:6/162: truncate d8/d1c/f23 892300 0 2024-04-26T04:52:53.940 INFO:tasks.workunit.client.0.smithi077.stdout:5/140: chown d1/d8/f22 925 1 2024-04-26T04:52:53.945 INFO:tasks.workunit.client.1.smithi078.stdout:7/458: write d9/da/d11/f2f [1828562,83670] 0 2024-04-26T04:52:53.947 INFO:tasks.workunit.client.1.smithi078.stdout:7/459: chown d9/da/d11/d13/c7e 61 1 2024-04-26T04:52:53.948 INFO:tasks.workunit.client.1.smithi078.stdout:2/438: rename d8/d11/d14/d69/d8f/d34/f7c to d8/f9a 0 2024-04-26T04:52:53.948 INFO:tasks.workunit.client.1.smithi078.stdout:2/439: readlink d8/l6f 0 2024-04-26T04:52:53.950 INFO:tasks.workunit.client.1.smithi078.stdout:6/532: link d2/d1a/d5/d3a/d42/f5a d2/d1a/d5/d3a/d42/f9c 0 2024-04-26T04:52:53.951 INFO:tasks.workunit.client.0.smithi077.stdout:1/247: write f7 [330918,67027] 0 2024-04-26T04:52:53.953 INFO:tasks.workunit.client.1.smithi078.stdout:3/456: symlink d16/d26/l9c 0 2024-04-26T04:52:53.955 INFO:tasks.workunit.client.1.smithi078.stdout:6/533: dwrite d2/de/f12 [0,4194304] 0 2024-04-26T04:52:53.956 INFO:tasks.workunit.client.0.smithi077.stdout:0/120: link d0/d19/d1e/f26 d0/d2c/f2e 0 2024-04-26T04:52:53.959 INFO:tasks.workunit.client.1.smithi078.stdout:0/537: dwrite d2/f3 [0,4194304] 0 2024-04-26T04:52:53.962 INFO:tasks.workunit.client.1.smithi078.stdout:9/414: symlink dc/df/d4b/d22/d2d/l7c 0 2024-04-26T04:52:53.964 INFO:tasks.workunit.client.0.smithi077.stdout:0/121: mkdir d0/d19/d1e/d2f 0 2024-04-26T04:52:53.964 INFO:tasks.workunit.client.0.smithi077.stdout:5/141: link d1/c1d d1/d12/d28/c2d 0 2024-04-26T04:52:53.964 INFO:tasks.workunit.client.0.smithi077.stdout:1/248: dwrite d9/f43 [0,4194304] 0 2024-04-26T04:52:53.965 INFO:tasks.workunit.client.0.smithi077.stdout:0/122: write d0/fe [18933,52459] 0 2024-04-26T04:52:53.967 INFO:tasks.workunit.client.1.smithi078.stdout:4/481: fdatasync d0/de/d22/d27/d7b/f82 0 2024-04-26T04:52:53.967 INFO:tasks.workunit.client.0.smithi077.stdout:0/123: chown d0/db/f22 12745465 1 2024-04-26T04:52:53.967 INFO:tasks.workunit.client.0.smithi077.stdout:6/163: getdents d8/d2a/d38 0 2024-04-26T04:52:53.969 INFO:tasks.workunit.client.0.smithi077.stdout:0/124: write d0/db/f22 [1017315,107530] 0 2024-04-26T04:52:53.970 INFO:tasks.workunit.client.0.smithi077.stdout:6/164: write f3 [5840506,15658] 0 2024-04-26T04:52:53.970 INFO:tasks.workunit.client.0.smithi077.stdout:5/142: creat d1/d12/d28/f2e x:0 0 0 2024-04-26T04:52:53.972 INFO:tasks.workunit.client.0.smithi077.stdout:5/143: fdatasync d1/d8/f22 0 2024-04-26T04:52:53.973 INFO:tasks.workunit.client.1.smithi078.stdout:2/440: write d8/d11/d14/f90 [1400159,49827] 0 2024-04-26T04:52:53.974 INFO:tasks.workunit.client.0.smithi077.stdout:1/249: dread d9/f24 [0,4194304] 0 2024-04-26T04:52:53.975 INFO:tasks.workunit.client.1.smithi078.stdout:2/441: fsync d8/d3e/f93 0 2024-04-26T04:52:53.975 INFO:tasks.workunit.client.0.smithi077.stdout:1/250: dread - d9/db/d27/f40 zero size 2024-04-26T04:52:53.976 INFO:tasks.workunit.client.1.smithi078.stdout:4/482: dwrite d0/de/d22/d27/d56/f61 [0,4194304] 0 2024-04-26T04:52:53.977 INFO:tasks.workunit.client.1.smithi078.stdout:2/442: write d8/d11/d14/d69/d8f/f88 [474486,9173] 0 2024-04-26T04:52:53.977 INFO:tasks.workunit.client.1.smithi078.stdout:4/483: chown d0/d25/f3a 0 1 2024-04-26T04:52:53.977 INFO:tasks.workunit.client.0.smithi077.stdout:0/125: write d0/f25 [869946,16559] 0 2024-04-26T04:52:53.978 INFO:tasks.workunit.client.1.smithi078.stdout:3/457: mknod d16/d26/d2e/d5f/d49/d77/c9d 0 2024-04-26T04:52:53.979 INFO:tasks.workunit.client.0.smithi077.stdout:0/126: readlink d0/db/d11/l18 0 2024-04-26T04:52:53.980 INFO:tasks.workunit.client.0.smithi077.stdout:5/144: symlink d1/d8/l2f 0 2024-04-26T04:52:53.980 INFO:tasks.workunit.client.1.smithi078.stdout:2/443: write d8/d11/d14/d69/d8f/f6c [3859809,98558] 0 2024-04-26T04:52:53.981 INFO:tasks.workunit.client.1.smithi078.stdout:3/458: truncate d16/d26/d2e/d74/f83 353378 0 2024-04-26T04:52:53.982 INFO:tasks.workunit.client.1.smithi078.stdout:0/538: symlink d2/d9/d60/lb3 0 2024-04-26T04:52:53.983 INFO:tasks.workunit.client.1.smithi078.stdout:3/459: fdatasync d16/d26/d2b/d66/f6f 0 2024-04-26T04:52:53.983 INFO:tasks.workunit.client.0.smithi077.stdout:1/251: mknod d9/db/d27/c48 0 2024-04-26T04:52:53.983 INFO:tasks.workunit.client.1.smithi078.stdout:9/415: creat dc/df/d4b/d22/d2a/f7d x:0 0 0 2024-04-26T04:52:53.984 INFO:tasks.workunit.client.1.smithi078.stdout:3/460: stat d16/d26/d2e/d5f/d49/d77/c9d 0 2024-04-26T04:52:53.985 INFO:tasks.workunit.client.0.smithi077.stdout:6/165: rename d8/c26 to d8/d1c/c3b 0 2024-04-26T04:52:53.985 INFO:tasks.workunit.client.1.smithi078.stdout:8/431: truncate d0/d19/d52/f29 1230801 0 2024-04-26T04:52:53.988 INFO:tasks.workunit.client.0.smithi077.stdout:1/252: creat d9/db/dd/f49 x:0 0 0 2024-04-26T04:52:53.989 INFO:tasks.workunit.client.0.smithi077.stdout:1/253: write d9/db/dd/f23 [2522774,5460] 0 2024-04-26T04:52:53.989 INFO:tasks.workunit.client.1.smithi078.stdout:1/511: dread d7/d1e/d9e/d17/f66 [4194304,4194304] 0 2024-04-26T04:52:53.990 INFO:tasks.workunit.client.1.smithi078.stdout:1/512: stat d7/c18 0 2024-04-26T04:52:53.990 INFO:tasks.workunit.client.0.smithi077.stdout:0/127: link d0/d19/f2d d0/db/f30 0 2024-04-26T04:52:53.991 INFO:tasks.workunit.client.0.smithi077.stdout:0/128: chown d0/d19/d1e 1 1 2024-04-26T04:52:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:53 smithi077 ceph-mon[26311]: pgmap v16: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 192 MiB/s wr, 390 op/s 2024-04-26T04:52:53.992 INFO:tasks.workunit.client.0.smithi077.stdout:5/145: mkdir d1/d30 0 2024-04-26T04:52:53.993 INFO:tasks.workunit.client.1.smithi078.stdout:5/385: link d7/da/d1e/l20 d7/d21/d2f/l80 0 2024-04-26T04:52:53.996 INFO:tasks.workunit.client.0.smithi077.stdout:1/254: dwrite d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:52:53.997 INFO:tasks.workunit.client.0.smithi077.stdout:1/255: read d9/db/d27/f28 [3592381,34338] 0 2024-04-26T04:52:53.999 INFO:tasks.workunit.client.1.smithi078.stdout:0/539: mkdir d2/d9/d60/d6a/db4 0 2024-04-26T04:52:54.003 INFO:tasks.workunit.client.0.smithi077.stdout:0/129: creat d0/db/f31 x:0 0 0 2024-04-26T04:52:54.005 INFO:tasks.workunit.client.1.smithi078.stdout:3/461: rmdir d16/d26/d2b/d31/d71 39 2024-04-26T04:52:54.007 INFO:tasks.workunit.client.0.smithi077.stdout:0/130: dread d0/f25 [0,4194304] 0 2024-04-26T04:52:54.008 INFO:tasks.workunit.client.0.smithi077.stdout:5/146: mkdir d1/d12/d21/d25/d31 0 2024-04-26T04:52:54.010 INFO:tasks.workunit.client.1.smithi078.stdout:3/462: dwrite d16/d26/d2e/d74/f76 [0,4194304] 0 2024-04-26T04:52:54.011 INFO:tasks.workunit.client.1.smithi078.stdout:6/534: creat d2/d1a/d5/f9d x:0 0 0 2024-04-26T04:52:54.012 INFO:tasks.workunit.client.0.smithi077.stdout:1/256: mkdir d9/db/dd/d34/d4a 0 2024-04-26T04:52:54.014 INFO:tasks.workunit.client.1.smithi078.stdout:7/460: write f7 [224182,22499] 0 2024-04-26T04:52:54.014 INFO:tasks.workunit.client.1.smithi078.stdout:1/513: symlink d7/d1e/d9e/db/d37/d58/la6 0 2024-04-26T04:52:54.017 INFO:tasks.workunit.client.1.smithi078.stdout:5/386: creat d7/d48/d57/d65/d46/f81 x:0 0 0 2024-04-26T04:52:54.018 INFO:tasks.workunit.client.1.smithi078.stdout:3/463: dwrite d16/d26/f32 [0,4194304] 0 2024-04-26T04:52:54.018 INFO:tasks.workunit.client.1.smithi078.stdout:5/387: chown d7/d18/f40 126262920 1 2024-04-26T04:52:54.018 INFO:tasks.workunit.client.1.smithi078.stdout:5/388: stat d7 0 2024-04-26T04:52:54.019 INFO:tasks.workunit.client.1.smithi078.stdout:0/540: mknod d2/d9/d60/d6a/d8f/cb5 0 2024-04-26T04:52:54.026 INFO:tasks.workunit.client.1.smithi078.stdout:6/535: symlink d2/de/d27/d58/l9e 0 2024-04-26T04:52:54.027 INFO:tasks.workunit.client.1.smithi078.stdout:4/484: creat d0/de/d22/fb7 x:0 0 0 2024-04-26T04:52:54.030 INFO:tasks.workunit.client.1.smithi078.stdout:1/514: mknod d7/d1e/d9e/db/d49/ca7 0 2024-04-26T04:52:54.031 INFO:tasks.workunit.client.1.smithi078.stdout:5/389: fdatasync d7/d18/d32/d37/f6d 0 2024-04-26T04:52:54.032 INFO:tasks.workunit.client.1.smithi078.stdout:5/390: chown d7/f39 16 1 2024-04-26T04:52:54.033 INFO:tasks.workunit.client.1.smithi078.stdout:0/541: mknod d2/d1e/d5a/d74/cb6 0 2024-04-26T04:52:54.034 INFO:tasks.workunit.client.1.smithi078.stdout:9/416: rmdir dc/df/d27/d7b 0 2024-04-26T04:52:54.035 INFO:tasks.workunit.client.1.smithi078.stdout:9/417: truncate dc/df/f2f 4575514 0 2024-04-26T04:52:54.035 INFO:tasks.workunit.client.1.smithi078.stdout:6/536: creat d2/de/d27/d4c/d5f/f9f x:0 0 0 2024-04-26T04:52:54.039 INFO:tasks.workunit.client.1.smithi078.stdout:1/515: unlink d7/d1e/d9e/f12 0 2024-04-26T04:52:54.039 INFO:tasks.workunit.client.1.smithi078.stdout:3/464: truncate d16/f20 299186 0 2024-04-26T04:52:54.040 INFO:tasks.workunit.client.1.smithi078.stdout:3/465: chown d16/d26/d2e/d74/f76 0 1 2024-04-26T04:52:54.045 INFO:tasks.workunit.client.1.smithi078.stdout:7/461: creat d9/da/d11/d13/f91 x:0 0 0 2024-04-26T04:52:54.046 INFO:tasks.workunit.client.1.smithi078.stdout:6/537: creat d2/fa0 x:0 0 0 2024-04-26T04:52:54.048 INFO:tasks.workunit.client.1.smithi078.stdout:0/542: rename d2/d9/d60/d6a/d8f to d2/d9/d18/db7 0 2024-04-26T04:52:54.049 INFO:tasks.workunit.client.1.smithi078.stdout:3/466: creat d16/d26/d2e/f9e x:0 0 0 2024-04-26T04:52:54.050 INFO:tasks.workunit.client.1.smithi078.stdout:6/538: mkdir d2/d1a/d5/da/d35/da1 0 2024-04-26T04:52:54.051 INFO:tasks.workunit.client.1.smithi078.stdout:8/432: write d0/d19/d52/f24 [324869,3184] 0 2024-04-26T04:52:54.051 INFO:tasks.workunit.client.1.smithi078.stdout:4/485: link d0/d25/f19 d0/d25/d20/d31/d47/db2/d4c/d8e/fb8 0 2024-04-26T04:52:54.054 INFO:tasks.workunit.client.1.smithi078.stdout:3/467: symlink d16/d7e/l9f 0 2024-04-26T04:52:54.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/539: truncate d2/d1a/d5/d11/f1c 5023149 0 2024-04-26T04:52:54.055 INFO:tasks.workunit.client.1.smithi078.stdout:0/543: dwrite d2/d1e/f67 [0,4194304] 0 2024-04-26T04:52:54.058 INFO:tasks.workunit.client.1.smithi078.stdout:0/544: write d2/f2e [4322991,19938] 0 2024-04-26T04:52:54.062 INFO:tasks.workunit.client.1.smithi078.stdout:8/433: dwrite d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:52:54.062 INFO:tasks.workunit.client.1.smithi078.stdout:6/540: dread d2/d1a/d5/d11/f4e [0,4194304] 0 2024-04-26T04:52:54.063 INFO:tasks.workunit.client.1.smithi078.stdout:6/541: fdatasync d2/f71 0 2024-04-26T04:52:54.069 INFO:tasks.workunit.client.1.smithi078.stdout:6/542: dwrite d2/d1a/d5/d11/f1c [0,4194304] 0 2024-04-26T04:52:54.070 INFO:tasks.workunit.client.1.smithi078.stdout:6/543: readlink d2/de/d27/l3f 0 2024-04-26T04:52:54.083 INFO:tasks.workunit.client.1.smithi078.stdout:9/418: rmdir dc 39 2024-04-26T04:52:54.084 INFO:tasks.workunit.client.0.smithi077.stdout:4/174: sync 2024-04-26T04:52:54.084 INFO:tasks.workunit.client.0.smithi077.stdout:9/114: sync 2024-04-26T04:52:54.090 INFO:tasks.workunit.client.0.smithi077.stdout:9/115: chown d9/c14 442 1 2024-04-26T04:52:54.093 INFO:tasks.workunit.client.1.smithi078.stdout:1/516: rename d7/d1e/d9e/d17/f66 to d7/d1e/d9e/dd/fa8 0 2024-04-26T04:52:54.095 INFO:tasks.workunit.client.0.smithi077.stdout:4/175: dwrite d4/d17/f24 [0,4194304] 0 2024-04-26T04:52:54.095 INFO:tasks.workunit.client.0.smithi077.stdout:1/257: creat d9/db/d2b/d3f/f4b x:0 0 0 2024-04-26T04:52:54.097 INFO:tasks.workunit.client.0.smithi077.stdout:6/166: dwrite d8/f13 [0,4194304] 0 2024-04-26T04:52:54.100 INFO:tasks.workunit.client.1.smithi078.stdout:3/468: write d16/d26/d2e/d74/f83 [137073,96517] 0 2024-04-26T04:52:54.106 INFO:tasks.workunit.client.0.smithi077.stdout:1/258: dwrite d9/db/d27/f30 [0,4194304] 0 2024-04-26T04:52:54.110 INFO:tasks.workunit.client.1.smithi078.stdout:2/444: sync 2024-04-26T04:52:54.119 INFO:tasks.workunit.client.0.smithi077.stdout:5/147: symlink d1/d12/d21/d25/d31/l32 0 2024-04-26T04:52:54.120 INFO:tasks.workunit.client.0.smithi077.stdout:5/148: dread - d1/d12/d28/f2e zero size 2024-04-26T04:52:54.129 INFO:tasks.workunit.client.0.smithi077.stdout:6/167: dread d8/f37 [0,4194304] 0 2024-04-26T04:52:54.134 INFO:tasks.workunit.client.1.smithi078.stdout:1/517: write d7/d1e/d9e/db/d37/d58/f5e [4343050,96562] 0 2024-04-26T04:52:54.138 INFO:tasks.workunit.client.0.smithi077.stdout:0/131: link d0/d2c/f2e d0/d19/f32 0 2024-04-26T04:52:54.141 INFO:tasks.workunit.client.1.smithi078.stdout:3/469: unlink d16/d26/d2b/d54/d5a/f70 0 2024-04-26T04:52:54.144 INFO:tasks.workunit.client.0.smithi077.stdout:4/176: symlink d4/d17/l36 0 2024-04-26T04:52:54.146 INFO:tasks.workunit.client.0.smithi077.stdout:6/168: chown f4 1608071503 1 2024-04-26T04:52:54.147 INFO:tasks.workunit.client.1.smithi078.stdout:3/470: dwrite d16/d26/d2b/d31/f52 [0,4194304] 0 2024-04-26T04:52:54.154 INFO:tasks.workunit.client.1.smithi078.stdout:5/391: sync 2024-04-26T04:52:54.154 INFO:tasks.workunit.client.1.smithi078.stdout:8/434: sync 2024-04-26T04:52:54.155 INFO:tasks.workunit.client.1.smithi078.stdout:8/435: chown d0/d19/d25/d36/l54 0 1 2024-04-26T04:52:54.155 INFO:tasks.workunit.client.0.smithi077.stdout:0/132: rename d0/db/l27 to d0/d28/l33 0 2024-04-26T04:52:54.157 INFO:tasks.workunit.client.1.smithi078.stdout:7/462: creat d9/da/d11/d13/d1a/d52/d5a/f92 x:0 0 0 2024-04-26T04:52:54.160 INFO:tasks.workunit.client.1.smithi078.stdout:7/463: write d9/da/d11/d13/d50/f7c [767385,53564] 0 2024-04-26T04:52:54.160 INFO:tasks.workunit.client.0.smithi077.stdout:5/149: mkdir d1/d30/d33 0 2024-04-26T04:52:54.161 INFO:tasks.workunit.client.1.smithi078.stdout:9/419: symlink dc/df/d4b/d22/d6e/l7e 0 2024-04-26T04:52:54.162 INFO:tasks.workunit.client.1.smithi078.stdout:8/436: dwrite d0/d19/d52/d35/d3f/f5e [0,4194304] 0 2024-04-26T04:52:54.162 INFO:tasks.workunit.client.0.smithi077.stdout:4/177: symlink d4/de/d19/l37 0 2024-04-26T04:52:54.165 INFO:tasks.workunit.client.0.smithi077.stdout:6/169: chown l5 451086 1 2024-04-26T04:52:54.166 INFO:tasks.workunit.client.0.smithi077.stdout:1/259: creat d9/db/f4c x:0 0 0 2024-04-26T04:52:54.173 INFO:tasks.workunit.client.1.smithi078.stdout:0/545: rmdir d2/d9/d18/db7/da8 0 2024-04-26T04:52:54.175 INFO:tasks.workunit.client.1.smithi078.stdout:3/471: chown d16/d26/d2b/f40 12 1 2024-04-26T04:52:54.180 INFO:tasks.workunit.client.0.smithi077.stdout:1/260: rename f2 to d9/db/dd/d34/f4d 0 2024-04-26T04:52:54.183 INFO:tasks.workunit.client.0.smithi077.stdout:0/133: chown d0/d19/d1e/f26 189051 1 2024-04-26T04:52:54.183 INFO:tasks.workunit.client.0.smithi077.stdout:0/134: fsync d0/f9 0 2024-04-26T04:52:54.187 INFO:tasks.workunit.client.1.smithi078.stdout:7/464: rmdir d9/da 39 2024-04-26T04:52:54.190 INFO:tasks.workunit.client.1.smithi078.stdout:9/420: rename dc/df/d27/f4f to dc/df/d4b/d22/d2a/f7f 0 2024-04-26T04:52:54.191 INFO:tasks.workunit.client.1.smithi078.stdout:8/437: mknod d0/d19/d25/d36/c85 0 2024-04-26T04:52:54.192 INFO:tasks.workunit.client.1.smithi078.stdout:8/438: chown d0/d19/d25/d36/d58 11 1 2024-04-26T04:52:54.192 INFO:tasks.workunit.client.0.smithi077.stdout:1/261: symlink d9/db/d27/l4e 0 2024-04-26T04:52:54.193 INFO:tasks.workunit.client.1.smithi078.stdout:8/439: write d0/d19/d52/d35/f4e [2184072,94150] 0 2024-04-26T04:52:54.194 INFO:tasks.workunit.client.1.smithi078.stdout:3/472: fdatasync f9 0 2024-04-26T04:52:54.199 INFO:tasks.workunit.client.1.smithi078.stdout:8/440: dwrite d0/d19/d52/d35/d44/f4f [0,4194304] 0 2024-04-26T04:52:54.199 INFO:tasks.workunit.client.1.smithi078.stdout:6/544: getdents d2/d1a/d5/d3a 0 2024-04-26T04:52:54.202 INFO:tasks.workunit.client.0.smithi077.stdout:4/178: rename d4/d17/d18/c1c to d4/de/d15/d1f/d35/c38 0 2024-04-26T04:52:54.206 INFO:tasks.workunit.client.1.smithi078.stdout:8/441: dwrite d0/d19/d25/f72 [0,4194304] 0 2024-04-26T04:52:54.212 INFO:tasks.workunit.client.0.smithi077.stdout:5/150: getdents d1/d12/d28 0 2024-04-26T04:52:54.222 INFO:tasks.workunit.client.0.smithi077.stdout:1/262: symlink d9/l4f 0 2024-04-26T04:52:54.227 INFO:tasks.workunit.client.0.smithi077.stdout:5/151: creat d1/d12/d21/f34 x:0 0 0 2024-04-26T04:52:54.228 INFO:tasks.workunit.client.0.smithi077.stdout:5/152: truncate d1/d8/f10 4973654 0 2024-04-26T04:52:54.229 INFO:tasks.workunit.client.0.smithi077.stdout:5/153: chown d1/d12/d21/f1b 39028 1 2024-04-26T04:52:54.230 INFO:tasks.workunit.client.0.smithi077.stdout:6/170: link d8/l1e d8/l3c 0 2024-04-26T04:52:54.231 INFO:tasks.workunit.client.0.smithi077.stdout:6/171: write d8/f11 [520426,76141] 0 2024-04-26T04:52:54.235 INFO:tasks.workunit.client.1.smithi078.stdout:8/442: unlink d0/d19/d25/d36/c85 0 2024-04-26T04:52:54.236 INFO:tasks.workunit.client.0.smithi077.stdout:1/263: stat d9/f1b 0 2024-04-26T04:52:54.238 INFO:tasks.workunit.client.1.smithi078.stdout:3/473: symlink d16/d26/d2b/d66/la0 0 2024-04-26T04:52:54.239 INFO:tasks.workunit.client.0.smithi077.stdout:5/154: mkdir d1/d30/d35 0 2024-04-26T04:52:54.240 INFO:tasks.workunit.client.0.smithi077.stdout:5/155: write d1/d12/f19 [3267430,15985] 0 2024-04-26T04:52:54.243 INFO:tasks.workunit.client.0.smithi077.stdout:6/172: mknod d8/d2a/d38/c3d 0 2024-04-26T04:52:54.245 INFO:tasks.workunit.client.0.smithi077.stdout:4/179: creat d4/f39 x:0 0 0 2024-04-26T04:52:54.245 INFO:tasks.workunit.client.1.smithi078.stdout:7/465: symlink d9/da/d11/d13/d50/d8a/d8f/l93 0 2024-04-26T04:52:54.246 INFO:tasks.workunit.client.0.smithi077.stdout:6/173: dread d8/f39 [0,4194304] 0 2024-04-26T04:52:54.247 INFO:tasks.workunit.client.1.smithi078.stdout:8/443: write d0/d19/d52/d3c/f60 [1734987,110993] 0 2024-04-26T04:52:54.252 INFO:tasks.workunit.client.0.smithi077.stdout:1/264: dwrite d9/f42 [0,4194304] 0 2024-04-26T04:52:54.252 INFO:tasks.workunit.client.0.smithi077.stdout:5/156: creat d1/d30/f36 x:0 0 0 2024-04-26T04:52:54.253 INFO:tasks.workunit.client.1.smithi078.stdout:5/392: link d7/cc d7/d11/c82 0 2024-04-26T04:52:54.254 INFO:tasks.workunit.client.1.smithi078.stdout:5/393: write d7/d48/d57/d65/f5b [893726,88702] 0 2024-04-26T04:52:54.254 INFO:tasks.workunit.client.1.smithi078.stdout:4/486: creat d0/d25/d20/d31/fb9 x:0 0 0 2024-04-26T04:52:54.259 INFO:tasks.workunit.client.1.smithi078.stdout:6/545: rmdir d2/d1a/d5/da/d35/da1 0 2024-04-26T04:52:54.259 INFO:tasks.workunit.client.1.smithi078.stdout:6/546: chown d2/d49/f47 190110095 1 2024-04-26T04:52:54.260 INFO:tasks.workunit.client.0.smithi077.stdout:6/174: creat d8/d1c/f3e x:0 0 0 2024-04-26T04:52:54.263 INFO:tasks.workunit.client.0.smithi077.stdout:4/180: dwrite d4/de/d15/d1f/f32 [4194304,4194304] 0 2024-04-26T04:52:54.264 INFO:tasks.workunit.client.0.smithi077.stdout:4/181: truncate d4/f39 698370 0 2024-04-26T04:52:54.264 INFO:tasks.workunit.client.1.smithi078.stdout:9/421: rmdir dc/df/d4b/d22 39 2024-04-26T04:52:54.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:53 smithi078 ceph-mon[33746]: pgmap v16: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 192 MiB/s wr, 390 op/s 2024-04-26T04:52:54.268 INFO:tasks.workunit.client.0.smithi077.stdout:4/182: dread d4/f8 [0,4194304] 0 2024-04-26T04:52:54.269 INFO:tasks.workunit.client.0.smithi077.stdout:5/157: symlink d1/d30/l37 0 2024-04-26T04:52:54.283 INFO:tasks.workunit.client.1.smithi078.stdout:2/445: rmdir d8/d11/d14 39 2024-04-26T04:52:54.284 INFO:tasks.workunit.client.0.smithi077.stdout:5/158: mkdir d1/d12/d28/d38 0 2024-04-26T04:52:54.285 INFO:tasks.workunit.client.1.smithi078.stdout:7/466: mknod d9/da/d11/d13/d1a/d65/c94 0 2024-04-26T04:52:54.287 INFO:tasks.workunit.client.1.smithi078.stdout:9/422: mkdir dc/df/d4b/d3e/d80 0 2024-04-26T04:52:54.287 INFO:tasks.workunit.client.1.smithi078.stdout:4/487: dread d0/de/d22/d27/d56/f60 [0,4194304] 0 2024-04-26T04:52:54.288 INFO:tasks.workunit.client.1.smithi078.stdout:8/444: creat d0/d19/d52/d35/d44/d6b/d6c/f86 x:0 0 0 2024-04-26T04:52:54.289 INFO:tasks.workunit.client.1.smithi078.stdout:6/547: dwrite d2/d1a/d5/d3a/d42/f7d [0,4194304] 0 2024-04-26T04:52:54.289 INFO:tasks.workunit.client.0.smithi077.stdout:4/183: symlink d4/d17/l3a 0 2024-04-26T04:52:54.291 INFO:tasks.workunit.client.1.smithi078.stdout:1/518: unlink d7/d1e/d9e/d17/f3c 0 2024-04-26T04:52:54.294 INFO:tasks.workunit.client.1.smithi078.stdout:8/445: dread d0/d19/d52/d35/d3f/f5e [0,4194304] 0 2024-04-26T04:52:54.302 INFO:tasks.workunit.client.1.smithi078.stdout:0/546: truncate d2/f70 36897 0 2024-04-26T04:52:54.305 INFO:tasks.workunit.client.1.smithi078.stdout:6/548: dread - d2/d1a/d41/f7e zero size 2024-04-26T04:52:54.305 INFO:tasks.workunit.client.0.smithi077.stdout:0/135: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:52:54.306 INFO:tasks.workunit.client.0.smithi077.stdout:4/184: creat d4/d17/f3b x:0 0 0 2024-04-26T04:52:54.307 INFO:tasks.workunit.client.0.smithi077.stdout:4/185: fsync d4/d17/f24 0 2024-04-26T04:52:54.308 INFO:tasks.workunit.client.0.smithi077.stdout:4/186: stat d4/f14 0 2024-04-26T04:52:54.312 INFO:tasks.workunit.client.0.smithi077.stdout:0/136: fdatasync d0/d19/f2d 0 2024-04-26T04:52:54.313 INFO:tasks.workunit.client.0.smithi077.stdout:0/137: readlink d0/l6 0 2024-04-26T04:52:54.314 INFO:tasks.workunit.client.0.smithi077.stdout:0/138: fsync d0/fe 0 2024-04-26T04:52:54.317 INFO:tasks.workunit.client.0.smithi077.stdout:4/187: dwrite d4/de/d15/d1f/f32 [0,4194304] 0 2024-04-26T04:52:54.318 INFO:tasks.workunit.client.1.smithi078.stdout:3/474: creat d16/d26/d2b/d31/d71/d8a/d98/fa1 x:0 0 0 2024-04-26T04:52:54.318 INFO:tasks.workunit.client.1.smithi078.stdout:8/446: write d0/d19/f1c [4910396,46955] 0 2024-04-26T04:52:54.320 INFO:tasks.workunit.client.1.smithi078.stdout:5/394: link d7/da/f24 d7/d18/d1c/f83 0 2024-04-26T04:52:54.322 INFO:tasks.workunit.client.0.smithi077.stdout:0/139: dwrite d0/fe [0,4194304] 0 2024-04-26T04:52:54.323 INFO:tasks.workunit.client.0.smithi077.stdout:4/188: stat d4/de/d15/d1f/d35/c38 0 2024-04-26T04:52:54.326 INFO:tasks.workunit.client.1.smithi078.stdout:7/467: truncate d9/da/f23 7767081 0 2024-04-26T04:52:54.327 INFO:tasks.workunit.client.0.smithi077.stdout:0/140: creat d0/d19/d1e/f34 x:0 0 0 2024-04-26T04:52:54.329 INFO:tasks.workunit.client.1.smithi078.stdout:9/423: rename dc/df/d37/d64 to dc/df/d27/d81 0 2024-04-26T04:52:54.330 INFO:tasks.workunit.client.0.smithi077.stdout:8/151: sync 2024-04-26T04:52:54.330 INFO:tasks.workunit.client.0.smithi077.stdout:3/146: sync 2024-04-26T04:52:54.330 INFO:tasks.workunit.client.0.smithi077.stdout:9/116: sync 2024-04-26T04:52:54.330 INFO:tasks.workunit.client.0.smithi077.stdout:2/149: sync 2024-04-26T04:52:54.330 INFO:tasks.workunit.client.0.smithi077.stdout:7/151: sync 2024-04-26T04:52:54.331 INFO:tasks.workunit.client.1.smithi078.stdout:0/547: creat d2/d9/d18/fb8 x:0 0 0 2024-04-26T04:52:54.332 INFO:tasks.workunit.client.0.smithi077.stdout:9/117: dread - d9/f26 zero size 2024-04-26T04:52:54.333 INFO:tasks.workunit.client.0.smithi077.stdout:0/141: creat d0/d19/f35 x:0 0 0 2024-04-26T04:52:54.333 INFO:tasks.workunit.client.0.smithi077.stdout:0/142: chown d0/d19 1754019 1 2024-04-26T04:52:54.335 INFO:tasks.workunit.client.1.smithi078.stdout:6/549: fdatasync d2/d1a/d5/d2a/d4d/f60 0 2024-04-26T04:52:54.337 INFO:tasks.workunit.client.0.smithi077.stdout:7/152: mkdir da/d15/d1a/d39 0 2024-04-26T04:52:54.339 INFO:tasks.workunit.client.0.smithi077.stdout:7/153: dread - da/d15/f2f zero size 2024-04-26T04:52:54.339 INFO:tasks.workunit.client.0.smithi077.stdout:2/150: creat d9/d15/d1a/f30 x:0 0 0 2024-04-26T04:52:54.340 INFO:tasks.workunit.client.1.smithi078.stdout:6/550: dwrite d2/d1a/d5/d2a/d4d/f60 [4194304,4194304] 0 2024-04-26T04:52:54.342 INFO:tasks.workunit.client.1.smithi078.stdout:6/551: write d2/de/d27/d4c/d5f/f9f [753702,51215] 0 2024-04-26T04:52:54.343 INFO:tasks.workunit.client.0.smithi077.stdout:8/152: truncate d4/df/f1f 373684 0 2024-04-26T04:52:54.343 INFO:tasks.workunit.client.0.smithi077.stdout:8/153: fdatasync d4/df/d22/d2d/d2b/f2f 0 2024-04-26T04:52:54.344 INFO:tasks.workunit.client.0.smithi077.stdout:9/118: mknod d9/dd/c2a 0 2024-04-26T04:52:54.344 INFO:tasks.workunit.client.0.smithi077.stdout:9/119: stat d9/dd 0 2024-04-26T04:52:54.345 INFO:tasks.workunit.client.0.smithi077.stdout:8/154: chown d4/df/d22/f34 1 1 2024-04-26T04:52:54.345 INFO:tasks.workunit.client.0.smithi077.stdout:9/120: readlink d9/l1d 0 2024-04-26T04:52:54.346 INFO:tasks.workunit.client.1.smithi078.stdout:2/446: link d8/d11/d18/f47 d8/d11/d18/d5d/d61/f9b 0 2024-04-26T04:52:54.346 INFO:tasks.workunit.client.0.smithi077.stdout:0/143: mknod d0/d19/d1e/c36 0 2024-04-26T04:52:54.348 INFO:tasks.workunit.client.1.smithi078.stdout:7/468: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d95 0 2024-04-26T04:52:54.351 INFO:tasks.workunit.client.0.smithi077.stdout:2/151: creat d9/d15/f31 x:0 0 0 2024-04-26T04:52:54.352 INFO:tasks.workunit.client.1.smithi078.stdout:9/424: dread dc/f54 [0,4194304] 0 2024-04-26T04:52:54.352 INFO:tasks.workunit.client.0.smithi077.stdout:9/121: dwrite d9/dd/f1a [0,4194304] 0 2024-04-26T04:52:54.353 INFO:tasks.workunit.client.1.smithi078.stdout:1/519: rename d7/d1e/d9e/db/d47/d4a/f9c to d7/d1e/d9e/db/d37/fa9 0 2024-04-26T04:52:54.353 INFO:tasks.workunit.client.1.smithi078.stdout:9/425: write dc/d55/f52 [994281,10117] 0 2024-04-26T04:52:54.355 INFO:tasks.workunit.client.0.smithi077.stdout:1/265: dwrite d9/db/f1c [0,4194304] 0 2024-04-26T04:52:54.356 INFO:tasks.workunit.client.1.smithi078.stdout:3/475: link d16/d26/f7f d16/d26/d2b/d54/d56/fa2 0 2024-04-26T04:52:54.358 INFO:tasks.workunit.client.0.smithi077.stdout:4/189: getdents d4/de/d15 0 2024-04-26T04:52:54.362 INFO:tasks.workunit.client.1.smithi078.stdout:6/552: symlink d2/d1a/d5/d11/la2 0 2024-04-26T04:52:54.363 INFO:tasks.workunit.client.1.smithi078.stdout:6/553: chown d2/d1a/d5/d2a/d4d/c75 5 1 2024-04-26T04:52:54.366 INFO:tasks.workunit.client.1.smithi078.stdout:7/469: mknod d9/da/d11/d13/d1a/d52/d5a/d41/c96 0 2024-04-26T04:52:54.369 INFO:tasks.workunit.client.0.smithi077.stdout:6/175: truncate d8/f22 191829 0 2024-04-26T04:52:54.370 INFO:tasks.workunit.client.0.smithi077.stdout:9/122: fsync d9/dd/d1b/f23 0 2024-04-26T04:52:54.372 INFO:tasks.workunit.client.1.smithi078.stdout:9/426: chown dc/df/d4b/d22/d2d/f4d 30 1 2024-04-26T04:52:54.372 INFO:tasks.workunit.client.0.smithi077.stdout:5/159: truncate d1/d12/f19 519278 0 2024-04-26T04:52:54.375 INFO:tasks.workunit.client.0.smithi077.stdout:3/147: rmdir d0/d7/d27 0 2024-04-26T04:52:54.377 INFO:tasks.workunit.client.1.smithi078.stdout:1/520: creat d7/d1e/d9e/db/d37/faa x:0 0 0 2024-04-26T04:52:54.378 INFO:tasks.workunit.client.1.smithi078.stdout:1/521: write d7/d1e/d9e/d17/fa1 [1385958,5509] 0 2024-04-26T04:52:54.378 INFO:tasks.workunit.client.1.smithi078.stdout:3/476: mknod d16/d26/d2e/ca3 0 2024-04-26T04:52:54.379 INFO:tasks.workunit.client.0.smithi077.stdout:8/155: mkdir d4/d17/d35/d37 0 2024-04-26T04:52:54.380 INFO:tasks.workunit.client.1.smithi078.stdout:8/447: creat d0/d19/d25/f87 x:0 0 0 2024-04-26T04:52:54.381 INFO:tasks.workunit.client.0.smithi077.stdout:4/190: fsync d4/de/d15/f16 0 2024-04-26T04:52:54.381 INFO:tasks.workunit.client.1.smithi078.stdout:8/448: write d0/d19/d25/d36/f37 [2407617,111298] 0 2024-04-26T04:52:54.382 INFO:tasks.workunit.client.1.smithi078.stdout:3/477: dread d16/d26/d2b/d31/f55 [0,4194304] 0 2024-04-26T04:52:54.384 INFO:tasks.workunit.client.1.smithi078.stdout:6/554: symlink d2/de/la3 0 2024-04-26T04:52:54.385 INFO:tasks.workunit.client.1.smithi078.stdout:6/555: write d2/d1a/d5/f8 [2808097,91633] 0 2024-04-26T04:52:54.387 INFO:tasks.workunit.client.1.smithi078.stdout:2/447: symlink d8/d11/d14/d69/d8f/d34/l9c 0 2024-04-26T04:52:54.388 INFO:tasks.workunit.client.0.smithi077.stdout:4/191: dwrite d4/d17/f3b [0,4194304] 0 2024-04-26T04:52:54.389 INFO:tasks.workunit.client.0.smithi077.stdout:2/152: creat d9/d15/d1a/d2b/f32 x:0 0 0 2024-04-26T04:52:54.391 INFO:tasks.workunit.client.1.smithi078.stdout:6/556: dwrite d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:54.393 INFO:tasks.workunit.client.0.smithi077.stdout:2/153: dread d9/ff [0,4194304] 0 2024-04-26T04:52:54.394 INFO:tasks.workunit.client.0.smithi077.stdout:2/154: chown d9/dc/d1d/c29 213 1 2024-04-26T04:52:54.400 INFO:tasks.workunit.client.0.smithi077.stdout:5/160: write d1/d8/f10 [3495364,32477] 0 2024-04-26T04:52:54.401 INFO:tasks.workunit.client.1.smithi078.stdout:4/488: dwrite d0/d25/d20/d31/d47/db2/d4c/d8e/fb8 [0,4194304] 0 2024-04-26T04:52:54.401 INFO:tasks.workunit.client.0.smithi077.stdout:5/161: fdatasync d1/d12/f1f 0 2024-04-26T04:52:54.408 INFO:tasks.workunit.client.1.smithi078.stdout:5/395: getdents d7 0 2024-04-26T04:52:54.413 INFO:tasks.workunit.client.1.smithi078.stdout:3/478: unlink d16/f39 0 2024-04-26T04:52:54.414 INFO:tasks.workunit.client.0.smithi077.stdout:0/144: link d0/l7 d0/d2c/l37 0 2024-04-26T04:52:54.417 INFO:tasks.workunit.client.0.smithi077.stdout:7/154: getdents da 0 2024-04-26T04:52:54.418 INFO:tasks.workunit.client.1.smithi078.stdout:5/396: dread d7/d11/f2b [0,4194304] 0 2024-04-26T04:52:54.422 INFO:tasks.workunit.client.0.smithi077.stdout:4/192: symlink d4/de/l3c 0 2024-04-26T04:52:54.424 INFO:tasks.workunit.client.0.smithi077.stdout:4/193: chown d4/de/d15/d1f/d25/c2c 45527 1 2024-04-26T04:52:54.426 INFO:tasks.workunit.client.1.smithi078.stdout:7/470: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d68/d77 0 2024-04-26T04:52:54.428 INFO:tasks.workunit.client.1.smithi078.stdout:6/557: symlink d2/d1a/d5/da/la4 0 2024-04-26T04:52:54.434 INFO:tasks.workunit.client.1.smithi078.stdout:6/558: dwrite d2/d1a/d5/d11/f4f [0,4194304] 0 2024-04-26T04:52:54.434 INFO:tasks.workunit.client.1.smithi078.stdout:4/489: mkdir d0/d25/d20/d31/d47/db2/d4c/dba 0 2024-04-26T04:52:54.435 INFO:tasks.workunit.client.1.smithi078.stdout:4/490: chown d0/d25/d20/d31/d47/db2/d44/d55/d5e/d75 2118 1 2024-04-26T04:52:54.437 INFO:tasks.workunit.client.1.smithi078.stdout:0/548: sync 2024-04-26T04:52:54.444 INFO:tasks.workunit.client.1.smithi078.stdout:7/471: dread d9/da/d11/d13/d1a/d52/d5a/d41/f47 [4194304,4194304] 0 2024-04-26T04:52:54.446 INFO:tasks.workunit.client.1.smithi078.stdout:1/522: write f1 [2629194,65160] 0 2024-04-26T04:52:54.448 INFO:tasks.workunit.client.0.smithi077.stdout:0/145: unlink d0/f24 0 2024-04-26T04:52:54.449 INFO:tasks.workunit.client.0.smithi077.stdout:3/148: write d0/d9/f23 [478976,103697] 0 2024-04-26T04:52:54.450 INFO:tasks.workunit.client.0.smithi077.stdout:7/155: mknod da/d15/c3a 0 2024-04-26T04:52:54.450 INFO:tasks.workunit.client.0.smithi077.stdout:3/149: chown d0/f1 7 1 2024-04-26T04:52:54.454 INFO:tasks.workunit.client.1.smithi078.stdout:8/449: dwrite d0/d19/d52/d5c/f74 [0,4194304] 0 2024-04-26T04:52:54.457 INFO:tasks.workunit.client.0.smithi077.stdout:6/176: creat d8/f3f x:0 0 0 2024-04-26T04:52:54.457 INFO:tasks.workunit.client.0.smithi077.stdout:3/150: dwrite d0/d7/dd/f2f [0,4194304] 0 2024-04-26T04:52:54.458 INFO:tasks.workunit.client.1.smithi078.stdout:2/448: write d8/d11/d14/d69/d8f/f49 [536308,13078] 0 2024-04-26T04:52:54.459 INFO:tasks.workunit.client.1.smithi078.stdout:2/449: write d8/d3e/f93 [460155,45019] 0 2024-04-26T04:52:54.461 INFO:tasks.workunit.client.0.smithi077.stdout:2/155: symlink d9/dc/d1d/l33 0 2024-04-26T04:52:54.462 INFO:tasks.workunit.client.1.smithi078.stdout:4/491: write d0/de/f34 [1097032,12722] 0 2024-04-26T04:52:54.463 INFO:tasks.workunit.client.0.smithi077.stdout:9/123: rmdir d9/dd/d1f 0 2024-04-26T04:52:54.466 INFO:tasks.workunit.client.0.smithi077.stdout:5/162: mknod d1/c39 0 2024-04-26T04:52:54.467 INFO:tasks.workunit.client.1.smithi078.stdout:3/479: creat d16/d26/d2e/d5f/d49/fa4 x:0 0 0 2024-04-26T04:52:54.468 INFO:tasks.workunit.client.1.smithi078.stdout:3/480: stat d16/d26/d2e/d74 0 2024-04-26T04:52:54.468 INFO:tasks.workunit.client.0.smithi077.stdout:1/266: getdents d9/db/d27 0 2024-04-26T04:52:54.469 INFO:tasks.workunit.client.1.smithi078.stdout:3/481: fdatasync d16/d26/d2b/d54/d56/f80 0 2024-04-26T04:52:54.469 INFO:tasks.workunit.client.1.smithi078.stdout:3/482: stat d16/d26/d2b/l6a 0 2024-04-26T04:52:54.470 INFO:tasks.workunit.client.0.smithi077.stdout:8/156: rmdir d4/d17/d35/d37 0 2024-04-26T04:52:54.475 INFO:tasks.workunit.client.1.smithi078.stdout:3/483: dwrite d16/d26/d2b/d54/d5a/f82 [0,4194304] 0 2024-04-26T04:52:54.477 INFO:tasks.workunit.client.0.smithi077.stdout:0/146: truncate d0/f25 54802 0 2024-04-26T04:52:54.478 INFO:tasks.workunit.client.0.smithi077.stdout:3/151: dread d0/d7/f8 [0,4194304] 0 2024-04-26T04:52:54.479 INFO:tasks.workunit.client.0.smithi077.stdout:7/156: symlink da/d13/d22/l3b 0 2024-04-26T04:52:54.480 INFO:tasks.workunit.client.0.smithi077.stdout:4/194: write d4/de/d15/f16 [3726223,123523] 0 2024-04-26T04:52:54.482 INFO:tasks.workunit.client.1.smithi078.stdout:6/559: mknod d2/d1a/d5/ca5 0 2024-04-26T04:52:54.485 INFO:tasks.workunit.client.0.smithi077.stdout:6/177: chown d8/l1e 7 1 2024-04-26T04:52:54.486 INFO:tasks.workunit.client.1.smithi078.stdout:0/549: write d2/f48 [754289,104305] 0 2024-04-26T04:52:54.486 INFO:tasks.workunit.client.0.smithi077.stdout:6/178: write d8/d1c/f28 [489142,58450] 0 2024-04-26T04:52:54.486 INFO:tasks.workunit.client.1.smithi078.stdout:0/550: fdatasync d2/d9/d18/f49 0 2024-04-26T04:52:54.488 INFO:tasks.workunit.client.1.smithi078.stdout:9/427: sync 2024-04-26T04:52:54.488 INFO:tasks.workunit.client.1.smithi078.stdout:4/492: rename d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/l30 to d0/d25/d20/d31/d47/db2/d44/d55/d5e/lbb 0 2024-04-26T04:52:54.495 INFO:tasks.workunit.client.1.smithi078.stdout:1/523: mkdir d7/d1e/d9e/db/d37/d58/d86/dab 0 2024-04-26T04:52:54.496 INFO:tasks.workunit.client.0.smithi077.stdout:1/267: creat d9/db/d2b/d3f/f50 x:0 0 0 2024-04-26T04:52:54.496 INFO:tasks.workunit.client.1.smithi078.stdout:1/524: fsync d7/d1e/d9e/db/f41 0 2024-04-26T04:52:54.498 INFO:tasks.workunit.client.0.smithi077.stdout:8/157: symlink d4/df/d22/d2d/l38 0 2024-04-26T04:52:54.499 INFO:tasks.workunit.client.0.smithi077.stdout:8/158: dread d4/df/d22/f34 [0,4194304] 0 2024-04-26T04:52:54.500 INFO:tasks.workunit.client.1.smithi078.stdout:8/450: creat d0/d19/d25/d36/d58/d5b/d61/d67/f88 x:0 0 0 2024-04-26T04:52:54.501 INFO:tasks.workunit.client.1.smithi078.stdout:8/451: write d0/d19/d52/f24 [1299061,48876] 0 2024-04-26T04:52:54.502 INFO:tasks.workunit.client.1.smithi078.stdout:6/560: dread - d2/d1a/d5/d2a/f56 zero size 2024-04-26T04:52:54.504 INFO:tasks.workunit.client.0.smithi077.stdout:3/152: rename d0/d7/dd to d0/d9/d32 0 2024-04-26T04:52:54.504 INFO:tasks.workunit.client.0.smithi077.stdout:9/124: write f2 [4355503,36995] 0 2024-04-26T04:52:54.505 INFO:tasks.workunit.client.0.smithi077.stdout:3/153: chown d0/d9/f2b 30 1 2024-04-26T04:52:54.505 INFO:tasks.workunit.client.1.smithi078.stdout:0/551: dread d2/d9/d31/f50 [0,4194304] 0 2024-04-26T04:52:54.506 INFO:tasks.workunit.client.0.smithi077.stdout:3/154: dread - d0/d9/f14 zero size 2024-04-26T04:52:54.507 INFO:tasks.workunit.client.1.smithi078.stdout:0/552: dread - d2/d9/d31/d4a/f80 zero size 2024-04-26T04:52:54.507 INFO:tasks.workunit.client.1.smithi078.stdout:8/452: dwrite d0/d19/d52/d5c/f74 [0,4194304] 0 2024-04-26T04:52:54.508 INFO:tasks.workunit.client.0.smithi077.stdout:9/125: read d9/dd/d1b/f23 [3042287,107942] 0 2024-04-26T04:52:54.510 INFO:tasks.workunit.client.0.smithi077.stdout:2/156: mkdir d9/dc/d1d/d34 0 2024-04-26T04:52:54.511 INFO:tasks.workunit.client.0.smithi077.stdout:2/157: dread - d9/d15/d1a/f1c zero size 2024-04-26T04:52:54.512 INFO:tasks.workunit.client.1.smithi078.stdout:9/428: write dc/df/f59 [3210422,22569] 0 2024-04-26T04:52:54.514 INFO:tasks.workunit.client.1.smithi078.stdout:4/493: symlink d0/d25/d20/d31/d47/db2/d44/d69/lbc 0 2024-04-26T04:52:54.516 INFO:tasks.workunit.client.0.smithi077.stdout:1/268: creat d9/db/dd/d34/f51 x:0 0 0 2024-04-26T04:52:54.516 INFO:tasks.workunit.client.1.smithi078.stdout:4/494: read d0/de/d22/d27/d56/d5a/f7d [92757,72223] 0 2024-04-26T04:52:54.517 INFO:tasks.workunit.client.1.smithi078.stdout:5/397: getdents d7/da/d1e 0 2024-04-26T04:52:54.518 INFO:tasks.workunit.client.0.smithi077.stdout:5/163: truncate d1/d12/f19 197849 0 2024-04-26T04:52:54.519 INFO:tasks.workunit.client.0.smithi077.stdout:5/164: chown d1/d12/d21/l29 4116 1 2024-04-26T04:52:54.519 INFO:tasks.workunit.client.1.smithi078.stdout:4/495: chown d0/d25/d20/d31/d47/db2/d4c/d62/c92 2962 1 2024-04-26T04:52:54.519 INFO:tasks.workunit.client.1.smithi078.stdout:5/398: truncate d7/d48/d57/d65/f67 976614 0 2024-04-26T04:52:54.519 INFO:tasks.workunit.client.1.smithi078.stdout:4/496: fsync d0/de/d22/d27/d56/d5a/f7d 0 2024-04-26T04:52:54.520 INFO:tasks.workunit.client.1.smithi078.stdout:3/484: mkdir d16/d3c/d81/da5 0 2024-04-26T04:52:54.520 INFO:tasks.workunit.client.1.smithi078.stdout:4/497: write d0/de/d22/d27/d56/f61 [1972900,122153] 0 2024-04-26T04:52:54.521 INFO:tasks.workunit.client.0.smithi077.stdout:6/179: rename d8/d1c/f23 to d8/d2a/f40 0 2024-04-26T04:52:54.524 INFO:tasks.workunit.client.1.smithi078.stdout:6/561: fdatasync d2/de/d27/f34 0 2024-04-26T04:52:54.524 INFO:tasks.workunit.client.0.smithi077.stdout:3/155: creat d0/d7/f33 x:0 0 0 2024-04-26T04:52:54.529 INFO:tasks.workunit.client.1.smithi078.stdout:8/453: rename d0/l39 to d0/d7a/l89 0 2024-04-26T04:52:54.529 INFO:tasks.workunit.client.0.smithi077.stdout:9/126: mkdir d9/dd/d1b/d2b 0 2024-04-26T04:52:54.530 INFO:tasks.workunit.client.1.smithi078.stdout:3/485: dwrite d16/d26/d2b/d31/d71/d8a/d98/fa1 [0,4194304] 0 2024-04-26T04:52:54.535 INFO:tasks.workunit.client.1.smithi078.stdout:9/429: dread dc/df/d17/f44 [0,4194304] 0 2024-04-26T04:52:54.536 INFO:tasks.workunit.client.0.smithi077.stdout:9/127: dwrite d9/f26 [0,4194304] 0 2024-04-26T04:52:54.538 INFO:tasks.workunit.client.0.smithi077.stdout:1/269: unlink d9/db/dd/c1e 0 2024-04-26T04:52:54.538 INFO:tasks.workunit.client.1.smithi078.stdout:9/430: dread dc/f54 [0,4194304] 0 2024-04-26T04:52:54.538 INFO:tasks.workunit.client.0.smithi077.stdout:1/270: stat d9/db/dd/f11 0 2024-04-26T04:52:54.544 INFO:tasks.workunit.client.1.smithi078.stdout:7/472: getdents d9/da/d11/d13/d50/d8a 0 2024-04-26T04:52:54.545 INFO:tasks.workunit.client.0.smithi077.stdout:8/159: write d4/df/d22/d2d/f1e [1347050,116894] 0 2024-04-26T04:52:54.545 INFO:tasks.workunit.client.0.smithi077.stdout:6/180: mknod d8/d2a/c41 0 2024-04-26T04:52:54.546 INFO:tasks.workunit.client.1.smithi078.stdout:1/525: symlink d7/d1e/d9e/db/d47/d4a/d87/lac 0 2024-04-26T04:52:54.547 INFO:tasks.workunit.client.0.smithi077.stdout:7/157: link da/f18 da/f3c 0 2024-04-26T04:52:54.547 INFO:tasks.workunit.client.0.smithi077.stdout:8/160: chown d4/df/d22/d2d/f1e 0 1 2024-04-26T04:52:54.548 INFO:tasks.workunit.client.0.smithi077.stdout:8/161: chown f0 54 1 2024-04-26T04:52:54.549 INFO:tasks.workunit.client.1.smithi078.stdout:5/399: unlink d7/f16 0 2024-04-26T04:52:54.553 INFO:tasks.workunit.client.0.smithi077.stdout:5/165: dwrite d1/d12/d28/f1c [0,4194304] 0 2024-04-26T04:52:54.556 INFO:tasks.workunit.client.0.smithi077.stdout:3/156: creat d0/d7/f34 x:0 0 0 2024-04-26T04:52:54.556 INFO:tasks.workunit.client.0.smithi077.stdout:4/195: creat d4/d17/f3d x:0 0 0 2024-04-26T04:52:54.557 INFO:tasks.workunit.client.0.smithi077.stdout:2/158: mknod d9/dc/c35 0 2024-04-26T04:52:54.558 INFO:tasks.workunit.client.0.smithi077.stdout:6/181: dwrite d8/ff [4194304,4194304] 0 2024-04-26T04:52:54.563 INFO:tasks.workunit.client.0.smithi077.stdout:8/162: dwrite d4/f9 [0,4194304] 0 2024-04-26T04:52:54.564 INFO:tasks.workunit.client.0.smithi077.stdout:9/128: creat d9/dd/d1b/f2c x:0 0 0 2024-04-26T04:52:54.566 INFO:tasks.workunit.client.0.smithi077.stdout:1/271: creat d9/db/d2b/d3f/f52 x:0 0 0 2024-04-26T04:52:54.567 INFO:tasks.workunit.client.0.smithi077.stdout:3/157: read d0/d9/d32/d29/f31 [2496218,88995] 0 2024-04-26T04:52:54.569 INFO:tasks.workunit.client.0.smithi077.stdout:0/147: getdents d0/d28 0 2024-04-26T04:52:54.576 INFO:tasks.workunit.client.0.smithi077.stdout:5/166: creat d1/d12/d21/d25/f3a x:0 0 0 2024-04-26T04:52:54.577 INFO:tasks.workunit.client.0.smithi077.stdout:5/167: write d1/d12/f13 [1365163,37497] 0 2024-04-26T04:52:54.579 INFO:tasks.workunit.client.0.smithi077.stdout:6/182: unlink c1 0 2024-04-26T04:52:54.581 INFO:tasks.workunit.client.0.smithi077.stdout:9/129: fsync f3 0 2024-04-26T04:52:54.585 INFO:tasks.workunit.client.0.smithi077.stdout:8/163: creat d4/df/d22/d2d/d2b/d33/f39 x:0 0 0 2024-04-26T04:52:54.587 INFO:tasks.workunit.client.0.smithi077.stdout:7/158: chown da/d13/f19 229611 1 2024-04-26T04:52:54.591 INFO:tasks.workunit.client.0.smithi077.stdout:5/168: symlink d1/d12/d21/d25/d31/l3b 0 2024-04-26T04:52:54.593 INFO:tasks.workunit.client.0.smithi077.stdout:6/183: stat d8/c16 0 2024-04-26T04:52:54.594 INFO:tasks.workunit.client.0.smithi077.stdout:6/184: chown d8/d2a/l2b 0 1 2024-04-26T04:52:54.595 INFO:tasks.workunit.client.0.smithi077.stdout:2/159: creat d9/dc/d1d/d34/f36 x:0 0 0 2024-04-26T04:52:54.596 INFO:tasks.workunit.client.0.smithi077.stdout:4/196: write d4/f8 [1062508,29726] 0 2024-04-26T04:52:54.596 INFO:tasks.workunit.client.1.smithi078.stdout:9/431: fsync dc/df/d4b/d22/d2d/f4d 0 2024-04-26T04:52:54.600 INFO:tasks.workunit.client.1.smithi078.stdout:9/432: dwrite dc/df/d27/f53 [0,4194304] 0 2024-04-26T04:52:54.602 INFO:tasks.workunit.client.0.smithi077.stdout:1/272: mkdir d9/d53 0 2024-04-26T04:52:54.603 INFO:tasks.workunit.client.0.smithi077.stdout:8/164: dread d4/df/f16 [0,4194304] 0 2024-04-26T04:52:54.606 INFO:tasks.workunit.client.0.smithi077.stdout:0/148: getdents d0/d19/d1e/d2f 0 2024-04-26T04:52:54.606 INFO:tasks.workunit.client.0.smithi077.stdout:1/273: chown d9/db/d27/f28 89215 1 2024-04-26T04:52:54.607 INFO:tasks.workunit.client.0.smithi077.stdout:4/197: dwrite d4/f39 [0,4194304] 0 2024-04-26T04:52:54.608 INFO:tasks.workunit.client.0.smithi077.stdout:2/160: dwrite d9/dc/d1d/d34/f36 [0,4194304] 0 2024-04-26T04:52:54.608 INFO:tasks.workunit.client.0.smithi077.stdout:8/165: dread d4/fe [0,4194304] 0 2024-04-26T04:52:54.609 INFO:tasks.workunit.client.1.smithi078.stdout:5/400: write d7/da/d1e/d4d/f6e [998995,25336] 0 2024-04-26T04:52:54.610 INFO:tasks.workunit.client.0.smithi077.stdout:4/198: fdatasync d4/de/d15/d1f/f32 0 2024-04-26T04:52:54.611 INFO:tasks.workunit.client.0.smithi077.stdout:0/149: write d0/f2 [688788,42840] 0 2024-04-26T04:52:54.611 INFO:tasks.workunit.client.0.smithi077.stdout:7/159: creat da/d13/f3d x:0 0 0 2024-04-26T04:52:54.613 INFO:tasks.workunit.client.0.smithi077.stdout:2/161: chown l6 21199 1 2024-04-26T04:52:54.614 INFO:tasks.workunit.client.0.smithi077.stdout:5/169: mkdir d1/d12/d21/d25/d31/d3c 0 2024-04-26T04:52:54.616 INFO:tasks.workunit.client.0.smithi077.stdout:8/166: chown d4/df/d22/d2d/d2b/d33/f39 32 1 2024-04-26T04:52:54.617 INFO:tasks.workunit.client.0.smithi077.stdout:5/170: chown d1/d12/d21/f34 15077 1 2024-04-26T04:52:54.618 INFO:tasks.workunit.client.1.smithi078.stdout:2/450: link d8/d11/d18/c2c d8/d11/d18/d5d/d61/c9d 0 2024-04-26T04:52:54.618 INFO:tasks.workunit.client.1.smithi078.stdout:0/553: truncate d2/d39/d96/fa4 56505 0 2024-04-26T04:52:54.619 INFO:tasks.workunit.client.0.smithi077.stdout:4/199: chown d4/de/d19/l21 121 1 2024-04-26T04:52:54.621 INFO:tasks.workunit.client.0.smithi077.stdout:5/171: dread - d1/d12/d21/d25/f3a zero size 2024-04-26T04:52:54.624 INFO:tasks.workunit.client.0.smithi077.stdout:1/274: dwrite d9/db/dd/f49 [0,4194304] 0 2024-04-26T04:52:54.625 INFO:tasks.workunit.client.1.smithi078.stdout:2/451: dwrite d8/d11/d18/d5d/d61/f62 [0,4194304] 0 2024-04-26T04:52:54.626 INFO:tasks.workunit.client.0.smithi077.stdout:0/150: dwrite d0/f4 [0,4194304] 0 2024-04-26T04:52:54.627 INFO:tasks.workunit.client.1.smithi078.stdout:1/526: rmdir d7/d1e/d9e/d4c/d3d 39 2024-04-26T04:52:54.627 INFO:tasks.workunit.client.0.smithi077.stdout:5/172: write d1/d12/f13 [2085901,25291] 0 2024-04-26T04:52:54.629 INFO:tasks.workunit.client.0.smithi077.stdout:0/151: read d0/f9 [691861,25560] 0 2024-04-26T04:52:54.630 INFO:tasks.workunit.client.1.smithi078.stdout:4/498: link d0/c16 d0/d1d/d46/cbd 0 2024-04-26T04:52:54.632 INFO:tasks.workunit.client.0.smithi077.stdout:4/200: dread d4/f39 [0,4194304] 0 2024-04-26T04:52:54.633 INFO:tasks.workunit.client.1.smithi078.stdout:6/562: creat d2/d1a/fa6 x:0 0 0 2024-04-26T04:52:54.634 INFO:tasks.workunit.client.0.smithi077.stdout:4/201: write d4/de/d19/f1b [1371146,78999] 0 2024-04-26T04:52:54.634 INFO:tasks.workunit.client.1.smithi078.stdout:1/527: dwrite d7/d1e/d9e/db/d49/f6c [0,4194304] 0 2024-04-26T04:52:54.635 INFO:tasks.workunit.client.0.smithi077.stdout:4/202: readlink d4/de/d15/l22 0 2024-04-26T04:52:54.636 INFO:tasks.workunit.client.0.smithi077.stdout:4/203: stat d4/de/d15/d1f/d25 0 2024-04-26T04:52:54.637 INFO:tasks.workunit.client.1.smithi078.stdout:6/563: dread - d2/d1a/d5/d3a/d69/f99 zero size 2024-04-26T04:52:54.638 INFO:tasks.workunit.client.1.smithi078.stdout:3/486: sync 2024-04-26T04:52:54.640 INFO:tasks.workunit.client.1.smithi078.stdout:1/528: fdatasync d7/d1e/d9e/db/d37/d58/f5e 0 2024-04-26T04:52:54.640 INFO:tasks.workunit.client.1.smithi078.stdout:4/499: dread d0/de/f1a [0,4194304] 0 2024-04-26T04:52:54.643 INFO:tasks.workunit.client.1.smithi078.stdout:8/454: write d0/d19/d25/d36/f7f [1103176,30488] 0 2024-04-26T04:52:54.645 INFO:tasks.workunit.client.1.smithi078.stdout:7/473: write d9/da/d11/d13/d1a/f1e [218526,49264] 0 2024-04-26T04:52:54.646 INFO:tasks.workunit.client.1.smithi078.stdout:8/455: write d0/d19/d52/d35/d44/f4d [372010,4241] 0 2024-04-26T04:52:54.647 INFO:tasks.workunit.client.1.smithi078.stdout:7/474: readlink d9/l49 0 2024-04-26T04:52:54.647 INFO:tasks.workunit.client.1.smithi078.stdout:1/529: dread d7/d1e/f31 [0,4194304] 0 2024-04-26T04:52:54.650 INFO:tasks.workunit.client.1.smithi078.stdout:1/530: read - d7/d1e/d9e/db/d37/faa zero size 2024-04-26T04:52:54.650 INFO:tasks.workunit.client.1.smithi078.stdout:5/401: mkdir d7/d18/d32/d84 0 2024-04-26T04:52:54.652 INFO:tasks.workunit.client.1.smithi078.stdout:4/500: dwrite d0/d25/d20/d31/fb9 [0,4194304] 0 2024-04-26T04:52:54.652 INFO:tasks.workunit.client.0.smithi077.stdout:9/130: rename d9/dd/f1a to d9/f2d 0 2024-04-26T04:52:54.654 INFO:tasks.workunit.client.1.smithi078.stdout:0/554: fsync d2/f57 0 2024-04-26T04:52:54.656 INFO:tasks.workunit.client.0.smithi077.stdout:3/158: getdents d0/d7/d15 0 2024-04-26T04:52:54.657 INFO:tasks.workunit.client.0.smithi077.stdout:4/204: dread d4/de/d19/f1b [4194304,4194304] 0 2024-04-26T04:52:54.657 INFO:tasks.workunit.client.1.smithi078.stdout:6/564: creat d2/d1a/d5/d2a/d4d/fa7 x:0 0 0 2024-04-26T04:52:54.658 INFO:tasks.workunit.client.1.smithi078.stdout:6/565: stat d2/de/d27/l3f 0 2024-04-26T04:52:54.658 INFO:tasks.workunit.client.1.smithi078.stdout:6/566: chown d2/de/d27/d58/l89 5 1 2024-04-26T04:52:54.663 INFO:tasks.workunit.client.1.smithi078.stdout:6/567: dwrite d2/de/d27/f34 [0,4194304] 0 2024-04-26T04:52:54.666 INFO:tasks.workunit.client.0.smithi077.stdout:6/185: link l5 d8/d2a/d38/l42 0 2024-04-26T04:52:54.666 INFO:tasks.workunit.client.1.smithi078.stdout:9/433: sync 2024-04-26T04:52:54.667 INFO:tasks.workunit.client.1.smithi078.stdout:1/531: fsync d7/f8a 0 2024-04-26T04:52:54.668 INFO:tasks.workunit.client.1.smithi078.stdout:9/434: dread - dc/df/d4b/d3e/f5d zero size 2024-04-26T04:52:54.669 INFO:tasks.workunit.client.1.smithi078.stdout:5/402: dread - d7/d11/f76 zero size 2024-04-26T04:52:54.671 INFO:tasks.workunit.client.1.smithi078.stdout:4/501: read d0/de/d22/d27/d56/d5a/f7d [238239,72156] 0 2024-04-26T04:52:54.674 INFO:tasks.workunit.client.0.smithi077.stdout:2/162: rename d9/d15/d1a/d2b/f32 to d9/d15/f37 0 2024-04-26T04:52:54.676 INFO:tasks.workunit.client.0.smithi077.stdout:3/159: rmdir d0/d7/d15 39 2024-04-26T04:52:54.677 INFO:tasks.workunit.client.0.smithi077.stdout:1/275: symlink d9/d53/l54 0 2024-04-26T04:52:54.680 INFO:tasks.workunit.client.1.smithi078.stdout:2/452: rename d8/d11/d6e/c89 to d8/d11/d14/d69/d8f/c9e 0 2024-04-26T04:52:54.682 INFO:tasks.workunit.client.0.smithi077.stdout:8/167: dread d4/f5 [0,4194304] 0 2024-04-26T04:52:54.687 INFO:tasks.workunit.client.1.smithi078.stdout:3/487: write d16/d26/f32 [2174869,8436] 0 2024-04-26T04:52:54.689 INFO:tasks.workunit.client.0.smithi077.stdout:8/168: dwrite d4/df/f16 [0,4194304] 0 2024-04-26T04:52:54.690 INFO:tasks.workunit.client.1.smithi078.stdout:5/403: stat d7/d48/d57/d65/d46/f4f 0 2024-04-26T04:52:54.697 INFO:tasks.workunit.client.0.smithi077.stdout:0/152: truncate d0/db/ff 2833799 0 2024-04-26T04:52:54.699 INFO:tasks.workunit.client.0.smithi077.stdout:3/160: creat d0/d9/d32/f35 x:0 0 0 2024-04-26T04:52:54.700 INFO:tasks.workunit.client.0.smithi077.stdout:4/205: dwrite d4/de/d15/d1f/f34 [0,4194304] 0 2024-04-26T04:52:54.701 INFO:tasks.workunit.client.1.smithi078.stdout:8/456: rename d0/d19/d52/d35/d3f to d0/d19/d25/d32/d8a 0 2024-04-26T04:52:54.702 INFO:tasks.workunit.client.0.smithi077.stdout:1/276: creat d9/db/d2b/d3f/f55 x:0 0 0 2024-04-26T04:52:54.702 INFO:tasks.workunit.client.0.smithi077.stdout:4/206: rename d4/de/d15/d1f/d35 to d4/de/d15/d1f/d35/d3e 22 2024-04-26T04:52:54.704 INFO:tasks.workunit.client.1.smithi078.stdout:6/568: mkdir d2/d1a/d5/da8 0 2024-04-26T04:52:54.706 INFO:tasks.workunit.client.0.smithi077.stdout:7/160: rename da/c17 to da/c3e 0 2024-04-26T04:52:54.708 INFO:tasks.workunit.client.1.smithi078.stdout:3/488: rename d16/d26/d2e/f75 to d16/d26/d2e/fa6 0 2024-04-26T04:52:54.709 INFO:tasks.workunit.client.0.smithi077.stdout:8/169: mknod d4/df/d22/d2d/c3a 0 2024-04-26T04:52:54.709 INFO:tasks.workunit.client.1.smithi078.stdout:3/489: dread - d16/d26/d2e/d5f/d49/f8f zero size 2024-04-26T04:52:54.711 INFO:tasks.workunit.client.0.smithi077.stdout:9/131: creat d9/f2e x:0 0 0 2024-04-26T04:52:54.712 INFO:tasks.workunit.client.1.smithi078.stdout:0/555: creat d2/d9/d31/d4a/fb9 x:0 0 0 2024-04-26T04:52:54.715 INFO:tasks.workunit.client.0.smithi077.stdout:3/161: rename d0/d9/d32/f2c to d0/d9/d32/f36 0 2024-04-26T04:52:54.715 INFO:tasks.workunit.client.1.smithi078.stdout:3/490: dwrite d16/d26/d2e/d5f/d49/fa4 [0,4194304] 0 2024-04-26T04:52:54.716 INFO:tasks.workunit.client.0.smithi077.stdout:8/170: dwrite d4/df/f16 [0,4194304] 0 2024-04-26T04:52:54.716 INFO:tasks.workunit.client.1.smithi078.stdout:2/453: creat d8/d3e/d52/f9f x:0 0 0 2024-04-26T04:52:54.717 INFO:tasks.workunit.client.0.smithi077.stdout:3/162: write d0/d7/f33 [577393,20494] 0 2024-04-26T04:52:54.718 INFO:tasks.workunit.client.1.smithi078.stdout:3/491: chown d16/d26/d2b/d31/d71/d8a 2473 1 2024-04-26T04:52:54.718 INFO:tasks.workunit.client.0.smithi077.stdout:8/171: chown d4/f9 157 1 2024-04-26T04:52:54.720 INFO:tasks.workunit.client.0.smithi077.stdout:4/207: rename d4 to d4/d3f 22 2024-04-26T04:52:54.720 INFO:tasks.workunit.client.1.smithi078.stdout:0/556: dwrite d2/d9/d31/d4a/f82 [0,4194304] 0 2024-04-26T04:52:54.721 INFO:tasks.workunit.client.1.smithi078.stdout:7/475: getdents d9/da/d33/d4b/d4c/d5b/d83 0 2024-04-26T04:52:54.723 INFO:tasks.workunit.client.0.smithi077.stdout:1/277: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:54.724 INFO:tasks.workunit.client.0.smithi077.stdout:5/173: rename d1/la to d1/l3d 0 2024-04-26T04:52:54.731 INFO:tasks.workunit.client.0.smithi077.stdout:4/208: dwrite d4/de/d15/d1f/f34 [0,4194304] 0 2024-04-26T04:52:54.732 INFO:tasks.workunit.client.1.smithi078.stdout:5/404: mkdir d7/d48/d57/d65/d70/d72/d85 0 2024-04-26T04:52:54.737 INFO:tasks.workunit.client.0.smithi077.stdout:2/163: creat d9/dc/d1d/f38 x:0 0 0 2024-04-26T04:52:54.738 INFO:tasks.workunit.client.1.smithi078.stdout:8/457: write d0/d19/d25/d32/d8a/f5e [3791430,35397] 0 2024-04-26T04:52:54.742 INFO:tasks.workunit.client.0.smithi077.stdout:3/163: write d0/d9/d32/f36 [209657,95394] 0 2024-04-26T04:52:54.743 INFO:tasks.workunit.client.0.smithi077.stdout:3/164: stat d0/d9/d32/d29 0 2024-04-26T04:52:54.744 INFO:tasks.workunit.client.0.smithi077.stdout:8/172: mknod d4/df/d22/c3b 0 2024-04-26T04:52:54.745 INFO:tasks.workunit.client.1.smithi078.stdout:8/458: dwrite d0/d19/d52/f24 [0,4194304] 0 2024-04-26T04:52:54.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/454: creat d8/fa0 x:0 0 0 2024-04-26T04:52:54.746 INFO:tasks.workunit.client.0.smithi077.stdout:2/164: dwrite d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:52:54.747 INFO:tasks.workunit.client.0.smithi077.stdout:9/132: rename d9/dd/d1b/f2c to d9/dd/d1b/d2b/f2f 0 2024-04-26T04:52:54.748 INFO:tasks.workunit.client.1.smithi078.stdout:2/455: write d8/d11/d14/d69/d8f/f88 [1516916,5023] 0 2024-04-26T04:52:54.751 INFO:tasks.workunit.client.0.smithi077.stdout:7/161: symlink da/d13/d22/d34/l3f 0 2024-04-26T04:52:54.752 INFO:tasks.workunit.client.1.smithi078.stdout:8/459: dread d0/d19/d52/d35/d44/f4f [0,4194304] 0 2024-04-26T04:52:54.752 INFO:tasks.workunit.client.1.smithi078.stdout:0/557: read d2/f70 [5720,31785] 0 2024-04-26T04:52:54.754 INFO:tasks.workunit.client.1.smithi078.stdout:7/476: creat d9/da/d33/d4b/d4c/d5b/d83/f97 x:0 0 0 2024-04-26T04:52:54.755 INFO:tasks.workunit.client.1.smithi078.stdout:7/477: write d9/da/d20/f6f [477939,67358] 0 2024-04-26T04:52:54.755 INFO:tasks.workunit.client.0.smithi077.stdout:9/133: dwrite d9/f26 [0,4194304] 0 2024-04-26T04:52:54.756 INFO:tasks.workunit.client.0.smithi077.stdout:5/174: symlink d1/d12/d21/l3e 0 2024-04-26T04:52:54.758 INFO:tasks.workunit.client.1.smithi078.stdout:9/435: getdents dc/df/d37 0 2024-04-26T04:52:54.758 INFO:tasks.workunit.client.0.smithi077.stdout:6/186: getdents d8/d2a 0 2024-04-26T04:52:54.760 INFO:tasks.workunit.client.1.smithi078.stdout:5/405: symlink d7/d48/d57/d65/d70/d72/l86 0 2024-04-26T04:52:54.760 INFO:tasks.workunit.client.1.smithi078.stdout:5/406: write d7/d11/f76 [544187,125519] 0 2024-04-26T04:52:54.762 INFO:tasks.workunit.client.0.smithi077.stdout:6/187: write f3 [4820654,61451] 0 2024-04-26T04:52:54.763 INFO:tasks.workunit.client.1.smithi078.stdout:1/532: getdents d7/d1e/d9e/dd/d20/d53 0 2024-04-26T04:52:54.764 INFO:tasks.workunit.client.0.smithi077.stdout:4/209: creat d4/de/d15/f40 x:0 0 0 2024-04-26T04:52:54.764 INFO:tasks.workunit.client.0.smithi077.stdout:0/153: link d0/d28/l2b d0/d19/d1e/l38 0 2024-04-26T04:52:54.765 INFO:tasks.workunit.client.1.smithi078.stdout:4/502: getdents d0/d25/d20/d31/d47/db2/d4c/d62/daa 0 2024-04-26T04:52:54.765 INFO:tasks.workunit.client.0.smithi077.stdout:6/188: write d8/d1c/f3e [442263,90768] 0 2024-04-26T04:52:54.772 INFO:tasks.workunit.client.0.smithi077.stdout:3/165: fsync d0/d9/d32/f36 0 2024-04-26T04:52:54.773 INFO:tasks.workunit.client.0.smithi077.stdout:3/166: read - d0/d9/d32/f28 zero size 2024-04-26T04:52:54.773 INFO:tasks.workunit.client.0.smithi077.stdout:2/165: mkdir d9/dc/d1d/d34/d39 0 2024-04-26T04:52:54.778 INFO:tasks.workunit.client.0.smithi077.stdout:1/278: symlink d9/l56 0 2024-04-26T04:52:54.779 INFO:tasks.workunit.client.0.smithi077.stdout:1/279: fdatasync d9/db/f4c 0 2024-04-26T04:52:54.781 INFO:tasks.workunit.client.0.smithi077.stdout:1/280: fsync d9/db/f39 0 2024-04-26T04:52:54.782 INFO:tasks.workunit.client.0.smithi077.stdout:3/167: dwrite d0/d9/f30 [0,4194304] 0 2024-04-26T04:52:54.783 INFO:tasks.workunit.client.1.smithi078.stdout:8/460: rmdir d0/d19/d52/d5c/d68 39 2024-04-26T04:52:54.785 INFO:tasks.workunit.client.0.smithi077.stdout:5/175: write d1/f5 [2655217,61234] 0 2024-04-26T04:52:54.786 INFO:tasks.workunit.client.1.smithi078.stdout:7/478: readlink d9/da/d11/d13/d1a/l67 0 2024-04-26T04:52:54.788 INFO:tasks.workunit.client.0.smithi077.stdout:3/168: dread d0/f1 [0,4194304] 0 2024-04-26T04:52:54.788 INFO:tasks.workunit.client.0.smithi077.stdout:9/134: creat d9/dd/f30 x:0 0 0 2024-04-26T04:52:54.789 INFO:tasks.workunit.client.0.smithi077.stdout:9/135: dread - d9/f2e zero size 2024-04-26T04:52:54.793 INFO:tasks.workunit.client.0.smithi077.stdout:0/154: dread d0/f25 [0,4194304] 0 2024-04-26T04:52:54.794 INFO:tasks.workunit.client.1.smithi078.stdout:4/503: creat d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/fbe x:0 0 0 2024-04-26T04:52:54.794 INFO:tasks.workunit.client.0.smithi077.stdout:0/155: readlink d0/db/d11/l18 0 2024-04-26T04:52:54.798 INFO:tasks.workunit.client.1.smithi078.stdout:6/569: getdents d2/d1a/d5c 0 2024-04-26T04:52:54.801 INFO:tasks.workunit.client.1.smithi078.stdout:3/492: creat d16/d9b/fa7 x:0 0 0 2024-04-26T04:52:54.801 INFO:tasks.workunit.client.0.smithi077.stdout:2/166: write d9/d15/d1a/f1c [320901,94937] 0 2024-04-26T04:52:54.802 INFO:tasks.workunit.client.1.smithi078.stdout:0/558: write d2/f57 [3033447,120944] 0 2024-04-26T04:52:54.802 INFO:tasks.workunit.client.0.smithi077.stdout:2/167: write f5 [2769880,12886] 0 2024-04-26T04:52:54.806 INFO:tasks.workunit.client.1.smithi078.stdout:8/461: dread d0/f10 [4194304,4194304] 0 2024-04-26T04:52:54.809 INFO:tasks.workunit.client.1.smithi078.stdout:7/479: creat d9/da/d11/d13/d1a/d52/d5a/d28/f98 x:0 0 0 2024-04-26T04:52:54.811 INFO:tasks.workunit.client.1.smithi078.stdout:8/462: dwrite d0/d19/d52/d35/d44/f5d [0,4194304] 0 2024-04-26T04:52:54.813 INFO:tasks.workunit.client.1.smithi078.stdout:1/533: symlink d7/d1e/d9e/dd/lad 0 2024-04-26T04:52:54.814 INFO:tasks.workunit.client.0.smithi077.stdout:8/173: dwrite d4/df/f1f [0,4194304] 0 2024-04-26T04:52:54.816 INFO:tasks.workunit.client.0.smithi077.stdout:5/176: symlink d1/d8/l3f 0 2024-04-26T04:52:54.820 INFO:tasks.workunit.client.1.smithi078.stdout:3/493: mkdir d16/d26/d2b/d54/d56/da8 0 2024-04-26T04:52:54.820 INFO:tasks.workunit.client.1.smithi078.stdout:8/463: dwrite d0/d19/d52/d5c/f74 [0,4194304] 0 2024-04-26T04:52:54.820 INFO:tasks.workunit.client.0.smithi077.stdout:7/162: dwrite da/f18 [0,4194304] 0 2024-04-26T04:52:54.823 INFO:tasks.workunit.client.0.smithi077.stdout:0/156: mkdir d0/d28/d39 0 2024-04-26T04:52:54.824 INFO:tasks.workunit.client.1.smithi078.stdout:3/494: dread d16/d26/d2e/d5f/d49/fa4 [0,4194304] 0 2024-04-26T04:52:54.827 INFO:tasks.workunit.client.1.smithi078.stdout:5/407: creat d7/f87 x:0 0 0 2024-04-26T04:52:54.830 INFO:tasks.workunit.client.1.smithi078.stdout:7/480: dread f7 [4194304,4194304] 0 2024-04-26T04:52:54.832 INFO:tasks.workunit.client.0.smithi077.stdout:0/157: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:52:54.833 INFO:tasks.workunit.client.1.smithi078.stdout:4/504: symlink d0/de/d22/d27/lbf 0 2024-04-26T04:52:54.837 INFO:tasks.workunit.client.1.smithi078.stdout:5/408: dwrite d7/da/f62 [0,4194304] 0 2024-04-26T04:52:54.837 INFO:tasks.workunit.client.1.smithi078.stdout:2/456: getdents d8/d11/d14/d69/d8f 0 2024-04-26T04:52:54.838 INFO:tasks.workunit.client.0.smithi077.stdout:8/174: creat d4/d17/f3c x:0 0 0 2024-04-26T04:52:54.839 INFO:tasks.workunit.client.1.smithi078.stdout:0/559: symlink d2/d9/lba 0 2024-04-26T04:52:54.839 INFO:tasks.workunit.client.0.smithi077.stdout:5/177: rename d1/c39 to d1/d12/d28/c40 0 2024-04-26T04:52:54.841 INFO:tasks.workunit.client.1.smithi078.stdout:3/495: creat d16/d26/d2b/fa9 x:0 0 0 2024-04-26T04:52:54.842 INFO:tasks.workunit.client.0.smithi077.stdout:5/178: write d1/d12/f2a [8838,91383] 0 2024-04-26T04:52:54.843 INFO:tasks.workunit.client.1.smithi078.stdout:9/436: sync 2024-04-26T04:52:54.845 INFO:tasks.workunit.client.1.smithi078.stdout:6/570: rename d2/d1a/d5 to d2/de/d27/da9 0 2024-04-26T04:52:54.847 INFO:tasks.workunit.client.1.smithi078.stdout:4/505: rmdir d0/d1d 39 2024-04-26T04:52:54.847 INFO:tasks.workunit.client.0.smithi077.stdout:6/189: creat d8/f43 x:0 0 0 2024-04-26T04:52:54.849 INFO:tasks.workunit.client.0.smithi077.stdout:4/210: symlink d4/de/d15/d1f/l41 0 2024-04-26T04:52:54.850 INFO:tasks.workunit.client.1.smithi078.stdout:7/481: truncate d9/d57/f7a 7237187 0 2024-04-26T04:52:54.852 INFO:tasks.workunit.client.1.smithi078.stdout:6/571: dwrite d2/de/d27/da9/d2a/f94 [0,4194304] 0 2024-04-26T04:52:54.855 INFO:tasks.workunit.client.0.smithi077.stdout:8/175: rmdir d4/df/d22/d2d 39 2024-04-26T04:52:54.858 INFO:tasks.workunit.client.0.smithi077.stdout:3/169: rename d0/d9/f14 to d0/d7/d15/f37 0 2024-04-26T04:52:54.859 INFO:tasks.workunit.client.0.smithi077.stdout:3/170: chown d0/d7/d15 4 1 2024-04-26T04:52:54.861 INFO:tasks.workunit.client.1.smithi078.stdout:7/482: dwrite d9/da/d11/d13/f25 [0,4194304] 0 2024-04-26T04:52:54.861 INFO:tasks.workunit.client.1.smithi078.stdout:9/437: creat dc/d55/f82 x:0 0 0 2024-04-26T04:52:54.863 INFO:tasks.workunit.client.1.smithi078.stdout:9/438: write dc/f60 [344136,79426] 0 2024-04-26T04:52:54.863 INFO:tasks.workunit.client.0.smithi077.stdout:6/190: mknod d8/d1c/c44 0 2024-04-26T04:52:54.863 INFO:tasks.workunit.client.1.smithi078.stdout:8/464: rename d0/d19 to d0/d19/d25/d8b 22 2024-04-26T04:52:54.867 INFO:tasks.workunit.client.1.smithi078.stdout:4/506: write d0/d25/d20/f2a [5021383,127246] 0 2024-04-26T04:52:54.870 INFO:tasks.workunit.client.0.smithi077.stdout:0/158: symlink d0/d2c/l3a 0 2024-04-26T04:52:54.873 INFO:tasks.workunit.client.0.smithi077.stdout:1/281: getdents d9/db 0 2024-04-26T04:52:54.874 INFO:tasks.workunit.client.0.smithi077.stdout:1/282: readlink d9/db/dd/l2e 0 2024-04-26T04:52:54.875 INFO:tasks.workunit.client.0.smithi077.stdout:8/176: readlink d4/lc 0 2024-04-26T04:52:54.876 INFO:tasks.workunit.client.0.smithi077.stdout:9/136: dread d9/f17 [0,4194304] 0 2024-04-26T04:52:54.879 INFO:tasks.workunit.client.0.smithi077.stdout:4/211: dwrite d4/de/d19/f2d [0,4194304] 0 2024-04-26T04:52:54.879 INFO:tasks.workunit.client.1.smithi078.stdout:0/560: symlink d2/d9/d78/lbb 0 2024-04-26T04:52:54.882 INFO:tasks.workunit.client.0.smithi077.stdout:5/179: rename d1/d12/f13 to d1/d12/d21/f41 0 2024-04-26T04:52:54.885 INFO:tasks.workunit.client.0.smithi077.stdout:2/168: dwrite f7 [0,4194304] 0 2024-04-26T04:52:54.885 INFO:tasks.workunit.client.0.smithi077.stdout:1/283: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:54.886 INFO:tasks.workunit.client.0.smithi077.stdout:7/163: creat da/d15/d1a/d39/f40 x:0 0 0 2024-04-26T04:52:54.886 INFO:tasks.workunit.client.0.smithi077.stdout:5/180: write d1/d8/fc [1727082,8527] 0 2024-04-26T04:52:54.889 INFO:tasks.workunit.client.1.smithi078.stdout:6/572: dread d2/de/d27/da9/f8 [0,4194304] 0 2024-04-26T04:52:54.892 INFO:tasks.workunit.client.1.smithi078.stdout:3/496: mknod d16/d26/caa 0 2024-04-26T04:52:54.893 INFO:tasks.workunit.client.0.smithi077.stdout:6/191: dread d8/f22 [0,4194304] 0 2024-04-26T04:52:54.895 INFO:tasks.workunit.client.0.smithi077.stdout:5/181: dwrite d1/d8/fc [0,4194304] 0 2024-04-26T04:52:54.899 INFO:tasks.workunit.client.1.smithi078.stdout:9/439: fdatasync dc/f3d 0 2024-04-26T04:52:54.904 INFO:tasks.workunit.client.1.smithi078.stdout:7/483: dwrite d9/da/d11/f45 [4194304,4194304] 0 2024-04-26T04:52:54.904 INFO:tasks.workunit.client.0.smithi077.stdout:4/212: readlink d4/lb 0 2024-04-26T04:52:54.906 INFO:tasks.workunit.client.0.smithi077.stdout:4/213: truncate d4/de/d15/f40 606040 0 2024-04-26T04:52:54.907 INFO:tasks.workunit.client.1.smithi078.stdout:8/465: creat d0/d19/d25/d36/d58/d5b/d61/f8c x:0 0 0 2024-04-26T04:52:54.912 INFO:tasks.workunit.client.1.smithi078.stdout:8/466: dwrite d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:54.913 INFO:tasks.workunit.client.1.smithi078.stdout:2/457: link d8/d11/d14/d5b/c73 d8/d11/d18/d5d/d8b/ca1 0 2024-04-26T04:52:54.914 INFO:tasks.workunit.client.1.smithi078.stdout:8/467: chown d0/d19/d25/d36/d58/d5b 1 1 2024-04-26T04:52:54.914 INFO:tasks.workunit.client.1.smithi078.stdout:2/458: stat l5 0 2024-04-26T04:52:54.915 INFO:tasks.workunit.client.0.smithi077.stdout:1/284: rmdir d9/db/dd 39 2024-04-26T04:52:54.917 INFO:tasks.workunit.client.1.smithi078.stdout:6/573: fsync d2/de/d27/da9/d3a/d42/f5a 0 2024-04-26T04:52:54.922 INFO:tasks.workunit.client.1.smithi078.stdout:9/440: write dc/df/d4b/f1c [2810288,72661] 0 2024-04-26T04:52:54.925 INFO:tasks.workunit.client.0.smithi077.stdout:6/192: mknod d8/d2a/d38/c45 0 2024-04-26T04:52:54.928 INFO:tasks.workunit.client.1.smithi078.stdout:1/534: rename d7/d1e/d9e/d4c/d3d/l2b to d7/d1e/d9e/db/d37/d58/d5c/d68/lae 0 2024-04-26T04:52:54.930 INFO:tasks.workunit.client.1.smithi078.stdout:7/484: mknod d9/da/d11/d13/d1a/d52/d5a/d41/c99 0 2024-04-26T04:52:54.931 INFO:tasks.workunit.client.0.smithi077.stdout:0/159: truncate d0/d2c/f2e 44098 0 2024-04-26T04:52:54.932 INFO:tasks.workunit.client.1.smithi078.stdout:4/507: write d0/d25/d20/d31/d47/db2/f5b [438701,41675] 0 2024-04-26T04:52:54.934 INFO:tasks.workunit.client.0.smithi077.stdout:8/177: creat d4/df/d22/d2d/f3d x:0 0 0 2024-04-26T04:52:54.935 INFO:tasks.workunit.client.0.smithi077.stdout:9/137: symlink d9/l31 0 2024-04-26T04:52:54.940 INFO:tasks.workunit.client.1.smithi078.stdout:6/574: mkdir d2/de/d27/da9/d3a/d69/daa 0 2024-04-26T04:52:54.941 INFO:tasks.workunit.client.1.smithi078.stdout:6/575: write d2/de/d27/da9/d2a/d4d/f61 [4045927,79489] 0 2024-04-26T04:52:54.943 INFO:tasks.workunit.client.0.smithi077.stdout:5/182: mkdir d1/d42 0 2024-04-26T04:52:54.945 INFO:tasks.workunit.client.1.smithi078.stdout:0/561: rename d2/l8b to d2/d9/d60/d6a/da5/lbc 0 2024-04-26T04:52:54.946 INFO:tasks.workunit.client.1.smithi078.stdout:1/535: creat d7/d1e/d9e/db/d49/d93/faf x:0 0 0 2024-04-26T04:52:54.948 INFO:tasks.workunit.client.1.smithi078.stdout:6/576: dwrite d2/de/d27/da9/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:52:54.949 INFO:tasks.workunit.client.1.smithi078.stdout:6/577: write d2/de/d27/da9/d11/f4f [641487,20467] 0 2024-04-26T04:52:54.951 INFO:tasks.workunit.client.0.smithi077.stdout:3/171: rename d0/d7/f11 to d0/f38 0 2024-04-26T04:52:54.952 INFO:tasks.workunit.client.1.smithi078.stdout:5/409: getdents d7/d11 0 2024-04-26T04:52:54.953 INFO:tasks.workunit.client.0.smithi077.stdout:3/172: read - d0/d9/d32/f35 zero size 2024-04-26T04:52:54.953 INFO:tasks.workunit.client.0.smithi077.stdout:3/173: truncate d0/f1c 851287 0 2024-04-26T04:52:54.963 INFO:tasks.workunit.client.0.smithi077.stdout:5/183: mknod d1/d30/c43 0 2024-04-26T04:52:54.963 INFO:tasks.workunit.client.0.smithi077.stdout:1/285: dread d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:52:54.964 INFO:tasks.workunit.client.0.smithi077.stdout:1/286: dread - d9/db/d2b/d3f/f41 zero size 2024-04-26T04:52:54.965 INFO:tasks.workunit.client.1.smithi078.stdout:3/497: sync 2024-04-26T04:52:54.966 INFO:tasks.workunit.client.0.smithi077.stdout:1/287: write d9/db/d2b/d3f/f55 [869297,5242] 0 2024-04-26T04:52:54.966 INFO:tasks.workunit.client.1.smithi078.stdout:3/498: chown d16/d26/d2b/d54/d56/fa2 1548317 1 2024-04-26T04:52:54.970 INFO:tasks.workunit.client.0.smithi077.stdout:8/178: write d4/df/d22/d2d/f1b [5013287,64432] 0 2024-04-26T04:52:54.973 INFO:tasks.workunit.client.0.smithi077.stdout:7/164: rename da/d13/f23 to da/d15/d1a/d39/f41 0 2024-04-26T04:52:54.974 INFO:tasks.workunit.client.0.smithi077.stdout:2/169: link d9/f12 d9/dc/f3a 0 2024-04-26T04:52:54.975 INFO:tasks.workunit.client.1.smithi078.stdout:2/459: truncate d8/d11/d18/d5d/d61/f62 2933589 0 2024-04-26T04:52:54.976 INFO:tasks.workunit.client.0.smithi077.stdout:9/138: dwrite d9/dd/d1b/f23 [4194304,4194304] 0 2024-04-26T04:52:54.976 INFO:tasks.workunit.client.1.smithi078.stdout:0/562: symlink d2/d39/lbd 0 2024-04-26T04:52:54.977 INFO:tasks.workunit.client.1.smithi078.stdout:0/563: write d2/d39/f8c [450104,26371] 0 2024-04-26T04:52:54.977 INFO:tasks.workunit.client.0.smithi077.stdout:6/193: write d8/d2a/f40 [1007784,122286] 0 2024-04-26T04:52:54.979 INFO:tasks.workunit.client.1.smithi078.stdout:1/536: creat d7/d1e/d9e/dd/d20/fb0 x:0 0 0 2024-04-26T04:52:54.980 INFO:tasks.workunit.client.1.smithi078.stdout:1/537: write d7/d1e/d9e/db/d37/f88 [96498,58049] 0 2024-04-26T04:52:54.982 INFO:tasks.workunit.client.1.smithi078.stdout:9/441: truncate dc/df/d4b/f79 1952681 0 2024-04-26T04:52:54.983 INFO:tasks.workunit.client.1.smithi078.stdout:9/442: stat dc/df/d4b/d22/d2a/d57 0 2024-04-26T04:52:54.984 INFO:tasks.workunit.client.0.smithi077.stdout:7/165: dwrite da/d13/d22/f25 [0,4194304] 0 2024-04-26T04:52:54.984 INFO:tasks.workunit.client.0.smithi077.stdout:0/160: write d0/db/d11/f14 [954536,82782] 0 2024-04-26T04:52:54.985 INFO:tasks.workunit.client.0.smithi077.stdout:0/161: readlink d0/l10 0 2024-04-26T04:52:54.987 INFO:tasks.workunit.client.1.smithi078.stdout:4/508: mknod d0/de/d22/cc0 0 2024-04-26T04:52:54.987 INFO:tasks.workunit.client.1.smithi078.stdout:4/509: fsync d0/de/f1a 0 2024-04-26T04:52:54.988 INFO:tasks.workunit.client.1.smithi078.stdout:6/578: mkdir d2/de/d27/da9/da/dab 0 2024-04-26T04:52:54.989 INFO:tasks.workunit.client.1.smithi078.stdout:6/579: fdatasync d2/d1a/d38/f68 0 2024-04-26T04:52:54.990 INFO:tasks.workunit.client.1.smithi078.stdout:6/580: write d2/de/d27/da9/d11/f1c [2502707,55984] 0 2024-04-26T04:52:54.995 INFO:tasks.workunit.client.1.smithi078.stdout:3/499: unlink d16/d26/d2b/d31/f34 0 2024-04-26T04:52:54.995 INFO:tasks.workunit.client.1.smithi078.stdout:3/500: stat d16 0 2024-04-26T04:52:54.998 INFO:tasks.workunit.client.0.smithi077.stdout:8/179: symlink d4/df/d18/l3e 0 2024-04-26T04:52:54.999 INFO:tasks.workunit.client.1.smithi078.stdout:2/460: mkdir d8/d11/d14/d69/da2 0 2024-04-26T04:52:55.000 INFO:tasks.workunit.client.1.smithi078.stdout:7/485: truncate d9/da/d11/d13/d1a/d2d/f3f 2297001 0 2024-04-26T04:52:55.000 INFO:tasks.workunit.client.1.smithi078.stdout:2/461: chown d8/d11/f48 16046 1 2024-04-26T04:52:55.001 INFO:tasks.workunit.client.1.smithi078.stdout:2/462: fsync d8/d11/d14/d69/d8f/f6c 0 2024-04-26T04:52:55.001 INFO:tasks.workunit.client.1.smithi078.stdout:0/564: dread - d2/d9/f75 zero size 2024-04-26T04:52:55.003 INFO:tasks.workunit.client.0.smithi077.stdout:2/170: unlink d9/dc/f2e 0 2024-04-26T04:52:55.003 INFO:tasks.workunit.client.0.smithi077.stdout:2/171: truncate d9/f17 4018740 0 2024-04-26T04:52:55.004 INFO:tasks.workunit.client.0.smithi077.stdout:2/172: readlink l6 0 2024-04-26T04:52:55.007 INFO:tasks.workunit.client.1.smithi078.stdout:4/510: unlink d0/de/d22/d27/f96 0 2024-04-26T04:52:55.008 INFO:tasks.workunit.client.1.smithi078.stdout:4/511: write d0/de/f91 [968923,86358] 0 2024-04-26T04:52:55.008 INFO:tasks.workunit.client.0.smithi077.stdout:9/139: mkdir d9/dd/d1b/d2b/d32 0 2024-04-26T04:52:55.012 INFO:tasks.workunit.client.1.smithi078.stdout:6/581: creat d2/de/fac x:0 0 0 2024-04-26T04:52:55.013 INFO:tasks.workunit.client.0.smithi077.stdout:6/194: fsync d8/f39 0 2024-04-26T04:52:55.015 INFO:tasks.workunit.client.1.smithi078.stdout:5/410: creat d7/d48/d57/d65/d70/d72/f88 x:0 0 0 2024-04-26T04:52:55.018 INFO:tasks.workunit.client.1.smithi078.stdout:3/501: read d16/f25 [3745793,39713] 0 2024-04-26T04:52:55.019 INFO:tasks.workunit.client.0.smithi077.stdout:7/166: unlink da/d15/f24 0 2024-04-26T04:52:55.021 INFO:tasks.workunit.client.1.smithi078.stdout:8/468: rename d0/d19/d52/d3c/c45 to d0/d19/d52/c8d 0 2024-04-26T04:52:55.022 INFO:tasks.workunit.client.1.smithi078.stdout:8/469: chown d0/d19/c41 549506732 1 2024-04-26T04:52:55.022 INFO:tasks.workunit.client.0.smithi077.stdout:3/174: truncate d0/d9/f30 3434798 0 2024-04-26T04:52:55.023 INFO:tasks.workunit.client.1.smithi078.stdout:8/470: write d0/d19/d25/d36/d58/d5b/d61/f8c [450793,21263] 0 2024-04-26T04:52:55.024 INFO:tasks.workunit.client.0.smithi077.stdout:3/175: truncate d0/d7/f34 221371 0 2024-04-26T04:52:55.024 INFO:tasks.workunit.client.0.smithi077.stdout:5/184: mknod d1/d30/d33/c44 0 2024-04-26T04:52:55.024 INFO:tasks.workunit.client.0.smithi077.stdout:5/185: fsync d1/d8/f10 0 2024-04-26T04:52:55.026 INFO:tasks.workunit.client.0.smithi077.stdout:5/186: chown d1/d12/d21/d25/d31/l3b 339 1 2024-04-26T04:52:55.026 INFO:tasks.workunit.client.1.smithi078.stdout:2/463: mkdir d8/d11/d18/da3 0 2024-04-26T04:52:55.026 INFO:tasks.workunit.client.1.smithi078.stdout:8/471: dread d0/d19/d25/d36/d58/d5b/d61/f8c [0,4194304] 0 2024-04-26T04:52:55.027 INFO:tasks.workunit.client.1.smithi078.stdout:2/464: chown d8/d11/d18/c2c 112828 1 2024-04-26T04:52:55.027 INFO:tasks.workunit.client.0.smithi077.stdout:5/187: chown d1/f5 117777563 1 2024-04-26T04:52:55.027 INFO:tasks.workunit.client.0.smithi077.stdout:1/288: mknod d9/db/dd/c57 0 2024-04-26T04:52:55.031 INFO:tasks.workunit.client.1.smithi078.stdout:8/472: dread d0/d19/d52/d35/d44/f5d [0,4194304] 0 2024-04-26T04:52:55.032 INFO:tasks.workunit.client.1.smithi078.stdout:2/465: dwrite d8/d3e/f6a [0,4194304] 0 2024-04-26T04:52:55.032 INFO:tasks.workunit.client.0.smithi077.stdout:4/214: rename d4/c7 to d4/de/d15/d1f/d35/c42 0 2024-04-26T04:52:55.033 INFO:tasks.workunit.client.1.smithi078.stdout:2/466: fsync d8/d11/d14/f67 0 2024-04-26T04:52:55.033 INFO:tasks.workunit.client.0.smithi077.stdout:5/188: dwrite d1/d12/d21/f34 [0,4194304] 0 2024-04-26T04:52:55.036 INFO:tasks.workunit.client.1.smithi078.stdout:8/473: dwrite d0/d19/d25/f72 [0,4194304] 0 2024-04-26T04:52:55.037 INFO:tasks.workunit.client.1.smithi078.stdout:8/474: chown d0/d19/d25/d36/f37 1 1 2024-04-26T04:52:55.043 INFO:tasks.workunit.client.1.smithi078.stdout:6/582: symlink d2/de/lad 0 2024-04-26T04:52:55.044 INFO:tasks.workunit.client.1.smithi078.stdout:6/583: dread - d2/de/d27/da9/d2a/f56 zero size 2024-04-26T04:52:55.045 INFO:tasks.workunit.client.0.smithi077.stdout:2/173: dread d9/dc/d1d/d1e/f13 [0,4194304] 0 2024-04-26T04:52:55.046 INFO:tasks.workunit.client.1.smithi078.stdout:6/584: write d2/de/d27/da9/d2a/d4d/f61 [3909130,49169] 0 2024-04-26T04:52:55.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/411: creat d7/d48/d57/f89 x:0 0 0 2024-04-26T04:52:55.050 INFO:tasks.workunit.client.0.smithi077.stdout:6/195: symlink d8/d1c/l46 0 2024-04-26T04:52:55.051 INFO:tasks.workunit.client.1.smithi078.stdout:3/502: mknod d16/d26/d2b/d31/d71/d8a/cab 0 2024-04-26T04:52:55.052 INFO:tasks.workunit.client.1.smithi078.stdout:3/503: chown d16/d26/d2b/l6a 19262 1 2024-04-26T04:52:55.052 INFO:tasks.workunit.client.1.smithi078.stdout:3/504: chown d16/d9b/fa7 677716989 1 2024-04-26T04:52:55.053 INFO:tasks.workunit.client.1.smithi078.stdout:3/505: write d16/d26/f2a [4242973,115039] 0 2024-04-26T04:52:55.055 INFO:tasks.workunit.client.0.smithi077.stdout:2/174: dwrite d9/dc/d1d/d1e/f21 [0,4194304] 0 2024-04-26T04:52:55.055 INFO:tasks.workunit.client.1.smithi078.stdout:5/412: read d7/da/f10 [8356020,97506] 0 2024-04-26T04:52:55.057 INFO:tasks.workunit.client.1.smithi078.stdout:1/538: rename d7/d1e/f2f to d7/d1e/d9e/db/d49/fb1 0 2024-04-26T04:52:55.058 INFO:tasks.workunit.client.0.smithi077.stdout:7/167: mkdir da/d15/d42 0 2024-04-26T04:52:55.059 INFO:tasks.workunit.client.0.smithi077.stdout:8/180: write d4/df/d22/f34 [318158,51618] 0 2024-04-26T04:52:55.059 INFO:tasks.workunit.client.0.smithi077.stdout:7/168: chown da/d15/d1a/l1e 454852496 1 2024-04-26T04:52:55.061 INFO:tasks.workunit.client.0.smithi077.stdout:8/181: chown d4/df/d22/d2d/f1c 6185 1 2024-04-26T04:52:55.061 INFO:tasks.workunit.client.0.smithi077.stdout:0/162: fdatasync d0/d2c/f2e 0 2024-04-26T04:52:55.068 INFO:tasks.workunit.client.0.smithi077.stdout:1/289: chown d9/db/d27 3114548 1 2024-04-26T04:52:55.073 INFO:tasks.workunit.client.1.smithi078.stdout:2/467: mknod d8/d11/d14/d69/d8f/ca4 0 2024-04-26T04:52:55.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/140: link d9/f22 d9/dd/d28/f33 0 2024-04-26T04:52:55.077 INFO:tasks.workunit.client.1.smithi078.stdout:4/512: creat d0/d25/d20/d31/d47/db2/d4c/dba/fc1 x:0 0 0 2024-04-26T04:52:55.080 INFO:tasks.workunit.client.1.smithi078.stdout:8/475: dread d0/d19/d52/d35/d44/f4f [0,4194304] 0 2024-04-26T04:52:55.081 INFO:tasks.workunit.client.0.smithi077.stdout:6/196: rename d8/d2a/l2b to d8/d2a/l47 0 2024-04-26T04:52:55.083 INFO:tasks.workunit.client.1.smithi078.stdout:6/585: creat d2/de/d27/da9/da/d35/fae x:0 0 0 2024-04-26T04:52:55.084 INFO:tasks.workunit.client.1.smithi078.stdout:6/586: write d2/d1a/f43 [1265417,63789] 0 2024-04-26T04:52:55.086 INFO:tasks.workunit.client.0.smithi077.stdout:2/175: dread d9/fe [0,4194304] 0 2024-04-26T04:52:55.087 INFO:tasks.workunit.client.0.smithi077.stdout:2/176: chown d9/dc/d1d/c29 9642718 1 2024-04-26T04:52:55.090 INFO:tasks.workunit.client.0.smithi077.stdout:8/182: rename d4/df/d22/d2d/c3a to d4/d17/c3f 0 2024-04-26T04:52:55.090 INFO:tasks.workunit.client.0.smithi077.stdout:9/141: rename d9 to d9/dd/d1b/d2b/d32/d34 22 2024-04-26T04:52:55.091 INFO:tasks.workunit.client.0.smithi077.stdout:8/183: stat d4/fd 0 2024-04-26T04:52:55.092 INFO:tasks.workunit.client.0.smithi077.stdout:9/142: write d9/f26 [297824,110066] 0 2024-04-26T04:52:55.093 INFO:tasks.workunit.client.1.smithi078.stdout:5/413: mkdir d7/da/d8a 0 2024-04-26T04:52:55.095 INFO:tasks.workunit.client.0.smithi077.stdout:3/176: symlink d0/l39 0 2024-04-26T04:52:55.101 INFO:tasks.workunit.client.1.smithi078.stdout:0/565: creat d2/d9/d31/d4a/fbe x:0 0 0 2024-04-26T04:52:55.102 INFO:tasks.workunit.client.1.smithi078.stdout:2/468: symlink d8/d11/d6e/la5 0 2024-04-26T04:52:55.103 INFO:tasks.workunit.client.1.smithi078.stdout:2/469: write d8/d11/d14/f67 [4978143,88088] 0 2024-04-26T04:52:55.104 INFO:tasks.workunit.client.1.smithi078.stdout:4/513: fsync d0/d25/d20/f15 0 2024-04-26T04:52:55.105 INFO:tasks.workunit.client.1.smithi078.stdout:4/514: fsync d0/d25/d20/f2b 0 2024-04-26T04:52:55.106 INFO:tasks.workunit.client.0.smithi077.stdout:2/177: creat d9/dc/d1d/d1e/f3b x:0 0 0 2024-04-26T04:52:55.107 INFO:tasks.workunit.client.1.smithi078.stdout:4/515: dread - d0/de/d22/d27/d56/d5a/fb6 zero size 2024-04-26T04:52:55.107 INFO:tasks.workunit.client.0.smithi077.stdout:6/197: dread d8/f27 [0,4194304] 0 2024-04-26T04:52:55.107 INFO:tasks.workunit.client.1.smithi078.stdout:8/476: mknod d0/d19/d25/d32/c8e 0 2024-04-26T04:52:55.107 INFO:tasks.workunit.client.0.smithi077.stdout:2/178: chown d9/f2c 21730 1 2024-04-26T04:52:55.113 INFO:tasks.workunit.client.0.smithi077.stdout:8/184: stat d4/d17/l23 0 2024-04-26T04:52:55.114 INFO:tasks.workunit.client.0.smithi077.stdout:5/189: dwrite d1/d12/f19 [0,4194304] 0 2024-04-26T04:52:55.114 INFO:tasks.workunit.client.1.smithi078.stdout:6/587: symlink d2/de/d27/d4c/d5f/laf 0 2024-04-26T04:52:55.115 INFO:tasks.workunit.client.1.smithi078.stdout:3/506: mknod d16/d26/d2e/d5f/cac 0 2024-04-26T04:52:55.115 INFO:tasks.workunit.client.1.smithi078.stdout:3/507: fsync d16/d3c/f5b 0 2024-04-26T04:52:55.116 INFO:tasks.workunit.client.0.smithi077.stdout:0/163: creat d0/d19/d1e/d2f/f3b x:0 0 0 2024-04-26T04:52:55.116 INFO:tasks.workunit.client.1.smithi078.stdout:3/508: chown d16/d26/d2e/d74/f97 119790 1 2024-04-26T04:52:55.117 INFO:tasks.workunit.client.0.smithi077.stdout:0/164: fsync d0/f1 0 2024-04-26T04:52:55.118 INFO:tasks.workunit.client.0.smithi077.stdout:5/190: write d1/d12/d28/f1c [1878130,72925] 0 2024-04-26T04:52:55.118 INFO:tasks.workunit.client.1.smithi078.stdout:6/588: dread d2/de/d27/d4c/d5f/f9f [0,4194304] 0 2024-04-26T04:52:55.118 INFO:tasks.workunit.client.1.smithi078.stdout:0/566: mknod d2/d9/d18/db7/cbf 0 2024-04-26T04:52:55.119 INFO:tasks.workunit.client.1.smithi078.stdout:6/589: write d2/de/d27/da9/d3a/d69/f99 [108345,66465] 0 2024-04-26T04:52:55.120 INFO:tasks.workunit.client.1.smithi078.stdout:2/470: mknod d8/d11/d18/ca6 0 2024-04-26T04:52:55.120 INFO:tasks.workunit.client.0.smithi077.stdout:0/165: write d0/d19/d1e/f20 [188084,31424] 0 2024-04-26T04:52:55.120 INFO:tasks.workunit.client.1.smithi078.stdout:6/590: fdatasync d2/de/d27/da9/d11/f1c 0 2024-04-26T04:52:55.121 INFO:tasks.workunit.client.0.smithi077.stdout:1/290: unlink d9/db/d2b/c45 0 2024-04-26T04:52:55.123 INFO:tasks.workunit.client.1.smithi078.stdout:4/516: mknod d0/d25/d20/d31/d47/db2/d44/d69/cc2 0 2024-04-26T04:52:55.124 INFO:tasks.workunit.client.1.smithi078.stdout:4/517: chown d0/d25/d20/d31/d47/db2/fa3 25750557 1 2024-04-26T04:52:55.125 INFO:tasks.workunit.client.1.smithi078.stdout:4/518: fsync d0/d25/d20/d31/d47/db2/f4d 0 2024-04-26T04:52:55.126 INFO:tasks.workunit.client.1.smithi078.stdout:0/567: dread d2/d9/d60/d6a/f89 [0,4194304] 0 2024-04-26T04:52:55.126 INFO:tasks.workunit.client.1.smithi078.stdout:4/519: read d0/d25/d20/f4f [349457,125426] 0 2024-04-26T04:52:55.127 INFO:tasks.workunit.client.1.smithi078.stdout:4/520: stat d0/d25/d20/l8c 0 2024-04-26T04:52:55.128 INFO:tasks.workunit.client.1.smithi078.stdout:9/443: rename dc/df/d4b/c6a to dc/c83 0 2024-04-26T04:52:55.129 INFO:tasks.workunit.client.0.smithi077.stdout:2/179: symlink d9/dc/d1d/d1e/l3c 0 2024-04-26T04:52:55.130 INFO:tasks.workunit.client.0.smithi077.stdout:0/166: dwrite d0/db/f16 [0,4194304] 0 2024-04-26T04:52:55.131 INFO:tasks.workunit.client.1.smithi078.stdout:3/509: mkdir d16/d26/d2e/d5f/d49/dad 0 2024-04-26T04:52:55.131 INFO:tasks.workunit.client.0.smithi077.stdout:6/198: dread d8/ff [0,4194304] 0 2024-04-26T04:52:55.131 INFO:tasks.workunit.client.0.smithi077.stdout:0/167: chown d0/f2 5798380 1 2024-04-26T04:52:55.132 INFO:tasks.workunit.client.1.smithi078.stdout:0/568: dwrite d2/d9/d31/d4a/f9b [0,4194304] 0 2024-04-26T04:52:55.140 INFO:tasks.workunit.client.0.smithi077.stdout:6/199: dwrite d8/f37 [0,4194304] 0 2024-04-26T04:52:55.142 INFO:tasks.workunit.client.0.smithi077.stdout:9/143: mkdir d9/d35 0 2024-04-26T04:52:55.148 INFO:tasks.workunit.client.0.smithi077.stdout:3/177: mkdir d0/d7/d1f/d3a 0 2024-04-26T04:52:55.150 INFO:tasks.workunit.client.0.smithi077.stdout:5/191: mkdir d1/d12/d45 0 2024-04-26T04:52:55.152 INFO:tasks.workunit.client.1.smithi078.stdout:1/539: sync 2024-04-26T04:52:55.153 INFO:tasks.workunit.client.1.smithi078.stdout:8/477: dwrite d0/d19/d25/d36/d58/f70 [0,4194304] 0 2024-04-26T04:52:55.153 INFO:tasks.workunit.client.0.smithi077.stdout:1/291: rename d9/db/dd/lf to d9/d53/l58 0 2024-04-26T04:52:55.157 INFO:tasks.workunit.client.1.smithi078.stdout:4/521: mkdir d0/d25/d20/d31/d47/db2/dc3 0 2024-04-26T04:52:55.159 INFO:tasks.workunit.client.1.smithi078.stdout:4/522: stat d0/d25/d20/d31/d47/db2/dc3 0 2024-04-26T04:52:55.159 INFO:tasks.workunit.client.1.smithi078.stdout:8/478: dwrite d0/d19/d25/d36/f7f [0,4194304] 0 2024-04-26T04:52:55.160 INFO:tasks.workunit.client.0.smithi077.stdout:7/169: rename da/c2d to da/c43 0 2024-04-26T04:52:55.161 INFO:tasks.workunit.client.0.smithi077.stdout:1/292: dwrite f7 [0,4194304] 0 2024-04-26T04:52:55.161 INFO:tasks.workunit.client.1.smithi078.stdout:7/486: rename d9/da/l62 to d9/d57/l9a 0 2024-04-26T04:52:55.161 INFO:tasks.workunit.client.1.smithi078.stdout:8/479: write d0/d19/d52/d35/d44/d6b/d6c/f86 [931331,97428] 0 2024-04-26T04:52:55.164 INFO:tasks.workunit.client.1.smithi078.stdout:5/414: sync 2024-04-26T04:52:55.165 INFO:tasks.workunit.client.1.smithi078.stdout:3/510: symlink d16/d26/d2b/lae 0 2024-04-26T04:52:55.167 INFO:tasks.workunit.client.1.smithi078.stdout:0/569: mknod d2/d9/d18/db7/cc0 0 2024-04-26T04:52:55.168 INFO:tasks.workunit.client.0.smithi077.stdout:1/293: read d9/db/d2b/d3f/f55 [803679,22897] 0 2024-04-26T04:52:55.169 INFO:tasks.workunit.client.0.smithi077.stdout:0/168: truncate d0/d19/f2d 606091 0 2024-04-26T04:52:55.171 INFO:tasks.workunit.client.0.smithi077.stdout:1/294: write f7 [1294148,16953] 0 2024-04-26T04:52:55.178 INFO:tasks.workunit.client.0.smithi077.stdout:0/169: dwrite d0/db/d11/f14 [0,4194304] 0 2024-04-26T04:52:55.178 INFO:tasks.workunit.client.0.smithi077.stdout:7/170: creat da/d15/f44 x:0 0 0 2024-04-26T04:52:55.179 INFO:tasks.workunit.client.1.smithi078.stdout:7/487: creat d9/da/d20/f9b x:0 0 0 2024-04-26T04:52:55.180 INFO:tasks.workunit.client.0.smithi077.stdout:0/170: write d0/f9 [1191486,35766] 0 2024-04-26T04:52:55.183 INFO:tasks.workunit.client.1.smithi078.stdout:3/511: symlink d16/d26/d2b/d66/laf 0 2024-04-26T04:52:55.186 INFO:tasks.workunit.client.0.smithi077.stdout:6/200: unlink d8/l19 0 2024-04-26T04:52:55.187 INFO:tasks.workunit.client.1.smithi078.stdout:0/570: truncate d2/d9/d78/d84/f88 103385 0 2024-04-26T04:52:55.188 INFO:tasks.workunit.client.1.smithi078.stdout:2/471: rmdir d8/d11/d18/da3 0 2024-04-26T04:52:55.189 INFO:tasks.workunit.client.0.smithi077.stdout:1/295: fdatasync d9/f24 0 2024-04-26T04:52:55.191 INFO:tasks.workunit.client.1.smithi078.stdout:3/512: dwrite d16/d26/d2b/d31/f65 [0,4194304] 0 2024-04-26T04:52:55.191 INFO:tasks.workunit.client.0.smithi077.stdout:6/201: dread d8/d1c/f25 [0,4194304] 0 2024-04-26T04:52:55.192 INFO:tasks.workunit.client.0.smithi077.stdout:9/144: mkdir d9/dd/d1b/d2b/d32/d36 0 2024-04-26T04:52:55.193 INFO:tasks.workunit.client.0.smithi077.stdout:9/145: readlink d9/dd/le 0 2024-04-26T04:52:55.193 INFO:tasks.workunit.client.0.smithi077.stdout:6/202: chown d8/d2a/l30 449528908 1 2024-04-26T04:52:55.193 INFO:tasks.workunit.client.0.smithi077.stdout:9/146: chown d9/f22 3688583 1 2024-04-26T04:52:55.200 INFO:tasks.workunit.client.0.smithi077.stdout:5/192: symlink d1/d12/d28/d38/l46 0 2024-04-26T04:52:55.203 INFO:tasks.workunit.client.0.smithi077.stdout:9/147: dwrite d9/f1c [0,4194304] 0 2024-04-26T04:52:55.204 INFO:tasks.workunit.client.0.smithi077.stdout:7/171: unlink da/d15/d1a/d39/f40 0 2024-04-26T04:52:55.211 INFO:tasks.workunit.client.0.smithi077.stdout:2/180: creat d9/f3d x:0 0 0 2024-04-26T04:52:55.212 INFO:tasks.workunit.client.0.smithi077.stdout:7/172: dwrite da/d15/f44 [0,4194304] 0 2024-04-26T04:52:55.213 INFO:tasks.workunit.client.0.smithi077.stdout:2/181: chown d9/dc/d1d/d1e/l3c 574552647 1 2024-04-26T04:52:55.214 INFO:tasks.workunit.client.1.smithi078.stdout:5/415: creat d7/d18/d32/f8b x:0 0 0 2024-04-26T04:52:55.214 INFO:tasks.workunit.client.1.smithi078.stdout:5/416: dread - d7/d18/d1c/d5a/f7c zero size 2024-04-26T04:52:55.218 INFO:tasks.workunit.client.0.smithi077.stdout:8/185: truncate d4/df/f16 4064567 0 2024-04-26T04:52:55.228 INFO:tasks.workunit.client.1.smithi078.stdout:6/591: rename d2/de/f48 to d2/de/d27/da9/d2a/fb0 0 2024-04-26T04:52:55.230 INFO:tasks.workunit.client.1.smithi078.stdout:9/444: link dc/df/d4b/d3e/l65 dc/df/d4b/l84 0 2024-04-26T04:52:55.230 INFO:tasks.workunit.client.1.smithi078.stdout:9/445: chown dc/df/d4b/d22/d6e 139614 1 2024-04-26T04:52:55.234 INFO:tasks.workunit.client.1.smithi078.stdout:8/480: creat d0/d19/d52/d35/f8f x:0 0 0 2024-04-26T04:52:55.236 INFO:tasks.workunit.client.1.smithi078.stdout:4/523: truncate d0/d25/f19 3958474 0 2024-04-26T04:52:55.237 INFO:tasks.workunit.client.0.smithi077.stdout:3/178: write d0/f1 [4998315,44028] 0 2024-04-26T04:52:55.237 INFO:tasks.workunit.client.1.smithi078.stdout:5/417: dread d7/d48/d57/d65/d46/f4f [0,4194304] 0 2024-04-26T04:52:55.238 INFO:tasks.workunit.client.0.smithi077.stdout:3/179: stat d0/d9/d32/f28 0 2024-04-26T04:52:55.238 INFO:tasks.workunit.client.1.smithi078.stdout:1/540: dwrite d7/d1e/f36 [0,4194304] 0 2024-04-26T04:52:55.239 INFO:tasks.workunit.client.0.smithi077.stdout:3/180: write d0/f1b [748885,28352] 0 2024-04-26T04:52:55.239 INFO:tasks.workunit.client.1.smithi078.stdout:0/571: mkdir d2/dc1 0 2024-04-26T04:52:55.240 INFO:tasks.workunit.client.1.smithi078.stdout:4/524: truncate d0/d25/d20/d31/d47/db2/f4d 921846 0 2024-04-26T04:52:55.240 INFO:tasks.workunit.client.0.smithi077.stdout:3/181: truncate d0/d9/d32/f35 66242 0 2024-04-26T04:52:55.243 INFO:tasks.workunit.client.0.smithi077.stdout:2/182: mknod d9/d15/d1a/d2b/c3e 0 2024-04-26T04:52:55.244 INFO:tasks.workunit.client.0.smithi077.stdout:2/183: chown d9/dc/c35 1123 1 2024-04-26T04:52:55.245 INFO:tasks.workunit.client.1.smithi078.stdout:3/513: fdatasync d16/d26/d2b/d54/d56/f7d 0 2024-04-26T04:52:55.248 INFO:tasks.workunit.client.1.smithi078.stdout:7/488: rename d9/da/d11/d13/d1a/f1e to d9/da/d11/d13/d50/d8a/f9c 0 2024-04-26T04:52:55.250 INFO:tasks.workunit.client.1.smithi078.stdout:9/446: mknod dc/df/d37/c85 0 2024-04-26T04:52:55.252 INFO:tasks.workunit.client.0.smithi077.stdout:8/186: chown d4/l29 92 1 2024-04-26T04:52:55.254 INFO:tasks.workunit.client.0.smithi077.stdout:1/296: creat d9/db/dd/d34/d4a/f59 x:0 0 0 2024-04-26T04:52:55.260 INFO:tasks.workunit.client.1.smithi078.stdout:1/541: dread d7/d1e/d9e/db/f41 [0,4194304] 0 2024-04-26T04:52:55.261 INFO:tasks.workunit.client.1.smithi078.stdout:5/418: symlink d7/d18/d32/d37/d79/l8c 0 2024-04-26T04:52:55.262 INFO:tasks.workunit.client.1.smithi078.stdout:2/472: symlink d8/d11/d14/d69/d8f/d34/la7 0 2024-04-26T04:52:55.263 INFO:tasks.workunit.client.1.smithi078.stdout:2/473: chown d8/d11/d18/d5d/d61/c1d 0 1 2024-04-26T04:52:55.264 INFO:tasks.workunit.client.1.smithi078.stdout:2/474: stat d8/d11/d18/l4d 0 2024-04-26T04:52:55.268 INFO:tasks.workunit.client.0.smithi077.stdout:3/182: rmdir d0/d7 39 2024-04-26T04:52:55.271 INFO:tasks.workunit.client.0.smithi077.stdout:2/184: unlink d9/dc/d1d/d1e/l3c 0 2024-04-26T04:52:55.271 INFO:tasks.workunit.client.0.smithi077.stdout:2/185: readlink l6 0 2024-04-26T04:52:55.271 INFO:tasks.workunit.client.0.smithi077.stdout:2/186: readlink d9/d15/d1a/l2d 0 2024-04-26T04:52:55.276 INFO:tasks.workunit.client.0.smithi077.stdout:1/297: creat d9/db/dd/d34/f5a x:0 0 0 2024-04-26T04:52:55.277 INFO:tasks.workunit.client.0.smithi077.stdout:1/298: fdatasync d9/db/dd/d34/f5a 0 2024-04-26T04:52:55.279 INFO:tasks.workunit.client.0.smithi077.stdout:6/203: creat d8/f48 x:0 0 0 2024-04-26T04:52:55.280 INFO:tasks.workunit.client.1.smithi078.stdout:1/542: mkdir d7/d1e/d9e/db/d49/db2 0 2024-04-26T04:52:55.281 INFO:tasks.workunit.client.0.smithi077.stdout:5/193: rename d1/d8/c11 to d1/c47 0 2024-04-26T04:52:55.282 INFO:tasks.workunit.client.0.smithi077.stdout:5/194: fsync d1/d8/f10 0 2024-04-26T04:52:55.288 INFO:tasks.workunit.client.1.smithi078.stdout:4/525: mknod d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab/cc4 0 2024-04-26T04:52:55.289 INFO:tasks.workunit.client.0.smithi077.stdout:7/173: creat da/f45 x:0 0 0 2024-04-26T04:52:55.290 INFO:tasks.workunit.client.1.smithi078.stdout:0/572: sync 2024-04-26T04:52:55.291 INFO:tasks.workunit.client.1.smithi078.stdout:6/592: sync 2024-04-26T04:52:55.291 INFO:tasks.workunit.client.0.smithi077.stdout:2/187: fdatasync d9/fe 0 2024-04-26T04:52:55.293 INFO:tasks.workunit.client.1.smithi078.stdout:8/481: rename d0/d19/c3d to d0/d19/d25/d36/c90 0 2024-04-26T04:52:55.295 INFO:tasks.workunit.client.1.smithi078.stdout:1/543: creat d7/d1e/d9e/dd/d20/fb3 x:0 0 0 2024-04-26T04:52:55.299 INFO:tasks.workunit.client.0.smithi077.stdout:9/148: getdents d9/dd/d1b/d2b/d32 0 2024-04-26T04:52:55.301 INFO:tasks.workunit.client.0.smithi077.stdout:7/174: creat da/d13/d22/f46 x:0 0 0 2024-04-26T04:52:55.302 INFO:tasks.workunit.client.0.smithi077.stdout:7/175: write da/d15/f2a [270778,116380] 0 2024-04-26T04:52:55.304 INFO:tasks.workunit.client.1.smithi078.stdout:3/514: truncate d16/f22 1308838 0 2024-04-26T04:52:55.305 INFO:tasks.workunit.client.0.smithi077.stdout:2/188: creat d9/d15/d1a/f3f x:0 0 0 2024-04-26T04:52:55.306 INFO:tasks.workunit.client.1.smithi078.stdout:6/593: write d2/d1a/d5c/f98 [4660243,91369] 0 2024-04-26T04:52:55.307 INFO:tasks.workunit.client.1.smithi078.stdout:9/447: dwrite dc/df/f1d [0,4194304] 0 2024-04-26T04:52:55.308 INFO:tasks.workunit.client.1.smithi078.stdout:0/573: mknod d2/d9/d60/d6a/da5/cc2 0 2024-04-26T04:52:55.309 INFO:tasks.workunit.client.1.smithi078.stdout:6/594: read d2/de/d27/da9/da/f4a [473351,30895] 0 2024-04-26T04:52:55.309 INFO:tasks.workunit.client.1.smithi078.stdout:0/574: truncate d2/f48 1587763 0 2024-04-26T04:52:55.310 INFO:tasks.workunit.client.1.smithi078.stdout:0/575: chown d2/d9/d78/d84/f8e 17899391 1 2024-04-26T04:52:55.312 INFO:tasks.workunit.client.1.smithi078.stdout:0/576: truncate d2/d9/d18/db7/d92/fad 1037514 0 2024-04-26T04:52:55.312 INFO:tasks.workunit.client.1.smithi078.stdout:7/489: rename d9/da/d11/d13/f17 to d9/da/d33/d4b/d4c/d5b/d83/f9d 0 2024-04-26T04:52:55.314 INFO:tasks.workunit.client.1.smithi078.stdout:8/482: symlink d0/d19/d52/d35/d44/d6b/l91 0 2024-04-26T04:52:55.318 INFO:tasks.workunit.client.1.smithi078.stdout:8/483: dread d0/d19/d52/d35/d44/d6b/d6c/f86 [0,4194304] 0 2024-04-26T04:52:55.319 INFO:tasks.workunit.client.0.smithi077.stdout:5/195: getdents d1/d12/d45 0 2024-04-26T04:52:55.319 INFO:tasks.workunit.client.1.smithi078.stdout:2/475: write d8/d11/f3f [269705,104523] 0 2024-04-26T04:52:55.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/526: chown d0/d25/d20/d31/d47/db2/d4c/d8e/fb8 11114986 1 2024-04-26T04:52:55.327 INFO:tasks.workunit.client.0.smithi077.stdout:7/176: rename da/d13/f3d to da/d15/d1a/d39/f47 0 2024-04-26T04:52:55.328 INFO:tasks.workunit.client.1.smithi078.stdout:6/595: mknod d2/d1a/d38/cb1 0 2024-04-26T04:52:55.329 INFO:tasks.workunit.client.0.smithi077.stdout:8/187: chown d4/df/f16 15534 1 2024-04-26T04:52:55.329 INFO:tasks.workunit.client.0.smithi077.stdout:8/188: chown d4/c14 599 1 2024-04-26T04:52:55.332 INFO:tasks.workunit.client.0.smithi077.stdout:6/204: creat d8/f49 x:0 0 0 2024-04-26T04:52:55.333 INFO:tasks.workunit.client.0.smithi077.stdout:1/299: truncate d9/db/dd/f11 2351788 0 2024-04-26T04:52:55.336 INFO:tasks.workunit.client.1.smithi078.stdout:7/490: mkdir d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e 0 2024-04-26T04:52:55.341 INFO:tasks.workunit.client.0.smithi077.stdout:7/177: rename da/d13/f31 to da/d15/f48 0 2024-04-26T04:52:55.344 INFO:tasks.workunit.client.1.smithi078.stdout:3/515: mkdir d16/d26/d2e/d5f/d49/dad/db0 0 2024-04-26T04:52:55.344 INFO:tasks.workunit.client.1.smithi078.stdout:3/516: write d16/d26/d2b/d31/f59 [1001231,47818] 0 2024-04-26T04:52:55.345 INFO:tasks.workunit.client.1.smithi078.stdout:3/517: chown d16/d26/d2e/d5f/d49/c63 36768 1 2024-04-26T04:52:55.348 INFO:tasks.workunit.client.0.smithi077.stdout:9/149: truncate f2 455565 0 2024-04-26T04:52:55.350 INFO:tasks.workunit.client.0.smithi077.stdout:5/196: symlink d1/d12/d21/d25/d31/d3c/l48 0 2024-04-26T04:52:55.350 INFO:tasks.workunit.client.0.smithi077.stdout:5/197: stat d1/d12/d21/d25/d31/l32 0 2024-04-26T04:52:55.353 INFO:tasks.workunit.client.1.smithi078.stdout:7/491: fdatasync d9/da/f66 0 2024-04-26T04:52:55.353 INFO:tasks.workunit.client.1.smithi078.stdout:1/544: dwrite d7/d1e/d9e/f9f [4194304,4194304] 0 2024-04-26T04:52:55.354 INFO:tasks.workunit.client.0.smithi077.stdout:7/178: stat da/fc 0 2024-04-26T04:52:55.354 INFO:tasks.workunit.client.1.smithi078.stdout:7/492: dread - d9/da/d11/d13/d1a/d52/d5a/f92 zero size 2024-04-26T04:52:55.360 INFO:tasks.workunit.client.1.smithi078.stdout:2/476: unlink d8/c68 0 2024-04-26T04:52:55.362 INFO:tasks.workunit.client.1.smithi078.stdout:4/527: getdents d0/de/d22/d27/db5 0 2024-04-26T04:52:55.362 INFO:tasks.workunit.client.1.smithi078.stdout:2/477: truncate d8/d11/d18/d5d/d61/f76 510166 0 2024-04-26T04:52:55.362 INFO:tasks.workunit.client.1.smithi078.stdout:2/478: readlink l4 0 2024-04-26T04:52:55.365 INFO:tasks.workunit.client.1.smithi078.stdout:2/479: dread d8/d3e/f6a [0,4194304] 0 2024-04-26T04:52:55.365 INFO:tasks.workunit.client.1.smithi078.stdout:9/448: creat dc/df/d4b/f86 x:0 0 0 2024-04-26T04:52:55.366 INFO:tasks.workunit.client.0.smithi077.stdout:5/198: chown d1/c14 826112886 1 2024-04-26T04:52:55.366 INFO:tasks.workunit.client.1.smithi078.stdout:9/449: readlink dc/df/d27/l34 0 2024-04-26T04:52:55.366 INFO:tasks.workunit.client.1.smithi078.stdout:9/450: write dc/f60 [348690,43789] 0 2024-04-26T04:52:55.368 INFO:tasks.workunit.client.1.smithi078.stdout:6/596: mknod d2/d1a/cb2 0 2024-04-26T04:52:55.368 INFO:tasks.workunit.client.0.smithi077.stdout:2/189: link d9/d15/d1a/d2b/c3e d9/c40 0 2024-04-26T04:52:55.369 INFO:tasks.workunit.client.1.smithi078.stdout:6/597: chown d2/de/d27/f64 84701839 1 2024-04-26T04:52:55.370 INFO:tasks.workunit.client.1.smithi078.stdout:5/419: rename d7/d48/d57/d65/d70/d72/l86 to d7/l8d 0 2024-04-26T04:52:55.370 INFO:tasks.workunit.client.0.smithi077.stdout:8/189: creat d4/f40 x:0 0 0 2024-04-26T04:52:55.371 INFO:tasks.workunit.client.1.smithi078.stdout:3/518: dwrite d16/f25 [0,4194304] 0 2024-04-26T04:52:55.374 INFO:tasks.workunit.client.1.smithi078.stdout:1/545: chown d7/d1e/d9e/dd/l1a 77063 1 2024-04-26T04:52:55.375 INFO:tasks.workunit.client.1.smithi078.stdout:0/577: sync 2024-04-26T04:52:55.377 INFO:tasks.workunit.client.1.smithi078.stdout:7/493: creat d9/da/d11/d13/d1a/d65/f9f x:0 0 0 2024-04-26T04:52:55.380 INFO:tasks.workunit.client.1.smithi078.stdout:7/494: write d9/da/d11/d13/d1a/d52/d5a/f44 [4830345,61443] 0 2024-04-26T04:52:55.381 INFO:tasks.workunit.client.1.smithi078.stdout:1/546: dwrite d7/d1e/f36 [0,4194304] 0 2024-04-26T04:52:55.381 INFO:tasks.workunit.client.0.smithi077.stdout:5/199: creat d1/d8/f49 x:0 0 0 2024-04-26T04:52:55.389 INFO:tasks.workunit.client.0.smithi077.stdout:5/200: dwrite d1/d12/d28/f1c [4194304,4194304] 0 2024-04-26T04:52:55.391 INFO:tasks.workunit.client.1.smithi078.stdout:9/451: creat dc/df/d27/d81/f87 x:0 0 0 2024-04-26T04:52:55.393 INFO:tasks.workunit.client.0.smithi077.stdout:6/205: getdents d8/d1c 0 2024-04-26T04:52:55.395 INFO:tasks.workunit.client.0.smithi077.stdout:1/300: getdents d9/db/dd/d34 0 2024-04-26T04:52:55.397 INFO:tasks.workunit.client.1.smithi078.stdout:0/578: dread d2/d9/d18/f45 [0,4194304] 0 2024-04-26T04:52:55.398 INFO:tasks.workunit.client.1.smithi078.stdout:5/420: mkdir d7/d18/d1c/d8e 0 2024-04-26T04:52:55.406 INFO:tasks.workunit.client.0.smithi077.stdout:5/201: rename d1/d8/c20 to d1/d30/d33/c4a 0 2024-04-26T04:52:55.406 INFO:tasks.workunit.client.1.smithi078.stdout:4/528: dread d0/d1d/d46/f6e [0,4194304] 0 2024-04-26T04:52:55.407 INFO:tasks.workunit.client.1.smithi078.stdout:4/529: chown d0/de/f34 8360 1 2024-04-26T04:52:55.407 INFO:tasks.workunit.client.0.smithi077.stdout:5/202: write d1/d8/fc [3138799,66269] 0 2024-04-26T04:52:55.408 INFO:tasks.workunit.client.0.smithi077.stdout:5/203: write d1/d8/fc [1128742,109098] 0 2024-04-26T04:52:55.409 INFO:tasks.workunit.client.0.smithi077.stdout:1/301: unlink d9/d53/l54 0 2024-04-26T04:52:55.413 INFO:tasks.workunit.client.1.smithi078.stdout:6/598: write d2/de/d27/da9/d3a/f62 [1937620,58962] 0 2024-04-26T04:52:55.415 INFO:tasks.workunit.client.0.smithi077.stdout:7/179: getdents da/d15 0 2024-04-26T04:52:55.416 INFO:tasks.workunit.client.0.smithi077.stdout:9/150: dread d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:52:55.417 INFO:tasks.workunit.client.0.smithi077.stdout:2/190: creat d9/dc/f41 x:0 0 0 2024-04-26T04:52:55.418 INFO:tasks.workunit.client.1.smithi078.stdout:9/452: creat dc/df/d17/f88 x:0 0 0 2024-04-26T04:52:55.419 INFO:tasks.workunit.client.0.smithi077.stdout:5/204: dwrite d1/d12/d21/f17 [0,4194304] 0 2024-04-26T04:52:55.421 INFO:tasks.workunit.client.1.smithi078.stdout:8/484: rename d0/d19/d52/d5c/d68 to d0/d19/d25/d32/d8a/d92 0 2024-04-26T04:52:55.425 INFO:tasks.workunit.client.1.smithi078.stdout:0/579: mkdir d2/d9/d60/dc3 0 2024-04-26T04:52:55.425 INFO:tasks.workunit.client.1.smithi078.stdout:0/580: stat d2/d39/d96 0 2024-04-26T04:52:55.429 INFO:tasks.workunit.client.0.smithi077.stdout:1/302: dread d9/db/d27/f30 [0,4194304] 0 2024-04-26T04:52:55.429 INFO:tasks.workunit.client.0.smithi077.stdout:1/303: write d9/f42 [4799912,61533] 0 2024-04-26T04:52:55.435 INFO:tasks.workunit.client.1.smithi078.stdout:4/530: creat d0/d25/d20/d31/d47/db2/fc5 x:0 0 0 2024-04-26T04:52:55.435 INFO:tasks.workunit.client.0.smithi077.stdout:6/206: truncate d8/d2a/f32 948919 0 2024-04-26T04:52:55.436 INFO:tasks.workunit.client.1.smithi078.stdout:4/531: dread - d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/fbe zero size 2024-04-26T04:52:55.436 INFO:tasks.workunit.client.1.smithi078.stdout:4/532: dread - d0/f93 zero size 2024-04-26T04:52:55.439 INFO:tasks.workunit.client.1.smithi078.stdout:4/533: dread d0/de/f1a [4194304,4194304] 0 2024-04-26T04:52:55.440 INFO:tasks.workunit.client.1.smithi078.stdout:4/534: chown d0/d25/d20/d31/d47/db2 9 1 2024-04-26T04:52:55.440 INFO:tasks.workunit.client.0.smithi077.stdout:2/191: readlink d9/dc/d1d/d1e/l18 0 2024-04-26T04:52:55.442 INFO:tasks.workunit.client.0.smithi077.stdout:8/190: getdents d4/d17/d35 0 2024-04-26T04:52:55.443 INFO:tasks.workunit.client.1.smithi078.stdout:2/480: creat d8/d11/d18/d5d/fa8 x:0 0 0 2024-04-26T04:52:55.444 INFO:tasks.workunit.client.0.smithi077.stdout:5/205: rmdir d1/d12/d21/d25/d31/d3c 39 2024-04-26T04:52:55.449 INFO:tasks.workunit.client.0.smithi077.stdout:6/207: dwrite f3 [0,4194304] 0 2024-04-26T04:52:55.450 INFO:tasks.workunit.client.1.smithi078.stdout:5/421: rename d7/da/d1e/d4d to d7/d18/d32/d37/d8f 0 2024-04-26T04:52:55.451 INFO:tasks.workunit.client.1.smithi078.stdout:5/422: truncate d7/d48/d57/d65/d46/f54 582556 0 2024-04-26T04:52:55.452 INFO:tasks.workunit.client.1.smithi078.stdout:5/423: stat d7/da/f73 0 2024-04-26T04:52:55.452 INFO:tasks.workunit.client.0.smithi077.stdout:7/180: dwrite da/d15/f48 [0,4194304] 0 2024-04-26T04:52:55.453 INFO:tasks.workunit.client.1.smithi078.stdout:5/424: truncate d7/da/d1e/f4c 4499763 0 2024-04-26T04:52:55.453 INFO:tasks.workunit.client.0.smithi077.stdout:1/304: creat d9/db/dd/f5b x:0 0 0 2024-04-26T04:52:55.457 INFO:tasks.workunit.client.1.smithi078.stdout:2/481: dread d8/d11/d14/f67 [4194304,4194304] 0 2024-04-26T04:52:55.457 INFO:tasks.workunit.client.1.smithi078.stdout:8/485: stat d0/d19/d52/c15 0 2024-04-26T04:52:55.458 INFO:tasks.workunit.client.1.smithi078.stdout:0/581: creat d2/d9/d60/d6a/fc4 x:0 0 0 2024-04-26T04:52:55.462 INFO:tasks.workunit.client.0.smithi077.stdout:8/191: dwrite d4/f40 [0,4194304] 0 2024-04-26T04:52:55.462 INFO:tasks.workunit.client.1.smithi078.stdout:7/495: creat d9/da/fa0 x:0 0 0 2024-04-26T04:52:55.464 INFO:tasks.workunit.client.1.smithi078.stdout:8/486: dwrite d0/d19/d52/d35/f8f [0,4194304] 0 2024-04-26T04:52:55.465 INFO:tasks.workunit.client.1.smithi078.stdout:7/496: write d9/da/d11/d13/d1a/f4e [696377,13938] 0 2024-04-26T04:52:55.465 INFO:tasks.workunit.client.0.smithi077.stdout:5/206: dread d1/d12/d21/f34 [0,4194304] 0 2024-04-26T04:52:55.465 INFO:tasks.workunit.client.0.smithi077.stdout:7/181: write da/d15/f2a [4358022,41853] 0 2024-04-26T04:52:55.466 INFO:tasks.workunit.client.1.smithi078.stdout:1/547: rmdir d7/d1e/d9e/db/d49/db2 0 2024-04-26T04:52:55.467 INFO:tasks.workunit.client.0.smithi077.stdout:9/151: dwrite d9/dd/d28/f33 [0,4194304] 0 2024-04-26T04:52:55.467 INFO:tasks.workunit.client.0.smithi077.stdout:2/192: rename d9/dc/d1d/f38 to d9/d15/d1a/d2b/f42 0 2024-04-26T04:52:55.471 INFO:tasks.workunit.client.0.smithi077.stdout:5/207: truncate d1/d12/d28/f2e 827170 0 2024-04-26T04:52:55.472 INFO:tasks.workunit.client.0.smithi077.stdout:7/182: write da/d15/d1a/f20 [522663,109611] 0 2024-04-26T04:52:55.474 INFO:tasks.workunit.client.0.smithi077.stdout:6/208: stat d8/c29 0 2024-04-26T04:52:55.475 INFO:tasks.workunit.client.0.smithi077.stdout:6/209: chown d8/c16 0 1 2024-04-26T04:52:55.475 INFO:tasks.workunit.client.1.smithi078.stdout:1/548: dwrite d7/d1e/d9e/dd/d20/fb3 [0,4194304] 0 2024-04-26T04:52:55.478 INFO:tasks.workunit.client.1.smithi078.stdout:1/549: write d7/d1e/d9e/db/d47/f90 [891672,52192] 0 2024-04-26T04:52:55.478 INFO:tasks.workunit.client.1.smithi078.stdout:5/425: stat d7/d21/c78 0 2024-04-26T04:52:55.480 INFO:tasks.workunit.client.1.smithi078.stdout:5/426: readlink d7/d18/d32/d37/d79/l8c 0 2024-04-26T04:52:55.481 INFO:tasks.workunit.client.1.smithi078.stdout:5/427: chown d7/d11/f47 7 1 2024-04-26T04:52:55.482 INFO:tasks.workunit.client.1.smithi078.stdout:0/582: rename d2/d1e/f69 to d2/d9/d18/d7d/fc5 0 2024-04-26T04:52:55.482 INFO:tasks.workunit.client.1.smithi078.stdout:5/428: truncate d7/d48/d57/f89 559937 0 2024-04-26T04:52:55.483 INFO:tasks.workunit.client.1.smithi078.stdout:3/519: link d16/d26/c5e d16/d26/d2b/d31/cb1 0 2024-04-26T04:52:55.485 INFO:tasks.workunit.client.1.smithi078.stdout:3/520: dread - d16/d26/d2e/d5f/d49/f8f zero size 2024-04-26T04:52:55.485 INFO:tasks.workunit.client.0.smithi077.stdout:6/210: dwrite d8/f49 [0,4194304] 0 2024-04-26T04:52:55.485 INFO:tasks.workunit.client.0.smithi077.stdout:8/192: mknod d4/df/d18/c41 0 2024-04-26T04:52:55.486 INFO:tasks.workunit.client.1.smithi078.stdout:0/583: dread d2/f6 [0,4194304] 0 2024-04-26T04:52:55.488 INFO:tasks.workunit.client.1.smithi078.stdout:0/584: dread d2/d9/d18/f8a [0,4194304] 0 2024-04-26T04:52:55.495 INFO:tasks.workunit.client.1.smithi078.stdout:8/487: truncate d0/d19/d52/d35/d44/f4f 791732 0 2024-04-26T04:52:55.496 INFO:tasks.workunit.client.1.smithi078.stdout:8/488: chown d0/d19/d25/d36/d58/d5b 19127 1 2024-04-26T04:52:55.498 INFO:tasks.workunit.client.0.smithi077.stdout:9/152: rename d9/l31 to d9/dd/d1b/d2b/l37 0 2024-04-26T04:52:55.501 INFO:tasks.workunit.client.0.smithi077.stdout:5/208: creat d1/d12/d21/d25/f4b x:0 0 0 2024-04-26T04:52:55.503 INFO:tasks.workunit.client.0.smithi077.stdout:7/183: fdatasync da/ff 0 2024-04-26T04:52:55.505 INFO:tasks.workunit.client.0.smithi077.stdout:1/305: fdatasync d9/f1d 0 2024-04-26T04:52:55.506 INFO:tasks.workunit.client.1.smithi078.stdout:6/599: dwrite d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:52:55.506 INFO:tasks.workunit.client.1.smithi078.stdout:9/453: fsync dc/df/f2f 0 2024-04-26T04:52:55.508 INFO:tasks.workunit.client.0.smithi077.stdout:7/184: dread da/f3c [0,4194304] 0 2024-04-26T04:52:55.511 INFO:tasks.workunit.client.0.smithi077.stdout:2/193: mknod d9/dc/c43 0 2024-04-26T04:52:55.513 INFO:tasks.workunit.client.1.smithi078.stdout:1/550: creat d7/d1e/d9e/db/d49/fb4 x:0 0 0 2024-04-26T04:52:55.514 INFO:tasks.workunit.client.1.smithi078.stdout:1/551: dread - d7/d1e/d9e/db/d37/fa9 zero size 2024-04-26T04:52:55.515 INFO:tasks.workunit.client.0.smithi077.stdout:9/153: creat d9/dd/d1b/d2b/f38 x:0 0 0 2024-04-26T04:52:55.517 INFO:tasks.workunit.client.0.smithi077.stdout:5/209: rename d1/d12/l1a to d1/d42/l4c 0 2024-04-26T04:52:55.519 INFO:tasks.workunit.client.1.smithi078.stdout:5/429: mkdir d7/d21/d2f/d90 0 2024-04-26T04:52:55.520 INFO:tasks.workunit.client.1.smithi078.stdout:5/430: truncate d7/da/f75 646365 0 2024-04-26T04:52:55.525 INFO:tasks.workunit.client.1.smithi078.stdout:4/535: truncate d0/de/f34 2280590 0 2024-04-26T04:52:55.525 INFO:tasks.workunit.client.1.smithi078.stdout:4/536: chown d0/c80 100 1 2024-04-26T04:52:55.529 INFO:tasks.workunit.client.0.smithi077.stdout:2/194: mkdir d9/d15/d44 0 2024-04-26T04:52:55.530 INFO:tasks.workunit.client.0.smithi077.stdout:1/306: dwrite d9/db/f4c [0,4194304] 0 2024-04-26T04:52:55.534 INFO:tasks.workunit.client.0.smithi077.stdout:9/154: rename f7 to d9/dd/d1b/d2b/d32/f39 0 2024-04-26T04:52:55.535 INFO:tasks.workunit.client.0.smithi077.stdout:9/155: read d9/dd/d1b/f23 [361262,83274] 0 2024-04-26T04:52:55.537 INFO:tasks.workunit.client.0.smithi077.stdout:5/210: mknod d1/d12/c4d 0 2024-04-26T04:52:55.539 INFO:tasks.workunit.client.0.smithi077.stdout:8/193: creat d4/f42 x:0 0 0 2024-04-26T04:52:55.542 INFO:tasks.workunit.client.0.smithi077.stdout:5/211: dread d1/d12/f19 [0,4194304] 0 2024-04-26T04:52:55.543 INFO:tasks.workunit.client.0.smithi077.stdout:8/194: dread d4/df/f16 [0,4194304] 0 2024-04-26T04:52:55.544 INFO:tasks.workunit.client.0.smithi077.stdout:7/185: link da/d13/d22/f25 da/d15/d1a/f49 0 2024-04-26T04:52:55.544 INFO:tasks.workunit.client.0.smithi077.stdout:8/195: dread - d4/df/d18/f26 zero size 2024-04-26T04:52:55.545 INFO:tasks.workunit.client.0.smithi077.stdout:7/186: truncate da/d15/d1a/f20 1122640 0 2024-04-26T04:52:55.546 INFO:tasks.workunit.client.1.smithi078.stdout:1/552: mkdir d7/d1e/d9e/db/d47/d4a/d87/db5 0 2024-04-26T04:52:55.546 INFO:tasks.workunit.client.0.smithi077.stdout:7/187: chown da/f18 104986286 1 2024-04-26T04:52:55.546 INFO:tasks.workunit.client.1.smithi078.stdout:1/553: readlink d7/d1e/d9e/db/d47/l69 0 2024-04-26T04:52:55.547 INFO:tasks.workunit.client.0.smithi077.stdout:7/188: write da/d13/d22/f46 [209714,86974] 0 2024-04-26T04:52:55.551 INFO:tasks.workunit.client.0.smithi077.stdout:8/196: dwrite d4/df/f25 [0,4194304] 0 2024-04-26T04:52:55.552 INFO:tasks.workunit.client.1.smithi078.stdout:3/521: symlink d16/d7e/lb2 0 2024-04-26T04:52:55.552 INFO:tasks.workunit.client.0.smithi077.stdout:8/197: chown d4/df/d18/f26 86390 1 2024-04-26T04:52:55.553 INFO:tasks.workunit.client.0.smithi077.stdout:1/307: mkdir d9/db/d2b/d5c 0 2024-04-26T04:52:55.555 INFO:tasks.workunit.client.1.smithi078.stdout:4/537: mknod d0/d25/d20/d31/d47/cc6 0 2024-04-26T04:52:55.556 INFO:tasks.workunit.client.0.smithi077.stdout:1/308: dread d9/db/dd/d34/f38 [0,4194304] 0 2024-04-26T04:52:55.556 INFO:tasks.workunit.client.1.smithi078.stdout:4/538: chown d0/de/d22/fb7 42590609 1 2024-04-26T04:52:55.557 INFO:tasks.workunit.client.1.smithi078.stdout:4/539: stat d0/d25/d20/d31/d47/db2/d44/d55/d5e 0 2024-04-26T04:52:55.558 INFO:tasks.workunit.client.0.smithi077.stdout:1/309: chown d9/db/dd/d34/f5a 20985 1 2024-04-26T04:52:55.564 INFO:tasks.workunit.client.0.smithi077.stdout:1/310: dwrite d9/db/f4c [0,4194304] 0 2024-04-26T04:52:55.565 INFO:tasks.workunit.client.0.smithi077.stdout:1/311: chown d9/db/dd/l26 449 1 2024-04-26T04:52:55.566 INFO:tasks.workunit.client.0.smithi077.stdout:2/195: mkdir d9/dc/d1d/d45 0 2024-04-26T04:52:55.567 INFO:tasks.workunit.client.0.smithi077.stdout:2/196: write d9/d15/f2f [699169,70220] 0 2024-04-26T04:52:55.570 INFO:tasks.workunit.client.0.smithi077.stdout:9/156: mkdir d9/d3a 0 2024-04-26T04:52:55.571 INFO:tasks.workunit.client.0.smithi077.stdout:9/157: write d9/dd/f30 [207561,99264] 0 2024-04-26T04:52:55.574 INFO:tasks.workunit.client.0.smithi077.stdout:5/212: creat d1/d12/d21/d25/d31/d3c/f4e x:0 0 0 2024-04-26T04:52:55.575 INFO:tasks.workunit.client.0.smithi077.stdout:5/213: read - d1/d12/f1f zero size 2024-04-26T04:52:55.580 INFO:tasks.workunit.client.0.smithi077.stdout:1/312: truncate d9/db/d27/f28 161882 0 2024-04-26T04:52:55.583 INFO:tasks.workunit.client.1.smithi078.stdout:5/431: sync 2024-04-26T04:52:55.584 INFO:tasks.workunit.client.0.smithi077.stdout:1/313: dread d9/db/dd/f49 [0,4194304] 0 2024-04-26T04:52:55.585 INFO:tasks.workunit.client.0.smithi077.stdout:1/314: write d9/db/d27/f40 [55089,85868] 0 2024-04-26T04:52:55.585 INFO:tasks.workunit.client.0.smithi077.stdout:5/214: mknod d1/d12/d21/d25/d31/d3c/c4f 0 2024-04-26T04:52:55.587 INFO:tasks.workunit.client.0.smithi077.stdout:8/198: link d4/df/d22/d2d/f1b d4/df/d22/f43 0 2024-04-26T04:52:55.591 INFO:tasks.workunit.client.0.smithi077.stdout:9/158: mknod d9/d3a/c3b 0 2024-04-26T04:52:55.594 INFO:tasks.workunit.client.1.smithi078.stdout:7/497: creat d9/da/d11/d13/d1a/d52/d5a/fa1 x:0 0 0 2024-04-26T04:52:55.595 INFO:tasks.workunit.client.0.smithi077.stdout:7/189: dread da/f11 [0,4194304] 0 2024-04-26T04:52:55.597 INFO:tasks.workunit.client.0.smithi077.stdout:7/190: read da/d13/d22/f46 [81223,94957] 0 2024-04-26T04:52:55.598 INFO:tasks.workunit.client.0.smithi077.stdout:5/215: dwrite d1/d12/d21/f41 [0,4194304] 0 2024-04-26T04:52:55.598 INFO:tasks.workunit.client.0.smithi077.stdout:5/216: dread - d1/d12/d21/d25/f4b zero size 2024-04-26T04:52:55.599 INFO:tasks.workunit.client.0.smithi077.stdout:5/217: write d1/d8/f49 [515614,87855] 0 2024-04-26T04:52:55.625 INFO:tasks.workunit.client.1.smithi078.stdout:8/489: unlink d0/d19/d25/d32/d8a/c6d 0 2024-04-26T04:52:55.626 INFO:tasks.workunit.client.0.smithi077.stdout:8/199: symlink d4/df/d18/l44 0 2024-04-26T04:52:55.627 INFO:tasks.workunit.client.1.smithi078.stdout:9/454: mkdir dc/df/d89 0 2024-04-26T04:52:55.629 INFO:tasks.workunit.client.0.smithi077.stdout:9/159: dread - d9/dd/d1b/d2b/d32/f39 zero size 2024-04-26T04:52:55.648 INFO:tasks.workunit.client.0.smithi077.stdout:9/160: dread d9/f2d [0,4194304] 0 2024-04-26T04:52:55.648 INFO:tasks.workunit.client.0.smithi077.stdout:9/161: chown d9/dd/d1b/d2b/d32/d36 42241 1 2024-04-26T04:52:55.656 INFO:tasks.workunit.client.1.smithi078.stdout:6/600: creat d2/de/d27/da9/da/dab/fb3 x:0 0 0 2024-04-26T04:52:55.658 INFO:tasks.workunit.client.1.smithi078.stdout:2/482: link d8/d11/l97 d8/d3e/d8e/la9 0 2024-04-26T04:52:55.659 INFO:tasks.workunit.client.0.smithi077.stdout:5/218: rmdir d1 39 2024-04-26T04:52:55.662 INFO:tasks.workunit.client.0.smithi077.stdout:9/162: symlink d9/d3a/l3c 0 2024-04-26T04:52:55.664 INFO:tasks.workunit.client.0.smithi077.stdout:1/315: rename d9/db/dd/l1f to d9/db/l5d 0 2024-04-26T04:52:55.665 INFO:tasks.workunit.client.0.smithi077.stdout:1/316: write d9/db/dd/f5b [113958,35384] 0 2024-04-26T04:52:55.667 INFO:tasks.workunit.client.1.smithi078.stdout:5/432: mkdir d7/da/d1e/d91 0 2024-04-26T04:52:55.668 INFO:tasks.workunit.client.1.smithi078.stdout:5/433: write d7/d18/d32/d37/f41 [1043710,71373] 0 2024-04-26T04:52:55.669 INFO:tasks.workunit.client.1.smithi078.stdout:8/490: creat d0/d19/d52/d35/d44/f93 x:0 0 0 2024-04-26T04:52:55.671 INFO:tasks.workunit.client.0.smithi077.stdout:8/200: truncate d4/df/d22/f43 1659940 0 2024-04-26T04:52:55.674 INFO:tasks.workunit.client.1.smithi078.stdout:6/601: symlink d2/de/d27/d58/lb4 0 2024-04-26T04:52:55.677 INFO:tasks.workunit.client.0.smithi077.stdout:8/201: dwrite d4/fd [0,4194304] 0 2024-04-26T04:52:55.677 INFO:tasks.workunit.client.0.smithi077.stdout:7/191: rename da/d13/d22/l3b to da/d13/l4a 0 2024-04-26T04:52:55.678 INFO:tasks.workunit.client.0.smithi077.stdout:7/192: stat da/d15/d1a/d39 0 2024-04-26T04:52:55.679 INFO:tasks.workunit.client.0.smithi077.stdout:1/317: truncate d9/f44 186513 0 2024-04-26T04:52:55.680 INFO:tasks.workunit.client.0.smithi077.stdout:7/193: write da/d15/f2f [350552,43743] 0 2024-04-26T04:52:55.680 INFO:tasks.workunit.client.1.smithi078.stdout:2/483: mknod d8/d11/d6e/caa 0 2024-04-26T04:52:55.680 INFO:tasks.workunit.client.0.smithi077.stdout:1/318: chown d9/l46 8 1 2024-04-26T04:52:55.682 INFO:tasks.workunit.client.1.smithi078.stdout:2/484: write d8/d11/d14/d69/d8f/d34/d70/f75 [763819,130363] 0 2024-04-26T04:52:55.687 INFO:tasks.workunit.client.1.smithi078.stdout:0/585: getdents d2/d9/d31/d4a 0 2024-04-26T04:52:55.689 INFO:tasks.workunit.client.0.smithi077.stdout:5/219: rename d1/d12/d21/f1b to d1/d12/f50 0 2024-04-26T04:52:55.692 INFO:tasks.workunit.client.1.smithi078.stdout:9/455: write f1 [79351,89527] 0 2024-04-26T04:52:55.692 INFO:tasks.workunit.client.0.smithi077.stdout:8/202: symlink d4/d17/l45 0 2024-04-26T04:52:55.693 INFO:tasks.workunit.client.1.smithi078.stdout:9/456: chown dc/df/d4b/l62 336085205 1 2024-04-26T04:52:55.693 INFO:tasks.workunit.client.1.smithi078.stdout:0/586: dwrite d2/f6c [0,4194304] 0 2024-04-26T04:52:55.693 INFO:tasks.workunit.client.0.smithi077.stdout:8/203: write d4/f40 [268027,35766] 0 2024-04-26T04:52:55.695 INFO:tasks.workunit.client.0.smithi077.stdout:8/204: write d4/df/d22/d2d/f1e [2052035,2555] 0 2024-04-26T04:52:55.698 INFO:tasks.workunit.client.0.smithi077.stdout:1/319: rmdir d9/db/dd/d34 39 2024-04-26T04:52:55.702 INFO:tasks.workunit.client.1.smithi078.stdout:5/434: write f5 [1205484,9830] 0 2024-04-26T04:52:55.703 INFO:tasks.workunit.client.1.smithi078.stdout:5/435: read d7/d18/d32/f4a [168529,35173] 0 2024-04-26T04:52:55.703 INFO:tasks.workunit.client.1.smithi078.stdout:5/436: dread - d7/d18/d1c/f7d zero size 2024-04-26T04:52:55.704 INFO:tasks.workunit.client.0.smithi077.stdout:5/220: rename d1/l9 to d1/d12/d21/l51 0 2024-04-26T04:52:55.706 INFO:tasks.workunit.client.0.smithi077.stdout:5/221: write d1/d8/ff [2341211,128302] 0 2024-04-26T04:52:55.706 INFO:tasks.workunit.client.1.smithi078.stdout:3/522: rename d16/d7e/l9f to d16/d3c/lb3 0 2024-04-26T04:52:55.707 INFO:tasks.workunit.client.0.smithi077.stdout:7/194: write da/d15/d1a/f49 [782576,73558] 0 2024-04-26T04:52:55.708 INFO:tasks.workunit.client.1.smithi078.stdout:6/602: write d2/de/d27/d4c/d5f/f9f [1569619,4483] 0 2024-04-26T04:52:55.712 INFO:tasks.workunit.client.0.smithi077.stdout:8/205: creat d4/df/f46 x:0 0 0 2024-04-26T04:52:55.716 INFO:tasks.workunit.client.0.smithi077.stdout:9/163: getdents d9 0 2024-04-26T04:52:55.716 INFO:tasks.workunit.client.0.smithi077.stdout:9/164: write d9/f11 [2419504,79075] 0 2024-04-26T04:52:55.717 INFO:tasks.workunit.client.0.smithi077.stdout:9/165: readlink d9/dd/d1b/l25 0 2024-04-26T04:52:55.718 INFO:tasks.workunit.client.0.smithi077.stdout:9/166: readlink d9/dd/d1b/l25 0 2024-04-26T04:52:55.721 INFO:tasks.workunit.client.0.smithi077.stdout:5/222: symlink d1/d12/l52 0 2024-04-26T04:52:55.722 INFO:tasks.workunit.client.1.smithi078.stdout:5/437: creat d7/d18/d1c/f92 x:0 0 0 2024-04-26T04:52:55.724 INFO:tasks.workunit.client.1.smithi078.stdout:7/498: rename d9/da/d11/d13/d1a/d52/d5a/d41/c96 to d9/da/d11/d13/d50/d8a/ca2 0 2024-04-26T04:52:55.725 INFO:tasks.workunit.client.1.smithi078.stdout:7/499: chown d9/fe 59 1 2024-04-26T04:52:55.725 INFO:tasks.workunit.client.1.smithi078.stdout:0/587: dwrite d2/d9/f20 [0,4194304] 0 2024-04-26T04:52:55.726 INFO:tasks.workunit.client.0.smithi077.stdout:8/206: mknod d4/df/c47 0 2024-04-26T04:52:55.726 INFO:tasks.workunit.client.0.smithi077.stdout:1/320: truncate d9/db/dd/d34/f38 1628864 0 2024-04-26T04:52:55.727 INFO:tasks.workunit.client.1.smithi078.stdout:0/588: chown d2/d9/d31/d73/f97 928 1 2024-04-26T04:52:55.727 INFO:tasks.workunit.client.1.smithi078.stdout:4/540: link d0/de/f7e d0/de/d22/d71/fc7 0 2024-04-26T04:52:55.727 INFO:tasks.workunit.client.0.smithi077.stdout:1/321: chown d9 53 1 2024-04-26T04:52:55.728 INFO:tasks.workunit.client.0.smithi077.stdout:9/167: mkdir d9/dd/d3d 0 2024-04-26T04:52:55.728 INFO:tasks.workunit.client.0.smithi077.stdout:7/195: dwrite da/f36 [0,4194304] 0 2024-04-26T04:52:55.729 INFO:tasks.workunit.client.0.smithi077.stdout:7/196: stat da/d13/c38 0 2024-04-26T04:52:55.729 INFO:tasks.workunit.client.1.smithi078.stdout:9/457: mkdir dc/df/d8a 0 2024-04-26T04:52:55.730 INFO:tasks.workunit.client.0.smithi077.stdout:7/197: dread - da/d15/d1a/d39/f47 zero size 2024-04-26T04:52:55.731 INFO:tasks.workunit.client.0.smithi077.stdout:5/223: rename d1/fe to d1/d30/f53 0 2024-04-26T04:52:55.732 INFO:tasks.workunit.client.0.smithi077.stdout:5/224: chown d1/d8/f49 185 1 2024-04-26T04:52:55.733 INFO:tasks.workunit.client.1.smithi078.stdout:1/554: link d7/l5d d7/d1e/d9e/d4c/lb6 0 2024-04-26T04:52:55.734 INFO:tasks.workunit.client.0.smithi077.stdout:8/207: stat d4/c20 0 2024-04-26T04:52:55.736 INFO:tasks.workunit.client.0.smithi077.stdout:8/208: chown d4/df/d22/f34 15 1 2024-04-26T04:52:55.736 INFO:tasks.workunit.client.1.smithi078.stdout:2/485: creat d8/d11/d14/fab x:0 0 0 2024-04-26T04:52:55.738 INFO:tasks.workunit.client.0.smithi077.stdout:1/322: rmdir d9/db/d2b 39 2024-04-26T04:52:55.739 INFO:tasks.workunit.client.0.smithi077.stdout:7/198: dwrite da/f45 [0,4194304] 0 2024-04-26T04:52:55.740 INFO:tasks.workunit.client.1.smithi078.stdout:5/438: creat d7/da/f93 x:0 0 0 2024-04-26T04:52:55.746 INFO:tasks.workunit.client.1.smithi078.stdout:8/491: rename d0/f1a to d0/d7a/d83/f94 0 2024-04-26T04:52:55.749 INFO:tasks.workunit.client.1.smithi078.stdout:3/523: link d16/d26/d2b/d31/f65 d16/d26/d2e/d5f/d49/fb4 0 2024-04-26T04:52:55.751 INFO:tasks.workunit.client.1.smithi078.stdout:3/524: write d16/d26/d2b/d54/f6c [660432,16978] 0 2024-04-26T04:52:55.753 INFO:tasks.workunit.client.0.smithi077.stdout:8/209: rename d4/df/d22/d2d/d2b/f2f to d4/df/d18/f48 0 2024-04-26T04:52:55.754 INFO:tasks.workunit.client.1.smithi078.stdout:8/492: dwrite d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:52:55.755 INFO:tasks.workunit.client.0.smithi077.stdout:7/199: creat da/d13/f4b x:0 0 0 2024-04-26T04:52:55.759 INFO:tasks.workunit.client.1.smithi078.stdout:2/486: creat d8/d11/d14/d69/d8f/d34/d70/fac x:0 0 0 2024-04-26T04:52:55.760 INFO:tasks.workunit.client.1.smithi078.stdout:2/487: fdatasync d8/d11/d14/f3b 0 2024-04-26T04:52:55.762 INFO:tasks.workunit.client.0.smithi077.stdout:7/200: dwrite da/f45 [0,4194304] 0 2024-04-26T04:52:55.762 INFO:tasks.workunit.client.1.smithi078.stdout:5/439: rename d7/d18/d32/d37/d8f/c74 to d7/d18/d32/d37/d8f/c94 0 2024-04-26T04:52:55.763 INFO:tasks.workunit.client.0.smithi077.stdout:7/201: write da/f36 [1085590,64605] 0 2024-04-26T04:52:55.765 INFO:tasks.workunit.client.0.smithi077.stdout:8/210: symlink d4/d17/d35/l49 0 2024-04-26T04:52:55.767 INFO:tasks.workunit.client.1.smithi078.stdout:6/603: rename d2/d1a/d41 to d2/de/d27/da9/da/d36/db5 0 2024-04-26T04:52:55.771 INFO:tasks.workunit.client.1.smithi078.stdout:3/525: symlink d16/d26/d2e/d74/lb5 0 2024-04-26T04:52:55.772 INFO:tasks.workunit.client.1.smithi078.stdout:9/458: mkdir dc/df/d89/d8b 0 2024-04-26T04:52:55.774 INFO:tasks.workunit.client.1.smithi078.stdout:9/459: dread dc/df/f2f [4194304,4194304] 0 2024-04-26T04:52:55.774 INFO:tasks.workunit.client.0.smithi077.stdout:7/202: unlink da/f45 0 2024-04-26T04:52:55.778 INFO:tasks.workunit.client.0.smithi077.stdout:8/211: rmdir d4/df 39 2024-04-26T04:52:55.780 INFO:tasks.workunit.client.1.smithi078.stdout:2/488: dread d8/d11/d18/d5d/d61/f3c [0,4194304] 0 2024-04-26T04:52:55.781 INFO:tasks.workunit.client.0.smithi077.stdout:5/225: link d1/l3d d1/d30/d35/l54 0 2024-04-26T04:52:55.782 INFO:tasks.workunit.client.1.smithi078.stdout:5/440: creat d7/d18/d1c/f95 x:0 0 0 2024-04-26T04:52:55.783 INFO:tasks.workunit.client.0.smithi077.stdout:1/323: dread d9/db/d2b/f3d [0,4194304] 0 2024-04-26T04:52:55.783 INFO:tasks.workunit.client.0.smithi077.stdout:1/324: readlink d9/db/dd/l17 0 2024-04-26T04:52:55.785 INFO:tasks.workunit.client.1.smithi078.stdout:7/500: rename d9/c6b to d9/da/d11/d13/d1a/d65/ca3 0 2024-04-26T04:52:55.786 INFO:tasks.workunit.client.1.smithi078.stdout:5/441: dread d7/d48/d57/d65/f5b [0,4194304] 0 2024-04-26T04:52:55.786 INFO:tasks.workunit.client.1.smithi078.stdout:7/501: truncate d9/da/d33/f70 661558 0 2024-04-26T04:52:55.787 INFO:tasks.workunit.client.1.smithi078.stdout:6/604: dread - d2/de/d27/da9/d3a/d42/f5a zero size 2024-04-26T04:52:55.788 INFO:tasks.workunit.client.1.smithi078.stdout:6/605: chown d2/d1a/d38/f68 6863884 1 2024-04-26T04:52:55.790 INFO:tasks.workunit.client.1.smithi078.stdout:4/541: write d0/de/d22/d27/d56/f5c [3247198,53815] 0 2024-04-26T04:52:55.790 INFO:tasks.workunit.client.0.smithi077.stdout:5/226: creat d1/d8/f55 x:0 0 0 2024-04-26T04:52:55.792 INFO:tasks.workunit.client.1.smithi078.stdout:3/526: mkdir d16/d26/d2b/db6 0 2024-04-26T04:52:55.792 INFO:tasks.workunit.client.0.smithi077.stdout:5/227: truncate d1/d12/d21/d25/f3a 577680 0 2024-04-26T04:52:55.797 INFO:tasks.workunit.client.0.smithi077.stdout:8/212: rename f0 to d4/df/d22/d2d/f4a 0 2024-04-26T04:52:55.799 INFO:tasks.workunit.client.1.smithi078.stdout:0/589: rename d2/d1e/d5a/d74 to d2/d9/d18/d7d/da9/dab/dc6 0 2024-04-26T04:52:55.799 INFO:tasks.workunit.client.1.smithi078.stdout:2/489: dread d8/d11/d14/d69/d8f/d34/f3d [0,4194304] 0 2024-04-26T04:52:55.801 INFO:tasks.workunit.client.0.smithi077.stdout:5/228: symlink d1/d12/d21/d25/d31/d3c/l56 0 2024-04-26T04:52:55.802 INFO:tasks.workunit.client.1.smithi078.stdout:4/542: dread d0/d25/d20/d31/d47/db2/d4c/d62/daa/fa6 [0,4194304] 0 2024-04-26T04:52:55.803 INFO:tasks.workunit.client.1.smithi078.stdout:5/442: read d7/d48/d57/d65/d46/f68 [929523,112214] 0 2024-04-26T04:52:55.805 INFO:tasks.workunit.client.1.smithi078.stdout:4/543: truncate d0/d25/d20/d31/d47/db2/fc5 707748 0 2024-04-26T04:52:55.810 INFO:tasks.workunit.client.0.smithi077.stdout:7/203: dwrite da/d13/f19 [0,4194304] 0 2024-04-26T04:52:55.812 INFO:tasks.workunit.client.1.smithi078.stdout:5/443: dwrite d7/d18/d1c/d5a/f7c [0,4194304] 0 2024-04-26T04:52:55.813 INFO:tasks.workunit.client.0.smithi077.stdout:1/325: dwrite d9/f1d [0,4194304] 0 2024-04-26T04:52:55.815 INFO:tasks.workunit.client.1.smithi078.stdout:4/544: dwrite d0/de/f1a [0,4194304] 0 2024-04-26T04:52:55.820 INFO:tasks.workunit.client.1.smithi078.stdout:1/555: rename d7/d2e/f5f to d7/d1e/d9e/db/d49/fb7 0 2024-04-26T04:52:55.821 INFO:tasks.workunit.client.0.smithi077.stdout:1/326: mkdir d9/db/d2b/d5e 0 2024-04-26T04:52:55.822 INFO:tasks.workunit.client.0.smithi077.stdout:7/204: dwrite da/d15/f2a [0,4194304] 0 2024-04-26T04:52:55.824 INFO:tasks.workunit.client.0.smithi077.stdout:1/327: write d9/db/dd/f5b [249610,77691] 0 2024-04-26T04:52:55.827 INFO:tasks.workunit.client.1.smithi078.stdout:0/590: dread d2/f3 [0,4194304] 0 2024-04-26T04:52:55.827 INFO:tasks.workunit.client.1.smithi078.stdout:2/490: dread - d8/d11/d14/d69/d8f/f5c zero size 2024-04-26T04:52:55.828 INFO:tasks.workunit.client.1.smithi078.stdout:7/502: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/f47 [4194304,4194304] 0 2024-04-26T04:52:55.829 INFO:tasks.workunit.client.0.smithi077.stdout:7/205: symlink da/l4c 0 2024-04-26T04:52:55.832 INFO:tasks.workunit.client.0.smithi077.stdout:7/206: chown da/d15 33407939 1 2024-04-26T04:52:55.832 INFO:tasks.workunit.client.1.smithi078.stdout:1/556: symlink d7/d1e/lb8 0 2024-04-26T04:52:55.834 INFO:tasks.workunit.client.1.smithi078.stdout:0/591: stat d2/d9/d31/d4a/c55 0 2024-04-26T04:52:55.834 INFO:tasks.workunit.client.1.smithi078.stdout:5/444: dwrite d7/d48/d57/d65/f67 [0,4194304] 0 2024-04-26T04:52:55.836 INFO:tasks.workunit.client.1.smithi078.stdout:1/557: dread d7/d1e/f7f [0,4194304] 0 2024-04-26T04:52:55.836 INFO:tasks.workunit.client.1.smithi078.stdout:9/460: rename dc/df/d4b/d22/d2d/l7c to dc/df/d8a/l8c 0 2024-04-26T04:52:55.836 INFO:tasks.workunit.client.0.smithi077.stdout:1/328: dwrite d9/f20 [0,4194304] 0 2024-04-26T04:52:55.837 INFO:tasks.workunit.client.0.smithi077.stdout:8/213: dread d4/df/d22/d2d/f1b [0,4194304] 0 2024-04-26T04:52:55.837 INFO:tasks.workunit.client.0.smithi077.stdout:5/229: dread d1/d30/f53 [0,4194304] 0 2024-04-26T04:52:55.839 INFO:tasks.workunit.client.0.smithi077.stdout:8/214: chown d4/d17/d35/l49 8934 1 2024-04-26T04:52:55.840 INFO:tasks.workunit.client.1.smithi078.stdout:1/558: truncate d7/d1e/d9e/db/d49/d93/faf 38847 0 2024-04-26T04:52:55.840 INFO:tasks.workunit.client.0.smithi077.stdout:7/207: unlink da/d15/d1a/l37 0 2024-04-26T04:52:55.840 INFO:tasks.workunit.client.0.smithi077.stdout:8/215: fdatasync d4/df/f1f 0 2024-04-26T04:52:55.840 INFO:tasks.workunit.client.0.smithi077.stdout:1/329: readlink d9/d53/l58 0 2024-04-26T04:52:55.842 INFO:tasks.workunit.client.0.smithi077.stdout:1/330: write d9/db/dd/d34/d4a/f59 [711973,41778] 0 2024-04-26T04:52:55.844 INFO:tasks.workunit.client.1.smithi078.stdout:5/445: mkdir d7/d18/d1c/d5a/d96 0 2024-04-26T04:52:55.844 INFO:tasks.workunit.client.0.smithi077.stdout:7/208: symlink da/d13/l4d 0 2024-04-26T04:52:55.845 INFO:tasks.workunit.client.0.smithi077.stdout:5/230: creat d1/d30/d35/f57 x:0 0 0 2024-04-26T04:52:55.846 INFO:tasks.workunit.client.1.smithi078.stdout:1/559: dread d7/d1e/d9e/db/d47/f90 [0,4194304] 0 2024-04-26T04:52:55.847 INFO:tasks.workunit.client.0.smithi077.stdout:7/209: truncate da/d13/d22/f2e 1987865 0 2024-04-26T04:52:55.848 INFO:tasks.workunit.client.0.smithi077.stdout:7/210: write da/d15/f48 [4900835,80786] 0 2024-04-26T04:52:55.849 INFO:tasks.workunit.client.0.smithi077.stdout:8/216: rename d4/df/d18/l44 to d4/df/d18/l4b 0 2024-04-26T04:52:55.850 INFO:tasks.workunit.client.1.smithi078.stdout:2/491: rmdir d8/d11/d14/d69/da2 0 2024-04-26T04:52:55.852 INFO:tasks.workunit.client.0.smithi077.stdout:1/331: creat d9/d53/f5f x:0 0 0 2024-04-26T04:52:55.854 INFO:tasks.workunit.client.0.smithi077.stdout:5/231: mknod d1/d12/d28/d38/c58 0 2024-04-26T04:52:55.858 INFO:tasks.workunit.client.0.smithi077.stdout:8/217: creat d4/df/d18/f4c x:0 0 0 2024-04-26T04:52:55.858 INFO:tasks.workunit.client.0.smithi077.stdout:8/218: readlink d4/df/d18/l3e 0 2024-04-26T04:52:55.862 INFO:tasks.workunit.client.1.smithi078.stdout:0/592: creat d2/fc7 x:0 0 0 2024-04-26T04:52:55.863 INFO:tasks.workunit.client.0.smithi077.stdout:5/232: symlink d1/d12/d45/l59 0 2024-04-26T04:52:55.863 INFO:tasks.workunit.client.0.smithi077.stdout:5/233: fsync d1/d12/d21/d25/f4b 0 2024-04-26T04:52:55.864 INFO:tasks.workunit.client.1.smithi078.stdout:7/503: rename d9/da/d11/d13/l36 to d9/da/la4 0 2024-04-26T04:52:55.866 INFO:tasks.workunit.client.0.smithi077.stdout:8/219: unlink d4/l2a 0 2024-04-26T04:52:55.867 INFO:tasks.workunit.client.1.smithi078.stdout:5/446: mknod d7/d48/d57/d65/c97 0 2024-04-26T04:52:55.867 INFO:tasks.workunit.client.0.smithi077.stdout:5/234: dread d1/d8/f49 [0,4194304] 0 2024-04-26T04:52:55.868 INFO:tasks.workunit.client.0.smithi077.stdout:1/332: creat d9/db/f60 x:0 0 0 2024-04-26T04:52:55.869 INFO:tasks.workunit.client.1.smithi078.stdout:4/545: link d0/de/d22/cc0 d0/d25/d20/d31/d47/db2/d44/d55/cc8 0 2024-04-26T04:52:55.870 INFO:tasks.workunit.client.0.smithi077.stdout:1/333: write d9/db/dd/d34/f51 [176314,74889] 0 2024-04-26T04:52:55.873 INFO:tasks.workunit.client.1.smithi078.stdout:4/546: dread d0/d25/d20/d31/d47/db2/d4c/d62/daa/fa6 [0,4194304] 0 2024-04-26T04:52:55.873 INFO:tasks.workunit.client.1.smithi078.stdout:9/461: rename dc/df/d4b/l62 to dc/df/d4b/d3e/d80/l8d 0 2024-04-26T04:52:55.874 INFO:tasks.workunit.client.1.smithi078.stdout:4/547: readlink d0/d25/d20/l18 0 2024-04-26T04:52:55.876 INFO:tasks.workunit.client.0.smithi077.stdout:5/235: rename d1/d12/d21/d25/f3a to d1/d30/d35/f5a 0 2024-04-26T04:52:55.876 INFO:tasks.workunit.client.0.smithi077.stdout:8/220: dread d4/df/d22/d2d/f1e [0,4194304] 0 2024-04-26T04:52:55.878 INFO:tasks.workunit.client.1.smithi078.stdout:1/560: fsync d7/d1e/d9e/db/d49/fb7 0 2024-04-26T04:52:55.878 INFO:tasks.workunit.client.1.smithi078.stdout:4/548: dread d0/de/d22/d27/d56/f60 [0,4194304] 0 2024-04-26T04:52:55.883 INFO:tasks.workunit.client.0.smithi077.stdout:5/236: symlink d1/d12/d28/l5b 0 2024-04-26T04:52:55.886 INFO:tasks.workunit.client.0.smithi077.stdout:8/221: creat d4/df/d22/d2d/d2b/f4d x:0 0 0 2024-04-26T04:52:55.886 INFO:tasks.workunit.client.1.smithi078.stdout:3/527: truncate d16/f17 2424946 0 2024-04-26T04:52:55.889 INFO:tasks.workunit.client.1.smithi078.stdout:9/462: dread dc/df/d4b/d22/d2d/f4d [4194304,4194304] 0 2024-04-26T04:52:55.890 INFO:tasks.workunit.client.1.smithi078.stdout:3/528: dread d16/d26/d2b/d54/f6c [0,4194304] 0 2024-04-26T04:52:55.890 INFO:tasks.workunit.client.1.smithi078.stdout:4/549: mknod d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab/cc9 0 2024-04-26T04:52:55.891 INFO:tasks.workunit.client.0.smithi077.stdout:8/222: creat d4/df/d22/d2d/d2b/d33/f4e x:0 0 0 2024-04-26T04:52:55.892 INFO:tasks.workunit.client.1.smithi078.stdout:4/550: write d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/f50 [343047,84897] 0 2024-04-26T04:52:55.892 INFO:tasks.workunit.client.0.smithi077.stdout:8/223: truncate d4/fe 635372 0 2024-04-26T04:52:55.892 INFO:tasks.workunit.client.1.smithi078.stdout:4/551: chown d0/de/d22/d27/d7b 5971 1 2024-04-26T04:52:55.893 INFO:tasks.workunit.client.0.smithi077.stdout:8/224: dread - d4/df/d18/f4c zero size 2024-04-26T04:52:55.895 INFO:tasks.workunit.client.1.smithi078.stdout:0/593: link d2/d1e/f2a d2/d39/d96/fc8 0 2024-04-26T04:52:55.898 INFO:tasks.workunit.client.1.smithi078.stdout:1/561: dread d7/d1e/d9e/db/d49/fb7 [0,4194304] 0 2024-04-26T04:52:55.899 INFO:tasks.workunit.client.1.smithi078.stdout:1/562: fsync d7/d1e/d9e/db/d37/f88 0 2024-04-26T04:52:55.900 INFO:tasks.workunit.client.0.smithi077.stdout:8/225: creat d4/d17/f4f x:0 0 0 2024-04-26T04:52:55.900 INFO:tasks.workunit.client.0.smithi077.stdout:8/226: readlink d4/d17/d35/l49 0 2024-04-26T04:52:55.901 INFO:tasks.workunit.client.1.smithi078.stdout:8/493: sync 2024-04-26T04:52:55.901 INFO:tasks.workunit.client.0.smithi077.stdout:5/237: link d1/l3d d1/d30/l5c 0 2024-04-26T04:52:55.902 INFO:tasks.workunit.client.0.smithi077.stdout:5/238: dread - d1/d8/f55 zero size 2024-04-26T04:52:55.902 INFO:tasks.workunit.client.1.smithi078.stdout:4/552: mknod d0/de/d22/d71/cca 0 2024-04-26T04:52:55.906 INFO:tasks.workunit.client.1.smithi078.stdout:8/494: dwrite d0/d19/d52/d35/f8f [0,4194304] 0 2024-04-26T04:52:55.907 INFO:tasks.workunit.client.1.smithi078.stdout:0/594: write d2/d9/d18/d7d/da9/dab/dc6/fb2 [5642137,59746] 0 2024-04-26T04:52:55.907 INFO:tasks.workunit.client.0.smithi077.stdout:8/227: dwrite d4/f13 [4194304,4194304] 0 2024-04-26T04:52:55.907 INFO:tasks.workunit.client.1.smithi078.stdout:8/495: chown d0/d19/d25/f87 2250 1 2024-04-26T04:52:55.909 INFO:tasks.workunit.client.1.smithi078.stdout:3/529: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7 0 2024-04-26T04:52:55.910 INFO:tasks.workunit.client.0.smithi077.stdout:7/211: dwrite da/f14 [0,4194304] 0 2024-04-26T04:52:55.920 INFO:tasks.workunit.client.1.smithi078.stdout:1/563: dwrite d7/d1e/d9e/db/d49/fb1 [0,4194304] 0 2024-04-26T04:52:55.920 INFO:tasks.workunit.client.1.smithi078.stdout:4/553: creat d0/d25/d20/d31/d47/db2/d4c/dba/fcb x:0 0 0 2024-04-26T04:52:55.922 INFO:tasks.workunit.client.0.smithi077.stdout:7/212: readlink da/d13/l4a 0 2024-04-26T04:52:55.922 INFO:tasks.workunit.client.0.smithi077.stdout:7/213: stat da/fc 0 2024-04-26T04:52:55.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/447: write d7/d48/d57/d65/f36 [1025599,14673] 0 2024-04-26T04:52:55.923 INFO:tasks.workunit.client.0.smithi077.stdout:7/214: chown da/ff 449 1 2024-04-26T04:52:55.926 INFO:tasks.workunit.client.0.smithi077.stdout:8/228: creat d4/df/d22/f50 x:0 0 0 2024-04-26T04:52:55.928 INFO:tasks.workunit.client.1.smithi078.stdout:1/564: rename d7/d2e/l97 to d7/d2e/lb9 0 2024-04-26T04:52:55.928 INFO:tasks.workunit.client.0.smithi077.stdout:7/215: mkdir da/d15/d42/d4e 0 2024-04-26T04:52:55.929 INFO:tasks.workunit.client.1.smithi078.stdout:0/595: symlink d2/dc1/lc9 0 2024-04-26T04:52:55.930 INFO:tasks.workunit.client.1.smithi078.stdout:0/596: readlink d2/d9/d31/d4a/l51 0 2024-04-26T04:52:55.930 INFO:tasks.workunit.client.0.smithi077.stdout:8/229: mknod d4/df/d18/c51 0 2024-04-26T04:52:55.931 INFO:tasks.workunit.client.0.smithi077.stdout:8/230: write d4/d17/f4f [486208,73087] 0 2024-04-26T04:52:55.932 INFO:tasks.workunit.client.1.smithi078.stdout:8/496: mkdir d0/d19/d52/d95 0 2024-04-26T04:52:55.933 INFO:tasks.workunit.client.1.smithi078.stdout:4/554: symlink d0/d25/lcc 0 2024-04-26T04:52:55.936 INFO:tasks.workunit.client.1.smithi078.stdout:1/565: rename d7/d1e/d9e/db/d47/d4a to d7/d1e/d9e/db/d49/dba 0 2024-04-26T04:52:55.939 INFO:tasks.workunit.client.1.smithi078.stdout:3/530: dread d16/d26/f32 [0,4194304] 0 2024-04-26T04:52:55.939 INFO:tasks.workunit.client.0.smithi077.stdout:7/216: dwrite da/d15/f44 [4194304,4194304] 0 2024-04-26T04:52:55.940 INFO:tasks.workunit.client.1.smithi078.stdout:0/597: creat d2/d9/d18/db7/d92/fca x:0 0 0 2024-04-26T04:52:55.942 INFO:tasks.workunit.client.1.smithi078.stdout:1/566: dread d7/d1e/d9e/dd/d20/fb3 [0,4194304] 0 2024-04-26T04:52:55.943 INFO:tasks.workunit.client.1.smithi078.stdout:3/531: dread d16/f25 [0,4194304] 0 2024-04-26T04:52:55.944 INFO:tasks.workunit.client.0.smithi077.stdout:7/217: dread da/f11 [0,4194304] 0 2024-04-26T04:52:55.946 INFO:tasks.workunit.client.1.smithi078.stdout:5/448: dread f3 [0,4194304] 0 2024-04-26T04:52:55.948 INFO:tasks.workunit.client.1.smithi078.stdout:5/449: read d7/da/d1e/f4c [2621209,124770] 0 2024-04-26T04:52:55.950 INFO:tasks.workunit.client.0.smithi077.stdout:7/218: rename da/d13/l4a to da/d15/d42/l4f 0 2024-04-26T04:52:55.951 INFO:tasks.workunit.client.0.smithi077.stdout:7/219: write da/d15/d1a/f20 [1532352,71075] 0 2024-04-26T04:52:55.951 INFO:tasks.workunit.client.1.smithi078.stdout:8/497: mknod d0/d19/d52/c96 0 2024-04-26T04:52:55.953 INFO:tasks.workunit.client.0.smithi077.stdout:7/220: read da/d13/d22/f2b [503736,121866] 0 2024-04-26T04:52:55.956 INFO:tasks.workunit.client.0.smithi077.stdout:5/239: truncate d1/d12/f19 625327 0 2024-04-26T04:52:55.957 INFO:tasks.workunit.client.0.smithi077.stdout:8/231: getdents d4/d17/d35 0 2024-04-26T04:52:55.958 INFO:tasks.workunit.client.0.smithi077.stdout:8/232: truncate d4/df/d22/f34 1418509 0 2024-04-26T04:52:55.961 INFO:tasks.workunit.client.0.smithi077.stdout:7/221: rename da/d15/d1a/d39/f47 to da/d15/d42/f50 0 2024-04-26T04:52:55.963 INFO:tasks.workunit.client.1.smithi078.stdout:8/498: dread d0/d19/d52/d35/d44/d6b/d6c/f86 [0,4194304] 0 2024-04-26T04:52:55.964 INFO:tasks.workunit.client.0.smithi077.stdout:5/240: unlink d1/d12/d28/d38/l46 0 2024-04-26T04:52:55.965 INFO:tasks.workunit.client.0.smithi077.stdout:8/233: creat d4/d17/d35/f52 x:0 0 0 2024-04-26T04:52:55.966 INFO:tasks.workunit.client.0.smithi077.stdout:8/234: chown d4/df/d22/d2d/f3d 3210954 1 2024-04-26T04:52:55.967 INFO:tasks.workunit.client.0.smithi077.stdout:8/235: write d4/df/d22/d2d/f1c [445027,39885] 0 2024-04-26T04:52:55.968 INFO:tasks.workunit.client.1.smithi078.stdout:5/450: creat d7/d48/d57/d65/d7a/f98 x:0 0 0 2024-04-26T04:52:55.969 INFO:tasks.workunit.client.0.smithi077.stdout:5/241: mknod d1/d12/c5d 0 2024-04-26T04:52:55.973 INFO:tasks.workunit.client.1.smithi078.stdout:8/499: mknod d0/d19/d52/d35/d44/d6b/d6c/c97 0 2024-04-26T04:52:55.976 INFO:tasks.workunit.client.1.smithi078.stdout:5/451: creat d7/d48/d57/f99 x:0 0 0 2024-04-26T04:52:55.977 INFO:tasks.workunit.client.1.smithi078.stdout:5/452: write d7/da/f93 [1007252,114677] 0 2024-04-26T04:52:55.977 INFO:tasks.workunit.client.0.smithi077.stdout:8/236: mknod d4/df/d22/d2d/c53 0 2024-04-26T04:52:55.980 INFO:tasks.workunit.client.0.smithi077.stdout:5/242: creat d1/d12/d45/f5e x:0 0 0 2024-04-26T04:52:55.981 INFO:tasks.workunit.client.1.smithi078.stdout:8/500: mknod d0/d19/d25/d36/c98 0 2024-04-26T04:52:55.982 INFO:tasks.workunit.client.0.smithi077.stdout:8/237: creat d4/df/d22/d2d/d2b/f54 x:0 0 0 2024-04-26T04:52:55.982 INFO:tasks.workunit.client.1.smithi078.stdout:3/532: link d16/d26/d2e/d5f/c99 d16/d3c/cb8 0 2024-04-26T04:52:55.983 INFO:tasks.workunit.client.0.smithi077.stdout:8/238: write d4/df/d18/f4c [113603,39920] 0 2024-04-26T04:52:55.984 INFO:tasks.workunit.client.1.smithi078.stdout:3/533: truncate d16/d26/d2b/d54/d5a/f5d 5152974 0 2024-04-26T04:52:55.984 INFO:tasks.workunit.client.0.smithi077.stdout:5/243: rmdir d1/d12/d28 39 2024-04-26T04:52:55.984 INFO:tasks.workunit.client.0.smithi077.stdout:5/244: chown d1/d12/c1e 47 1 2024-04-26T04:52:55.984 INFO:tasks.workunit.client.1.smithi078.stdout:3/534: write d16/d26/f62 [3926944,116722] 0 2024-04-26T04:52:55.986 INFO:tasks.workunit.client.1.smithi078.stdout:3/535: chown d16/d26/f2a 57308323 1 2024-04-26T04:52:55.986 INFO:tasks.workunit.client.1.smithi078.stdout:5/453: dread d7/f39 [0,4194304] 0 2024-04-26T04:52:55.987 INFO:tasks.workunit.client.1.smithi078.stdout:3/536: write d16/d26/f2c [3855264,73559] 0 2024-04-26T04:52:55.987 INFO:tasks.workunit.client.1.smithi078.stdout:5/454: dread d7/f39 [0,4194304] 0 2024-04-26T04:52:55.987 INFO:tasks.workunit.client.1.smithi078.stdout:8/501: creat d0/d19/d52/d35/d44/d6b/f99 x:0 0 0 2024-04-26T04:52:55.989 INFO:tasks.workunit.client.0.smithi077.stdout:8/239: dwrite d4/f40 [0,4194304] 0 2024-04-26T04:52:55.991 INFO:tasks.workunit.client.0.smithi077.stdout:5/245: rmdir d1/d42 39 2024-04-26T04:52:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:55 smithi077 ceph-mon[26311]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 192 MiB/s wr, 390 op/s 2024-04-26T04:52:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:55 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:55 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:55 smithi077 ceph-mon[26311]: Reconfiguring alertmanager.smithi077 (dependencies changed)... 2024-04-26T04:52:55.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:55 smithi077 ceph-mon[26311]: Reconfiguring daemon alertmanager.smithi077 on smithi077 2024-04-26T04:52:55.993 INFO:tasks.workunit.client.1.smithi078.stdout:3/537: fdatasync d16/d26/d2b/d54/d56/f7d 0 2024-04-26T04:52:55.995 INFO:tasks.workunit.client.0.smithi077.stdout:8/240: dread d4/df/d22/d2d/f1e [0,4194304] 0 2024-04-26T04:52:55.998 INFO:tasks.workunit.client.0.smithi077.stdout:8/241: mknod d4/d17/d35/c55 0 2024-04-26T04:52:56.000 INFO:tasks.workunit.client.0.smithi077.stdout:7/222: truncate da/d15/f44 1749712 0 2024-04-26T04:52:56.001 INFO:tasks.workunit.client.1.smithi078.stdout:3/538: read d16/f20 [291591,21231] 0 2024-04-26T04:52:56.002 INFO:tasks.workunit.client.0.smithi077.stdout:8/242: mkdir d4/df/d18/d56 0 2024-04-26T04:52:56.004 INFO:tasks.workunit.client.0.smithi077.stdout:7/223: dread da/d15/d1a/d39/f41 [0,4194304] 0 2024-04-26T04:52:56.005 INFO:tasks.workunit.client.0.smithi077.stdout:8/243: mknod d4/d17/c57 0 2024-04-26T04:52:56.008 INFO:tasks.workunit.client.0.smithi077.stdout:8/244: mknod d4/df/d22/d2d/d2b/c58 0 2024-04-26T04:52:56.010 INFO:tasks.workunit.client.0.smithi077.stdout:7/224: rename da/f16 to da/d13/d22/d34/f51 0 2024-04-26T04:52:56.011 INFO:tasks.workunit.client.1.smithi078.stdout:8/502: dread d0/d19/d52/f69 [0,4194304] 0 2024-04-26T04:52:56.012 INFO:tasks.workunit.client.1.smithi078.stdout:3/539: dread f9 [0,4194304] 0 2024-04-26T04:52:56.013 INFO:tasks.workunit.client.1.smithi078.stdout:3/540: truncate d16/d26/d2e/d5f/d49/f8f 26851 0 2024-04-26T04:52:56.013 INFO:tasks.workunit.client.0.smithi077.stdout:7/225: rename da/d15/d1a/l1e to da/d13/l52 0 2024-04-26T04:52:56.015 INFO:tasks.workunit.client.1.smithi078.stdout:5/455: write d7/d48/d57/d65/f5b [1878482,55922] 0 2024-04-26T04:52:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:55 smithi078 ceph-mon[33746]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 192 MiB/s wr, 390 op/s 2024-04-26T04:52:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:55 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:55 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:52:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:55 smithi078 ceph-mon[33746]: Reconfiguring alertmanager.smithi077 (dependencies changed)... 2024-04-26T04:52:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:55 smithi078 ceph-mon[33746]: Reconfiguring daemon alertmanager.smithi077 on smithi077 2024-04-26T04:52:56.018 INFO:tasks.workunit.client.1.smithi078.stdout:3/541: symlink d16/d26/d2b/lb9 0 2024-04-26T04:52:56.019 INFO:tasks.workunit.client.0.smithi077.stdout:7/226: dwrite da/fc [0,4194304] 0 2024-04-26T04:52:56.019 INFO:tasks.workunit.client.0.smithi077.stdout:8/245: link d4/df/d22/d2d/d2b/c58 d4/df/d22/d2d/c59 0 2024-04-26T04:52:56.020 INFO:tasks.workunit.client.0.smithi077.stdout:8/246: write d4/f40 [2895508,48506] 0 2024-04-26T04:52:56.022 INFO:tasks.workunit.client.1.smithi078.stdout:6/606: sync 2024-04-26T04:52:56.022 INFO:tasks.workunit.client.0.smithi077.stdout:8/247: truncate d4/df/d18/f26 570802 0 2024-04-26T04:52:56.027 INFO:tasks.workunit.client.0.smithi077.stdout:8/248: rmdir d4/df/d22/d2d/d2b/d33 39 2024-04-26T04:52:56.030 INFO:tasks.workunit.client.1.smithi078.stdout:6/607: mkdir d2/de/d27/da9/da/d36/db6 0 2024-04-26T04:52:56.036 INFO:tasks.workunit.client.0.smithi077.stdout:7/227: getdents da/d15/d1a 0 2024-04-26T04:52:56.038 INFO:tasks.workunit.client.1.smithi078.stdout:6/608: creat d2/de/d27/da9/da8/fb7 x:0 0 0 2024-04-26T04:52:56.038 INFO:tasks.workunit.client.1.smithi078.stdout:5/456: write d7/d48/d57/d65/f53 [676308,71828] 0 2024-04-26T04:52:56.038 INFO:tasks.workunit.client.1.smithi078.stdout:3/542: getdents d16/d26/d2b/d54/d56 0 2024-04-26T04:52:56.039 INFO:tasks.workunit.client.1.smithi078.stdout:8/503: dwrite d0/d19/d25/d36/d58/d5b/d61/f8c [0,4194304] 0 2024-04-26T04:52:56.041 INFO:tasks.workunit.client.1.smithi078.stdout:5/457: write d7/d21/f4e [655964,108784] 0 2024-04-26T04:52:56.042 INFO:tasks.workunit.client.1.smithi078.stdout:5/458: truncate f5 4361660 0 2024-04-26T04:52:56.046 INFO:tasks.workunit.client.0.smithi077.stdout:7/228: getdents da 0 2024-04-26T04:52:56.047 INFO:tasks.workunit.client.1.smithi078.stdout:3/543: dwrite d16/d26/d2e/d5f/d49/f4e [0,4194304] 0 2024-04-26T04:52:56.049 INFO:tasks.workunit.client.1.smithi078.stdout:3/544: write d16/d26/d2e/d5f/f9a [212452,68131] 0 2024-04-26T04:52:56.050 INFO:tasks.workunit.client.1.smithi078.stdout:8/504: rename d0/d7a/l89 to d0/d19/d52/d35/d44/d6b/d6c/l9a 0 2024-04-26T04:52:56.052 INFO:tasks.workunit.client.1.smithi078.stdout:6/609: getdents d2 0 2024-04-26T04:52:56.053 INFO:tasks.workunit.client.1.smithi078.stdout:8/505: creat d0/d19/d52/d35/d44/d6b/f9b x:0 0 0 2024-04-26T04:52:56.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/610: unlink d2/de/d27/da9/d3a/c90 0 2024-04-26T04:52:56.057 INFO:tasks.workunit.client.1.smithi078.stdout:8/506: mknod d0/d19/d52/d3c/c9c 0 2024-04-26T04:52:56.058 INFO:tasks.workunit.client.1.smithi078.stdout:6/611: creat d2/de/d27/da9/da/fb8 x:0 0 0 2024-04-26T04:52:56.059 INFO:tasks.workunit.client.1.smithi078.stdout:8/507: mkdir d0/d19/d52/d95/d9d 0 2024-04-26T04:52:56.063 INFO:tasks.workunit.client.1.smithi078.stdout:8/508: fdatasync d0/d19/d52/d35/f4e 0 2024-04-26T04:52:56.063 INFO:tasks.workunit.client.1.smithi078.stdout:5/459: dwrite d7/d18/f26 [4194304,4194304] 0 2024-04-26T04:52:56.071 INFO:tasks.workunit.client.1.smithi078.stdout:3/545: dread d16/d26/d2b/d54/d5a/f5d [0,4194304] 0 2024-04-26T04:52:56.071 INFO:tasks.workunit.client.1.smithi078.stdout:3/546: stat d16/d26/d2e/d74/l6b 0 2024-04-26T04:52:56.073 INFO:tasks.workunit.client.1.smithi078.stdout:3/547: write d16/d26/d2e/d74/f4f [1520981,120075] 0 2024-04-26T04:52:56.073 INFO:tasks.workunit.client.0.smithi077.stdout:7/229: dread da/fb [0,4194304] 0 2024-04-26T04:52:56.075 INFO:tasks.workunit.client.1.smithi078.stdout:7/504: sync 2024-04-26T04:52:56.075 INFO:tasks.workunit.client.1.smithi078.stdout:9/463: sync 2024-04-26T04:52:56.075 INFO:tasks.workunit.client.1.smithi078.stdout:2/492: sync 2024-04-26T04:52:56.077 INFO:tasks.workunit.client.0.smithi077.stdout:7/230: chown da/d13/c21 1624380445 1 2024-04-26T04:52:56.078 INFO:tasks.workunit.client.1.smithi078.stdout:2/493: creat d8/d11/d14/d69/d8f/d34/fad x:0 0 0 2024-04-26T04:52:56.080 INFO:tasks.workunit.client.0.smithi077.stdout:7/231: symlink da/d15/d1a/l53 0 2024-04-26T04:52:56.081 INFO:tasks.workunit.client.0.smithi077.stdout:7/232: stat da/d13/d22/d34/l3f 0 2024-04-26T04:52:56.081 INFO:tasks.workunit.client.1.smithi078.stdout:5/460: write d7/d18/d32/d37/f51 [985088,80760] 0 2024-04-26T04:52:56.081 INFO:tasks.workunit.client.1.smithi078.stdout:8/509: truncate d0/d19/d52/d35/f8f 1371763 0 2024-04-26T04:52:56.083 INFO:tasks.workunit.client.0.smithi077.stdout:7/233: getdents da/d15/d42/d4e 0 2024-04-26T04:52:56.083 INFO:tasks.workunit.client.0.smithi077.stdout:7/234: chown da/d15/d42/d4e 752140 1 2024-04-26T04:52:56.085 INFO:tasks.workunit.client.1.smithi078.stdout:9/464: dread dc/df/d4b/d22/d2d/f4d [4194304,4194304] 0 2024-04-26T04:52:56.085 INFO:tasks.workunit.client.1.smithi078.stdout:7/505: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/f51 [4194304,4194304] 0 2024-04-26T04:52:56.088 INFO:tasks.workunit.client.0.smithi077.stdout:7/235: dread da/d15/d1a/f49 [0,4194304] 0 2024-04-26T04:52:56.089 INFO:tasks.workunit.client.1.smithi078.stdout:9/465: chown dc/df/d4b/d22/d2a/l68 781 1 2024-04-26T04:52:56.089 INFO:tasks.workunit.client.1.smithi078.stdout:2/494: getdents d8/d11/d14 0 2024-04-26T04:52:56.089 INFO:tasks.workunit.client.1.smithi078.stdout:3/548: dwrite d16/f25 [0,4194304] 0 2024-04-26T04:52:56.090 INFO:tasks.workunit.client.1.smithi078.stdout:9/466: chown dc/d55 0 1 2024-04-26T04:52:56.091 INFO:tasks.workunit.client.1.smithi078.stdout:6/612: dread d2/d1a/f43 [0,4194304] 0 2024-04-26T04:52:56.096 INFO:tasks.workunit.client.1.smithi078.stdout:2/495: mknod d8/d11/d6e/cae 0 2024-04-26T04:52:56.097 INFO:tasks.workunit.client.1.smithi078.stdout:3/549: mkdir d16/d26/d2e/d5f/d49/d77/dba 0 2024-04-26T04:52:56.099 INFO:tasks.workunit.client.1.smithi078.stdout:7/506: write d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e [1704842,77308] 0 2024-04-26T04:52:56.100 INFO:tasks.workunit.client.1.smithi078.stdout:9/467: readlink dc/df/d4b/d3e/d80/l8d 0 2024-04-26T04:52:56.100 INFO:tasks.workunit.client.0.smithi077.stdout:7/236: link da/c3e da/d13/d22/c54 0 2024-04-26T04:52:56.102 INFO:tasks.workunit.client.1.smithi078.stdout:9/468: readlink dc/df/d4b/d3e/l63 0 2024-04-26T04:52:56.103 INFO:tasks.workunit.client.1.smithi078.stdout:8/510: dwrite d0/d19/d25/f2b [0,4194304] 0 2024-04-26T04:52:56.103 INFO:tasks.workunit.client.1.smithi078.stdout:6/613: rename d2/d49 to d2/de/d27/da9/da/d36/db9 0 2024-04-26T04:52:56.104 INFO:tasks.workunit.client.0.smithi077.stdout:7/237: fdatasync da/d15/d1a/f49 0 2024-04-26T04:52:56.105 INFO:tasks.workunit.client.1.smithi078.stdout:7/507: mkdir d9/da/d33/d4b/d4c/d5b/d83/da5 0 2024-04-26T04:52:56.107 INFO:tasks.workunit.client.1.smithi078.stdout:2/496: rename d8/d11/d14/d69/d8f/ca4 to d8/d3e/d8e/caf 0 2024-04-26T04:52:56.108 INFO:tasks.workunit.client.1.smithi078.stdout:6/614: unlink d2/d52/f7a 0 2024-04-26T04:52:56.109 INFO:tasks.workunit.client.1.smithi078.stdout:3/550: mkdir d16/d26/d2e/d5f/dbb 0 2024-04-26T04:52:56.110 INFO:tasks.workunit.client.1.smithi078.stdout:8/511: fdatasync d0/d19/d52/f29 0 2024-04-26T04:52:56.111 INFO:tasks.workunit.client.0.smithi077.stdout:7/238: dwrite da/d13/d22/f25 [0,4194304] 0 2024-04-26T04:52:56.111 INFO:tasks.workunit.client.1.smithi078.stdout:7/508: rmdir d9/da/d11/d13/d1a/d52/d5a/d41 39 2024-04-26T04:52:56.111 INFO:tasks.workunit.client.1.smithi078.stdout:3/551: dread - d16/d26/d2e/d5f/d49/d77/f95 zero size 2024-04-26T04:52:56.113 INFO:tasks.workunit.client.1.smithi078.stdout:7/509: truncate d9/da/d11/d13/d1a/d52/d5a/d28/f98 889052 0 2024-04-26T04:52:56.114 INFO:tasks.workunit.client.1.smithi078.stdout:7/510: write d9/da/d20/f9b [932450,92981] 0 2024-04-26T04:52:56.116 INFO:tasks.workunit.client.1.smithi078.stdout:7/511: dread - d9/da/d11/d13/d1a/d65/f9f zero size 2024-04-26T04:52:56.116 INFO:tasks.workunit.client.1.smithi078.stdout:7/512: readlink d9/l89 0 2024-04-26T04:52:56.125 INFO:tasks.workunit.client.1.smithi078.stdout:0/598: sync 2024-04-26T04:52:56.125 INFO:tasks.workunit.client.1.smithi078.stdout:4/555: sync 2024-04-26T04:52:56.125 INFO:tasks.workunit.client.1.smithi078.stdout:1/567: sync 2024-04-26T04:52:56.126 INFO:tasks.workunit.client.1.smithi078.stdout:0/599: dread - d2/d9/f75 zero size 2024-04-26T04:52:56.126 INFO:tasks.workunit.client.1.smithi078.stdout:0/600: chown d2/d1e/f67 0 1 2024-04-26T04:52:56.127 INFO:tasks.workunit.client.1.smithi078.stdout:1/568: truncate d7/d1e/d9e/db/d37/fa9 79623 0 2024-04-26T04:52:56.127 INFO:tasks.workunit.client.0.smithi077.stdout:4/215: sync 2024-04-26T04:52:56.127 INFO:tasks.workunit.client.0.smithi077.stdout:0/171: sync 2024-04-26T04:52:56.128 INFO:tasks.workunit.client.1.smithi078.stdout:1/569: chown d7/d1e/d9e/db/d49/ca7 611801817 1 2024-04-26T04:52:56.128 INFO:tasks.workunit.client.1.smithi078.stdout:4/556: read d0/d25/d20/fa9 [636253,78275] 0 2024-04-26T04:52:56.129 INFO:tasks.workunit.client.1.smithi078.stdout:9/469: rename dc/df/d4b/d22/d2a/l68 to dc/df/d4b/d22/d2a/d57/l8e 0 2024-04-26T04:52:56.130 INFO:tasks.workunit.client.0.smithi077.stdout:7/239: fsync da/f11 0 2024-04-26T04:52:56.131 INFO:tasks.workunit.client.1.smithi078.stdout:5/461: dread d7/d18/d1c/f83 [0,4194304] 0 2024-04-26T04:52:56.132 INFO:tasks.workunit.client.0.smithi077.stdout:0/172: symlink d0/d2c/l3c 0 2024-04-26T04:52:56.135 INFO:tasks.workunit.client.0.smithi077.stdout:4/216: mkdir d4/d17/d18/d43 0 2024-04-26T04:52:56.138 INFO:tasks.workunit.client.1.smithi078.stdout:3/552: fdatasync d16/d26/f7f 0 2024-04-26T04:52:56.140 INFO:tasks.workunit.client.0.smithi077.stdout:4/217: creat d4/de/f44 x:0 0 0 2024-04-26T04:52:56.144 INFO:tasks.workunit.client.0.smithi077.stdout:7/240: rename da/c43 to da/d15/d42/c55 0 2024-04-26T04:52:56.146 INFO:tasks.workunit.client.1.smithi078.stdout:7/513: mknod d9/da/d33/d4b/d4c/d5b/ca6 0 2024-04-26T04:52:56.146 INFO:tasks.workunit.client.1.smithi078.stdout:7/514: fsync f7 0 2024-04-26T04:52:56.147 INFO:tasks.workunit.client.0.smithi077.stdout:0/173: rename d0/db/d11/l18 to d0/d19/d1e/l3d 0 2024-04-26T04:52:56.148 INFO:tasks.workunit.client.0.smithi077.stdout:4/218: rename d4/de/d15/d1f/d35 to d4/de/d15/d1f/d35/d45 22 2024-04-26T04:52:56.150 INFO:tasks.workunit.client.0.smithi077.stdout:7/241: creat da/d15/d1a/f56 x:0 0 0 2024-04-26T04:52:56.151 INFO:tasks.workunit.client.1.smithi078.stdout:1/570: creat d7/d1e/d9e/db/d49/dba/d87/fbb x:0 0 0 2024-04-26T04:52:56.151 INFO:tasks.workunit.client.0.smithi077.stdout:0/174: rmdir d0/db/d11 39 2024-04-26T04:52:56.155 INFO:tasks.workunit.client.0.smithi077.stdout:7/242: creat da/f57 x:0 0 0 2024-04-26T04:52:56.155 INFO:tasks.workunit.client.1.smithi078.stdout:2/497: mkdir d8/d11/d18/d5d/d8b/db0 0 2024-04-26T04:52:56.157 INFO:tasks.workunit.client.1.smithi078.stdout:5/462: truncate d7/da/f10 8304998 0 2024-04-26T04:52:56.157 INFO:tasks.workunit.client.0.smithi077.stdout:0/175: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:52:56.159 INFO:tasks.workunit.client.0.smithi077.stdout:7/243: unlink da/f14 0 2024-04-26T04:52:56.164 INFO:tasks.workunit.client.0.smithi077.stdout:7/244: creat da/d13/d22/f58 x:0 0 0 2024-04-26T04:52:56.165 INFO:tasks.workunit.client.0.smithi077.stdout:0/176: dwrite d0/f3 [4194304,4194304] 0 2024-04-26T04:52:56.171 INFO:tasks.workunit.client.0.smithi077.stdout:4/219: dread d4/de/d19/f1b [0,4194304] 0 2024-04-26T04:52:56.175 INFO:tasks.workunit.client.1.smithi078.stdout:1/571: write d7/d1e/d9e/d4c/d3d/f91 [5643125,34711] 0 2024-04-26T04:52:56.180 INFO:tasks.workunit.client.1.smithi078.stdout:4/557: symlink d0/d25/d20/d31/d47/db2/d44/d55/lcd 0 2024-04-26T04:52:56.183 INFO:tasks.workunit.client.1.smithi078.stdout:2/498: rename d8/d11/d14/f19 to d8/d11/d18/fb1 0 2024-04-26T04:52:56.186 INFO:tasks.workunit.client.0.smithi077.stdout:4/220: symlink d4/d17/d18/d43/l46 0 2024-04-26T04:52:56.187 INFO:tasks.workunit.client.0.smithi077.stdout:7/245: link da/d15/f44 da/f59 0 2024-04-26T04:52:56.188 INFO:tasks.workunit.client.0.smithi077.stdout:7/246: fsync da/d13/d22/f2e 0 2024-04-26T04:52:56.190 INFO:tasks.workunit.client.0.smithi077.stdout:0/177: getdents d0 0 2024-04-26T04:52:56.192 INFO:tasks.workunit.client.0.smithi077.stdout:4/221: creat d4/de/f47 x:0 0 0 2024-04-26T04:52:56.192 INFO:tasks.workunit.client.1.smithi078.stdout:9/470: dwrite dc/f26 [0,4194304] 0 2024-04-26T04:52:56.194 INFO:tasks.workunit.client.1.smithi078.stdout:8/512: link d0/d19/d52/c96 d0/d19/d52/d95/c9e 0 2024-04-26T04:52:56.195 INFO:tasks.workunit.client.1.smithi078.stdout:8/513: dread - d0/d19/d52/d35/d44/d6b/f9b zero size 2024-04-26T04:52:56.196 INFO:tasks.workunit.client.1.smithi078.stdout:7/515: creat d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 x:0 0 0 2024-04-26T04:52:56.197 INFO:tasks.workunit.client.1.smithi078.stdout:3/553: write d16/f22 [2325396,29634] 0 2024-04-26T04:52:56.199 INFO:tasks.workunit.client.0.smithi077.stdout:4/222: rmdir d4/de/d15/d1f 39 2024-04-26T04:52:56.203 INFO:tasks.workunit.client.1.smithi078.stdout:4/558: creat d0/d25/d20/d31/d47/db2/d44/fce x:0 0 0 2024-04-26T04:52:56.205 INFO:tasks.workunit.client.1.smithi078.stdout:2/499: creat d8/d3e/d8e/d91/fb2 x:0 0 0 2024-04-26T04:52:56.207 INFO:tasks.workunit.client.1.smithi078.stdout:2/500: write d8/d11/d18/f98 [556332,30401] 0 2024-04-26T04:52:56.209 INFO:tasks.workunit.client.0.smithi077.stdout:7/247: truncate da/d15/d1a/f49 1455604 0 2024-04-26T04:52:56.210 INFO:tasks.workunit.client.0.smithi077.stdout:7/248: fsync da/d13/d22/f2e 0 2024-04-26T04:52:56.211 INFO:tasks.workunit.client.0.smithi077.stdout:7/249: fsync da/d13/f19 0 2024-04-26T04:52:56.214 INFO:tasks.workunit.client.0.smithi077.stdout:0/178: dread d0/db/ff [0,4194304] 0 2024-04-26T04:52:56.214 INFO:tasks.workunit.client.1.smithi078.stdout:0/601: getdents d2/d9/d78/d84 0 2024-04-26T04:52:56.216 INFO:tasks.workunit.client.0.smithi077.stdout:7/250: rename da/f18 to da/d13/d22/d34/f5a 0 2024-04-26T04:52:56.221 INFO:tasks.workunit.client.0.smithi077.stdout:4/223: dread d4/de/d15/f16 [0,4194304] 0 2024-04-26T04:52:56.222 INFO:tasks.workunit.client.0.smithi077.stdout:7/251: fdatasync da/d15/d42/f50 0 2024-04-26T04:52:56.223 INFO:tasks.workunit.client.0.smithi077.stdout:7/252: dread - da/d13/d22/d34/f51 zero size 2024-04-26T04:52:56.226 INFO:tasks.workunit.client.0.smithi077.stdout:0/179: creat d0/db/d11/f3e x:0 0 0 2024-04-26T04:52:56.226 INFO:tasks.workunit.client.0.smithi077.stdout:0/180: chown d0/d28 29 1 2024-04-26T04:52:56.227 INFO:tasks.workunit.client.0.smithi077.stdout:7/253: dread da/fb [0,4194304] 0 2024-04-26T04:52:56.227 INFO:tasks.workunit.client.0.smithi077.stdout:0/181: chown d0/d2c/l3a 22283213 1 2024-04-26T04:52:56.228 INFO:tasks.workunit.client.0.smithi077.stdout:0/182: write d0/f2 [3318860,20652] 0 2024-04-26T04:52:56.233 INFO:tasks.workunit.client.0.smithi077.stdout:4/224: symlink d4/de/d15/d1f/d25/l48 0 2024-04-26T04:52:56.234 INFO:tasks.workunit.client.0.smithi077.stdout:0/183: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:52:56.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/602: rmdir d2/d9/d18/db7/d92 39 2024-04-26T04:52:56.237 INFO:tasks.workunit.client.1.smithi078.stdout:3/554: dwrite d16/d26/d2e/d74/f97 [0,4194304] 0 2024-04-26T04:52:56.238 INFO:tasks.workunit.client.0.smithi077.stdout:7/254: link da/d15/d1a/d39/f41 da/d13/f5b 0 2024-04-26T04:52:56.238 INFO:tasks.workunit.client.0.smithi077.stdout:7/255: stat da/d15/c35 0 2024-04-26T04:52:56.239 INFO:tasks.workunit.client.1.smithi078.stdout:0/603: chown d2/d1e/d5a 1 1 2024-04-26T04:52:56.240 INFO:tasks.workunit.client.1.smithi078.stdout:1/572: dwrite d7/f8e [0,4194304] 0 2024-04-26T04:52:56.243 INFO:tasks.workunit.client.0.smithi077.stdout:4/225: mknod d4/de/d15/c49 0 2024-04-26T04:52:56.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/501: dwrite d8/d11/f31 [4194304,4194304] 0 2024-04-26T04:52:56.245 INFO:tasks.workunit.client.1.smithi078.stdout:2/502: dread - d8/d3e/d8e/d91/fb2 zero size 2024-04-26T04:52:56.246 INFO:tasks.workunit.client.1.smithi078.stdout:7/516: truncate d9/da/d20/f9b 913340 0 2024-04-26T04:52:56.249 INFO:tasks.workunit.client.1.smithi078.stdout:5/463: getdents d7/d18 0 2024-04-26T04:52:56.249 INFO:tasks.workunit.client.1.smithi078.stdout:2/503: truncate d8/d3e/d52/f9f 764039 0 2024-04-26T04:52:56.250 INFO:tasks.workunit.client.1.smithi078.stdout:6/615: getdents d2/de/d27/da9/d3a/d69 0 2024-04-26T04:52:56.256 INFO:tasks.workunit.client.1.smithi078.stdout:2/504: dwrite d8/f2b [0,4194304] 0 2024-04-26T04:52:56.257 INFO:tasks.workunit.client.0.smithi077.stdout:7/256: getdents da/d13/d22/d34 0 2024-04-26T04:52:56.258 INFO:tasks.workunit.client.1.smithi078.stdout:9/471: rename dc/df/d17/f44 to dc/df/f8f 0 2024-04-26T04:52:56.258 INFO:tasks.workunit.client.1.smithi078.stdout:4/559: dwrite d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [0,4194304] 0 2024-04-26T04:52:56.261 INFO:tasks.workunit.client.1.smithi078.stdout:9/472: write dc/df/d4b/d22/d6e/f71 [964578,127608] 0 2024-04-26T04:52:56.262 INFO:tasks.workunit.client.1.smithi078.stdout:2/505: dread d8/f2b [0,4194304] 0 2024-04-26T04:52:56.265 INFO:tasks.workunit.client.0.smithi077.stdout:4/226: getdents d4 0 2024-04-26T04:52:56.267 INFO:tasks.workunit.client.1.smithi078.stdout:0/604: symlink d2/d9/d60/lcb 0 2024-04-26T04:52:56.268 INFO:tasks.workunit.client.0.smithi077.stdout:7/257: truncate da/f59 472161 0 2024-04-26T04:52:56.270 INFO:tasks.workunit.client.0.smithi077.stdout:4/227: rename d4/l12 to d4/l4a 0 2024-04-26T04:52:56.271 INFO:tasks.workunit.client.1.smithi078.stdout:1/573: creat d7/d1e/d9e/dd/d20/fbc x:0 0 0 2024-04-26T04:52:56.272 INFO:tasks.workunit.client.0.smithi077.stdout:7/258: creat da/d13/f5c x:0 0 0 2024-04-26T04:52:56.272 INFO:tasks.workunit.client.1.smithi078.stdout:0/605: dwrite d2/d9/d31/d4a/f9b [0,4194304] 0 2024-04-26T04:52:56.273 INFO:tasks.workunit.client.0.smithi077.stdout:7/259: fdatasync da/f57 0 2024-04-26T04:52:56.274 INFO:tasks.workunit.client.1.smithi078.stdout:3/555: mkdir d16/d26/d2b/d31/d71/d8a/dbc 0 2024-04-26T04:52:56.274 INFO:tasks.workunit.client.1.smithi078.stdout:0/606: read d2/d1e/f9e [102122,3270] 0 2024-04-26T04:52:56.277 INFO:tasks.workunit.client.1.smithi078.stdout:7/517: mknod d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/ca8 0 2024-04-26T04:52:56.278 INFO:tasks.workunit.client.0.smithi077.stdout:7/260: mkdir da/d13/d22/d5d 0 2024-04-26T04:52:56.282 INFO:tasks.workunit.client.1.smithi078.stdout:4/560: read d0/de/f1a [4934122,46395] 0 2024-04-26T04:52:56.283 INFO:tasks.workunit.client.0.smithi077.stdout:7/261: mknod da/d13/d22/d34/c5e 0 2024-04-26T04:52:56.284 INFO:tasks.workunit.client.1.smithi078.stdout:8/514: link d0/d19/d25/l2e d0/d19/d52/d95/l9f 0 2024-04-26T04:52:56.284 INFO:tasks.workunit.client.1.smithi078.stdout:4/561: truncate d0/d25/d20/d31/d47/db2/d44/d69/f7a 852565 0 2024-04-26T04:52:56.286 INFO:tasks.workunit.client.1.smithi078.stdout:9/473: chown dc/df/d4b/d22/d2a/d57/l8e 44207 1 2024-04-26T04:52:56.289 INFO:tasks.workunit.client.0.smithi077.stdout:7/262: unlink da/fb 0 2024-04-26T04:52:56.290 INFO:tasks.workunit.client.0.smithi077.stdout:7/263: write da/d13/f4b [183445,76397] 0 2024-04-26T04:52:56.290 INFO:tasks.workunit.client.1.smithi078.stdout:2/506: mknod d8/d11/d6e/cb3 0 2024-04-26T04:52:56.292 INFO:tasks.workunit.client.0.smithi077.stdout:4/228: getdents d4/d17/d18 0 2024-04-26T04:52:56.296 INFO:tasks.workunit.client.1.smithi078.stdout:1/574: readlink d7/d1e/d9e/d4c/d3d/l8f 0 2024-04-26T04:52:56.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/229: mknod d4/d17/d18/d43/c4b 0 2024-04-26T04:52:56.300 INFO:tasks.workunit.client.0.smithi077.stdout:4/230: fdatasync d4/d17/f24 0 2024-04-26T04:52:56.302 INFO:tasks.workunit.client.0.smithi077.stdout:7/264: symlink da/d13/d22/d5d/l5f 0 2024-04-26T04:52:56.304 INFO:tasks.workunit.client.1.smithi078.stdout:0/607: creat d2/dc1/fcc x:0 0 0 2024-04-26T04:52:56.308 INFO:tasks.workunit.client.1.smithi078.stdout:5/464: mknod d7/da/d1e/d91/c9a 0 2024-04-26T04:52:56.311 INFO:tasks.workunit.client.0.smithi077.stdout:4/231: dwrite d4/f8 [4194304,4194304] 0 2024-04-26T04:52:56.313 INFO:tasks.workunit.client.1.smithi078.stdout:6/616: mkdir d2/de/d27/da9/d3a/d69/daa/dba 0 2024-04-26T04:52:56.315 INFO:tasks.workunit.client.1.smithi078.stdout:5/465: dwrite d7/d48/d57/d65/f59 [0,4194304] 0 2024-04-26T04:52:56.315 INFO:tasks.workunit.client.0.smithi077.stdout:4/232: chown d4/de/f44 121055315 1 2024-04-26T04:52:56.318 INFO:tasks.workunit.client.1.smithi078.stdout:4/562: truncate d0/de/d22/f98 9193592 0 2024-04-26T04:52:56.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/563: dread d0/d25/d20/d31/d47/db2/fc5 [0,4194304] 0 2024-04-26T04:52:56.322 INFO:tasks.workunit.client.0.smithi077.stdout:4/233: dwrite d4/f8 [4194304,4194304] 0 2024-04-26T04:52:56.322 INFO:tasks.workunit.client.0.smithi077.stdout:4/234: chown d4/d17/d18 56539113 1 2024-04-26T04:52:56.323 INFO:tasks.workunit.client.1.smithi078.stdout:9/474: rename dc/df/d4b/d22/d2a/f7f to dc/df/d4b/d22/d2a/d57/f90 0 2024-04-26T04:52:56.323 INFO:tasks.workunit.client.0.smithi077.stdout:4/235: read d4/f8 [7136682,68162] 0 2024-04-26T04:52:56.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/575: mknod d7/d1e/d9e/db/d49/cbd 0 2024-04-26T04:52:56.330 INFO:tasks.workunit.client.0.smithi077.stdout:4/236: mknod d4/d17/d18/c4c 0 2024-04-26T04:52:56.332 INFO:tasks.workunit.client.1.smithi078.stdout:0/608: unlink d2/d9/d60/d6a/f89 0 2024-04-26T04:52:56.334 INFO:tasks.workunit.client.0.smithi077.stdout:4/237: symlink d4/de/d19/l4d 0 2024-04-26T04:52:56.335 INFO:tasks.workunit.client.1.smithi078.stdout:3/556: write d16/d26/f37 [2981015,75766] 0 2024-04-26T04:52:56.339 INFO:tasks.workunit.client.1.smithi078.stdout:8/515: mknod d0/d19/d52/d35/d73/ca0 0 2024-04-26T04:52:56.339 INFO:tasks.workunit.client.1.smithi078.stdout:8/516: chown d0/d7a 131057889 1 2024-04-26T04:52:56.340 INFO:tasks.workunit.client.0.smithi077.stdout:4/238: mkdir d4/d4e 0 2024-04-26T04:52:56.341 INFO:tasks.workunit.client.1.smithi078.stdout:9/475: rmdir dc/df/d27 39 2024-04-26T04:52:56.343 INFO:tasks.workunit.client.1.smithi078.stdout:2/507: mkdir d8/d11/d18/d5d/d8b/db0/db4 0 2024-04-26T04:52:56.348 INFO:tasks.workunit.client.0.smithi077.stdout:4/239: rename d4/d17/d18/d43/c4b to d4/de/d15/c4f 0 2024-04-26T04:52:56.350 INFO:tasks.workunit.client.1.smithi078.stdout:5/466: mknod d7/d21/d2f/d90/c9b 0 2024-04-26T04:52:56.353 INFO:tasks.workunit.client.1.smithi078.stdout:4/564: symlink d0/de/d22/d27/db5/lcf 0 2024-04-26T04:52:56.359 INFO:tasks.workunit.client.0.smithi077.stdout:4/240: creat d4/de/d15/f50 x:0 0 0 2024-04-26T04:52:56.361 INFO:tasks.workunit.client.0.smithi077.stdout:4/241: mkdir d4/d17/d18/d43/d51 0 2024-04-26T04:52:56.365 INFO:tasks.workunit.client.0.smithi077.stdout:4/242: creat d4/de/d15/d1f/d35/f52 x:0 0 0 2024-04-26T04:52:56.369 INFO:tasks.workunit.client.1.smithi078.stdout:2/508: symlink d8/d11/d18/lb5 0 2024-04-26T04:52:56.369 INFO:tasks.workunit.client.1.smithi078.stdout:7/518: rmdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/d77 0 2024-04-26T04:52:56.370 INFO:tasks.workunit.client.1.smithi078.stdout:3/557: creat d16/d26/d2b/d31/d71/d8a/dbc/fbd x:0 0 0 2024-04-26T04:52:56.371 INFO:tasks.workunit.client.0.smithi077.stdout:4/243: dwrite d4/de/d15/d1f/f34 [0,4194304] 0 2024-04-26T04:52:56.371 INFO:tasks.workunit.client.1.smithi078.stdout:8/517: rename d0/d19/d52/d35/d44/f4f to d0/d19/fa1 0 2024-04-26T04:52:56.373 INFO:tasks.workunit.client.1.smithi078.stdout:2/509: chown d8/d11/f3f 1164375 1 2024-04-26T04:52:56.373 INFO:tasks.workunit.client.1.smithi078.stdout:4/565: mknod d0/de/d22/d71/dae/cd0 0 2024-04-26T04:52:56.375 INFO:tasks.workunit.client.1.smithi078.stdout:2/510: dread - d8/f84 zero size 2024-04-26T04:52:56.375 INFO:tasks.workunit.client.1.smithi078.stdout:2/511: symlink d8/d3e/d8e/lb6 2 2024-04-26T04:52:56.376 INFO:tasks.workunit.client.0.smithi077.stdout:4/244: mkdir d4/d17/d18/d43/d53 0 2024-04-26T04:52:56.376 INFO:tasks.workunit.client.1.smithi078.stdout:3/558: chown d16/f17 12716435 1 2024-04-26T04:52:56.380 INFO:tasks.workunit.client.1.smithi078.stdout:1/576: dwrite d7/d1e/d9e/dd/f55 [0,4194304] 0 2024-04-26T04:52:56.380 INFO:tasks.workunit.client.1.smithi078.stdout:6/617: dwrite d2/de/f33 [0,4194304] 0 2024-04-26T04:52:56.382 INFO:tasks.workunit.client.1.smithi078.stdout:3/559: read - d16/d26/d2b/fa9 zero size 2024-04-26T04:52:56.384 INFO:tasks.workunit.client.1.smithi078.stdout:9/476: dwrite dc/f39 [0,4194304] 0 2024-04-26T04:52:56.385 INFO:tasks.workunit.client.1.smithi078.stdout:2/512: mkdir d8/db7 0 2024-04-26T04:52:56.387 INFO:tasks.workunit.client.1.smithi078.stdout:2/513: write d8/d3e/f58 [862642,83501] 0 2024-04-26T04:52:56.387 INFO:tasks.workunit.client.0.smithi077.stdout:4/245: link d4/cc d4/de/d15/d1f/d35/c54 0 2024-04-26T04:52:56.390 INFO:tasks.workunit.client.1.smithi078.stdout:1/577: dread d7/d1e/d9e/db/d49/fb1 [0,4194304] 0 2024-04-26T04:52:56.390 INFO:tasks.workunit.client.1.smithi078.stdout:8/518: mkdir d0/d19/d52/d5c/da2 0 2024-04-26T04:52:56.390 INFO:tasks.workunit.client.1.smithi078.stdout:7/519: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d95 0 2024-04-26T04:52:56.391 INFO:tasks.workunit.client.1.smithi078.stdout:4/566: creat d0/d25/d20/d31/d47/db2/d44/d55/d5e/d75/fd1 x:0 0 0 2024-04-26T04:52:56.392 INFO:tasks.workunit.client.1.smithi078.stdout:9/477: dwrite dc/df/d4b/f86 [0,4194304] 0 2024-04-26T04:52:56.392 INFO:tasks.workunit.client.1.smithi078.stdout:6/618: mknod d2/de/d27/da9/da/dab/cbb 0 2024-04-26T04:52:56.395 INFO:tasks.workunit.client.1.smithi078.stdout:8/519: mknod d0/d19/d52/d35/d44/d6b/ca3 0 2024-04-26T04:52:56.415 INFO:tasks.workunit.client.0.smithi077.stdout:3/183: sync 2024-04-26T04:52:56.423 INFO:tasks.workunit.client.0.smithi077.stdout:3/184: write d0/d7/d15/f1a [2283866,28874] 0 2024-04-26T04:52:56.425 INFO:tasks.workunit.client.1.smithi078.stdout:7/520: dread - d9/da/d11/d13/d1a/d52/d5a/f26 zero size 2024-04-26T04:52:56.425 INFO:tasks.workunit.client.1.smithi078.stdout:3/560: mknod d16/d26/d2b/d31/d71/cbe 0 2024-04-26T04:52:56.426 INFO:tasks.workunit.client.0.smithi077.stdout:3/185: unlink d0/f1c 0 2024-04-26T04:52:56.427 INFO:tasks.workunit.client.1.smithi078.stdout:4/567: mknod d0/d25/d20/d31/d47/db2/d4c/dba/cd2 0 2024-04-26T04:52:56.429 INFO:tasks.workunit.client.1.smithi078.stdout:6/619: write d2/de/d27/da9/d3a/d42/f84 [532542,122620] 0 2024-04-26T04:52:56.430 INFO:tasks.workunit.client.1.smithi078.stdout:3/561: symlink d16/d26/d2b/d54/d5a/lbf 0 2024-04-26T04:52:56.431 INFO:tasks.workunit.client.1.smithi078.stdout:7/521: fsync d9/da/d11/d13/d1a/d52/d5a/f92 0 2024-04-26T04:52:56.431 INFO:tasks.workunit.client.0.smithi077.stdout:3/186: rmdir d0/d7/d1f/d3a 0 2024-04-26T04:52:56.432 INFO:tasks.workunit.client.1.smithi078.stdout:2/514: write d8/d11/d14/d69/d8f/f88 [2352113,51004] 0 2024-04-26T04:52:56.432 INFO:tasks.workunit.client.1.smithi078.stdout:8/520: dread d0/d19/d52/d35/d44/f26 [0,4194304] 0 2024-04-26T04:52:56.433 INFO:tasks.workunit.client.1.smithi078.stdout:4/568: creat d0/d1d/db4/fd3 x:0 0 0 2024-04-26T04:52:56.434 INFO:tasks.workunit.client.0.smithi077.stdout:3/187: creat d0/d9/d32/f3b x:0 0 0 2024-04-26T04:52:56.435 INFO:tasks.workunit.client.0.smithi077.stdout:3/188: read d0/d7/f34 [39970,122440] 0 2024-04-26T04:52:56.435 INFO:tasks.workunit.client.1.smithi078.stdout:4/569: fdatasync d0/f8 0 2024-04-26T04:52:56.436 INFO:tasks.workunit.client.0.smithi077.stdout:3/189: chown d0/d9/d32/f36 5800758 1 2024-04-26T04:52:56.436 INFO:tasks.workunit.client.1.smithi078.stdout:9/478: dwrite dc/df/d27/d81/f66 [0,4194304] 0 2024-04-26T04:52:56.437 INFO:tasks.workunit.client.0.smithi077.stdout:3/190: truncate d0/d7/f33 1291046 0 2024-04-26T04:52:56.437 INFO:tasks.workunit.client.1.smithi078.stdout:8/521: read - d0/d19/d25/d36/d58/d5b/d61/d67/f88 zero size 2024-04-26T04:52:56.439 INFO:tasks.workunit.client.1.smithi078.stdout:7/522: creat d9/da/d11/fa9 x:0 0 0 2024-04-26T04:52:56.444 INFO:tasks.workunit.client.1.smithi078.stdout:9/479: chown dc/df/d4b/l84 11369 1 2024-04-26T04:52:56.444 INFO:tasks.workunit.client.1.smithi078.stdout:4/570: dwrite d0/de/d22/fb7 [0,4194304] 0 2024-04-26T04:52:56.446 INFO:tasks.workunit.client.1.smithi078.stdout:4/571: read d0/d25/d20/f2a [4975276,112437] 0 2024-04-26T04:52:56.447 INFO:tasks.workunit.client.0.smithi077.stdout:3/191: rename d0/d7/d15/f1a to d0/d7/f3c 0 2024-04-26T04:52:56.448 INFO:tasks.workunit.client.1.smithi078.stdout:8/522: mkdir d0/d19/d25/d32/d8a/d92/da4 0 2024-04-26T04:52:56.450 INFO:tasks.workunit.client.1.smithi078.stdout:8/523: chown d0/d19/d52/d35/d44/c78 29 1 2024-04-26T04:52:56.453 INFO:tasks.workunit.client.1.smithi078.stdout:2/515: truncate d8/d11/d14/d69/d8f/f6b 865777 0 2024-04-26T04:52:56.454 INFO:tasks.workunit.client.1.smithi078.stdout:4/572: dwrite d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 [0,4194304] 0 2024-04-26T04:52:56.455 INFO:tasks.workunit.client.1.smithi078.stdout:5/467: truncate d7/f38 2964331 0 2024-04-26T04:52:56.456 INFO:tasks.workunit.client.1.smithi078.stdout:9/480: creat dc/df/d27/d81/f91 x:0 0 0 2024-04-26T04:52:56.457 INFO:tasks.workunit.client.1.smithi078.stdout:5/468: chown d7/d48/d57/d65/d46/c50 2522522 1 2024-04-26T04:52:56.457 INFO:tasks.workunit.client.1.smithi078.stdout:4/573: write d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 [1954810,17780] 0 2024-04-26T04:52:56.458 INFO:tasks.workunit.client.1.smithi078.stdout:9/481: chown dc/df/d17/c7a 3417780 1 2024-04-26T04:52:56.459 INFO:tasks.workunit.client.0.smithi077.stdout:3/192: creat d0/f3d x:0 0 0 2024-04-26T04:52:56.466 INFO:tasks.workunit.client.0.smithi077.stdout:3/193: truncate d0/fb 291132 0 2024-04-26T04:52:56.467 INFO:tasks.workunit.client.0.smithi077.stdout:3/194: write d0/f38 [1074450,107851] 0 2024-04-26T04:52:56.470 INFO:tasks.workunit.client.1.smithi078.stdout:9/482: readlink dc/df/d4b/d22/l51 0 2024-04-26T04:52:56.475 INFO:tasks.workunit.client.1.smithi078.stdout:4/574: rename d0/d1d/faf to d0/de/d22/d27/db5/fd4 0 2024-04-26T04:52:56.479 INFO:tasks.workunit.client.1.smithi078.stdout:0/609: sync 2024-04-26T04:52:56.480 INFO:tasks.workunit.client.1.smithi078.stdout:9/483: rename dc/df/d4b/d22/d6e to dc/df/d4b/d22/d2a/d92 0 2024-04-26T04:52:56.481 INFO:tasks.workunit.client.1.smithi078.stdout:4/575: mkdir d0/d25/d20/dd5 0 2024-04-26T04:52:56.481 INFO:tasks.workunit.client.1.smithi078.stdout:6/620: write d2/de/d27/da9/da/d35/f3e [929373,6524] 0 2024-04-26T04:52:56.482 INFO:tasks.workunit.client.1.smithi078.stdout:4/576: dread - d0/de/fad zero size 2024-04-26T04:52:56.483 INFO:tasks.workunit.client.1.smithi078.stdout:3/562: truncate d16/f69 150878 0 2024-04-26T04:52:56.485 INFO:tasks.workunit.client.1.smithi078.stdout:9/484: mkdir dc/df/d4b/d22/d2a/d57/d93 0 2024-04-26T04:52:56.487 INFO:tasks.workunit.client.1.smithi078.stdout:9/485: truncate dc/df/f1e 4765377 0 2024-04-26T04:52:56.489 INFO:tasks.workunit.client.1.smithi078.stdout:3/563: dwrite d16/d26/d2b/d54/d5a/f60 [4194304,4194304] 0 2024-04-26T04:52:56.489 INFO:tasks.workunit.client.1.smithi078.stdout:9/486: chown dc/df/d4b/d22/l51 16423626 1 2024-04-26T04:52:56.490 INFO:tasks.workunit.client.1.smithi078.stdout:6/621: symlink d2/de/d27/da9/d3a/d69/daa/dba/lbc 0 2024-04-26T04:52:56.495 INFO:tasks.workunit.client.1.smithi078.stdout:9/487: dread dc/df/d4b/f86 [0,4194304] 0 2024-04-26T04:52:56.496 INFO:tasks.workunit.client.1.smithi078.stdout:8/524: write d0/d19/d52/d35/d44/d6b/d6c/f86 [346295,66121] 0 2024-04-26T04:52:56.499 INFO:tasks.workunit.client.1.smithi078.stdout:3/564: truncate d16/f30 989837 0 2024-04-26T04:52:56.500 INFO:tasks.workunit.client.1.smithi078.stdout:5/469: write d7/d18/f44 [1400662,7030] 0 2024-04-26T04:52:56.500 INFO:tasks.workunit.client.1.smithi078.stdout:7/523: dwrite d9/f10 [0,4194304] 0 2024-04-26T04:52:56.503 INFO:tasks.workunit.client.1.smithi078.stdout:7/524: readlink d9/da/d11/d13/d1a/d2d/l78 0 2024-04-26T04:52:56.504 INFO:tasks.workunit.client.1.smithi078.stdout:6/622: getdents d2/de/d27/da9/da/d36/db9/d97 0 2024-04-26T04:52:56.504 INFO:tasks.workunit.client.1.smithi078.stdout:8/525: truncate d0/d19/d25/d36/d58/d5b/d61/d67/f88 71354 0 2024-04-26T04:52:56.506 INFO:tasks.workunit.client.1.smithi078.stdout:3/565: symlink d16/d3c/lc0 0 2024-04-26T04:52:56.507 INFO:tasks.workunit.client.1.smithi078.stdout:9/488: dread dc/df/d4b/d22/d2d/f4d [4194304,4194304] 0 2024-04-26T04:52:56.507 INFO:tasks.workunit.client.1.smithi078.stdout:2/516: truncate d8/d11/d14/f90 1353150 0 2024-04-26T04:52:56.511 INFO:tasks.workunit.client.1.smithi078.stdout:8/526: creat d0/d19/d52/d3c/fa5 x:0 0 0 2024-04-26T04:52:56.513 INFO:tasks.workunit.client.1.smithi078.stdout:4/577: dread - d0/d25/d20/d31/d47/db2/d44/d69/fa1 zero size 2024-04-26T04:52:56.513 INFO:tasks.workunit.client.1.smithi078.stdout:7/525: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/faa x:0 0 0 2024-04-26T04:52:56.515 INFO:tasks.workunit.client.1.smithi078.stdout:0/610: dread d2/d9/d18/d7d/da9/dab/dc6/f9c [0,4194304] 0 2024-04-26T04:52:56.515 INFO:tasks.workunit.client.1.smithi078.stdout:5/470: dwrite d7/d18/d1c/f71 [0,4194304] 0 2024-04-26T04:52:56.515 INFO:tasks.workunit.client.1.smithi078.stdout:2/517: rmdir d8/d11/d14/d69/d8f/d34/d70 39 2024-04-26T04:52:56.516 INFO:tasks.workunit.client.1.smithi078.stdout:3/566: truncate d16/d26/d2e/d5f/d49/fb4 5154666 0 2024-04-26T04:52:56.517 INFO:tasks.workunit.client.0.smithi077.stdout:3/195: write d0/fb [969579,1135] 0 2024-04-26T04:52:56.518 INFO:tasks.workunit.client.1.smithi078.stdout:4/578: chown d0/d25/d20/d31/d47/db2/d44/d55/d5e 205130533 1 2024-04-26T04:52:56.519 INFO:tasks.workunit.client.1.smithi078.stdout:8/527: mknod d0/d19/d52/d35/d44/ca6 0 2024-04-26T04:52:56.519 INFO:tasks.workunit.client.0.smithi077.stdout:3/196: truncate d0/d9/d32/f3b 604594 0 2024-04-26T04:52:56.519 INFO:tasks.workunit.client.1.smithi078.stdout:6/623: dwrite d2/d1a/f9b [0,4194304] 0 2024-04-26T04:52:56.520 INFO:tasks.workunit.client.1.smithi078.stdout:7/526: write d9/da/d11/d13/d1a/d52/d5a/d41/d68/faa [391860,116212] 0 2024-04-26T04:52:56.522 INFO:tasks.workunit.client.1.smithi078.stdout:7/527: chown d9/da/d11/f3a 2376 1 2024-04-26T04:52:56.527 INFO:tasks.workunit.client.1.smithi078.stdout:5/471: mkdir d7/d48/d57/d65/d7a/d9c 0 2024-04-26T04:52:56.531 INFO:tasks.workunit.client.1.smithi078.stdout:6/624: creat d2/d52/fbd x:0 0 0 2024-04-26T04:52:56.533 INFO:tasks.workunit.client.0.smithi077.stdout:3/197: rename d0/d7/f3c to d0/d7/f3e 0 2024-04-26T04:52:56.554 INFO:tasks.workunit.client.0.smithi077.stdout:3/198: dwrite d0/d7/f34 [0,4194304] 0 2024-04-26T04:52:56.559 INFO:tasks.workunit.client.0.smithi077.stdout:3/199: dread d0/f1b [0,4194304] 0 2024-04-26T04:52:56.561 INFO:tasks.workunit.client.0.smithi077.stdout:3/200: truncate d0/d7/d15/f37 235168 0 2024-04-26T04:52:56.563 INFO:tasks.workunit.client.0.smithi077.stdout:3/201: creat d0/d7/d15/f3f x:0 0 0 2024-04-26T04:52:56.572 INFO:tasks.workunit.client.1.smithi078.stdout:1/578: sync 2024-04-26T04:52:56.577 INFO:tasks.workunit.client.1.smithi078.stdout:4/579: symlink d0/d25/d20/d31/d47/db2/d44/ld6 0 2024-04-26T04:52:56.578 INFO:tasks.workunit.client.1.smithi078.stdout:3/567: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1 0 2024-04-26T04:52:56.578 INFO:tasks.workunit.client.1.smithi078.stdout:5/472: dread d7/d18/d32/d37/f41 [0,4194304] 0 2024-04-26T04:52:56.579 INFO:tasks.workunit.client.0.smithi077.stdout:3/202: dread d0/d9/d32/f2f [0,4194304] 0 2024-04-26T04:52:56.581 INFO:tasks.workunit.client.0.smithi077.stdout:3/203: creat d0/d9/d32/d2a/f40 x:0 0 0 2024-04-26T04:52:56.582 INFO:tasks.workunit.client.1.smithi078.stdout:4/580: chown d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/fbe 523061 1 2024-04-26T04:52:56.585 INFO:tasks.workunit.client.1.smithi078.stdout:2/518: dwrite d8/f1f [0,4194304] 0 2024-04-26T04:52:56.585 INFO:tasks.workunit.client.1.smithi078.stdout:8/528: mkdir d0/d19/d25/d32/d8a/d92/da4/da7 0 2024-04-26T04:52:56.586 INFO:tasks.workunit.client.1.smithi078.stdout:5/473: creat d7/d18/d1c/d5a/f9d x:0 0 0 2024-04-26T04:52:56.586 INFO:tasks.workunit.client.1.smithi078.stdout:0/611: dwrite d2/f70 [0,4194304] 0 2024-04-26T04:52:56.586 INFO:tasks.workunit.client.1.smithi078.stdout:6/625: rename d2/de/d27/da9/d2a/l25 to d2/de/d27/da9/d3a/d69/daa/lbe 0 2024-04-26T04:52:56.592 INFO:tasks.workunit.client.0.smithi077.stdout:6/211: sync 2024-04-26T04:52:56.593 INFO:tasks.workunit.client.1.smithi078.stdout:3/568: dwrite d16/d26/d2e/d74/f97 [0,4194304] 0 2024-04-26T04:52:56.597 INFO:tasks.workunit.client.0.smithi077.stdout:6/212: dread d8/d1c/f28 [0,4194304] 0 2024-04-26T04:52:56.600 INFO:tasks.workunit.client.1.smithi078.stdout:5/474: creat d7/d18/d1c/f9e x:0 0 0 2024-04-26T04:52:56.600 INFO:tasks.workunit.client.1.smithi078.stdout:8/529: dwrite d0/d19/d25/f72 [0,4194304] 0 2024-04-26T04:52:56.600 INFO:tasks.workunit.client.1.smithi078.stdout:5/475: chown f3 917 1 2024-04-26T04:52:56.601 INFO:tasks.workunit.client.1.smithi078.stdout:2/519: unlink d8/d11/d18/fb1 0 2024-04-26T04:52:56.602 INFO:tasks.workunit.client.1.smithi078.stdout:8/530: write d0/d19/d25/f72 [2765510,1262] 0 2024-04-26T04:52:56.602 INFO:tasks.workunit.client.1.smithi078.stdout:0/612: mknod d2/d9/ccd 0 2024-04-26T04:52:56.604 INFO:tasks.workunit.client.1.smithi078.stdout:1/579: link d7/d1e/d9e/db/d37/d58/d5c/d68/c8d d7/d1e/d9e/cbe 0 2024-04-26T04:52:56.604 INFO:tasks.workunit.client.1.smithi078.stdout:3/569: creat d16/d26/d2b/d66/fc2 x:0 0 0 2024-04-26T04:52:56.605 INFO:tasks.workunit.client.1.smithi078.stdout:4/581: dwrite d0/d25/d20/f4f [0,4194304] 0 2024-04-26T04:52:56.605 INFO:tasks.workunit.client.1.smithi078.stdout:5/476: fdatasync f3 0 2024-04-26T04:52:56.605 INFO:tasks.workunit.client.1.smithi078.stdout:0/613: fdatasync d2/fc7 0 2024-04-26T04:52:56.607 INFO:tasks.workunit.client.1.smithi078.stdout:2/520: rmdir d8/d11/d14/d69/d8f/d34 39 2024-04-26T04:52:56.609 INFO:tasks.workunit.client.1.smithi078.stdout:4/582: readlink d0/d25/d20/l18 0 2024-04-26T04:52:56.610 INFO:tasks.workunit.client.1.smithi078.stdout:6/626: rename d2/de/d27/d4c to d2/de/d27/da9/d2a/dbf 0 2024-04-26T04:52:56.648 INFO:tasks.workunit.client.1.smithi078.stdout:1/580: creat d7/fbf x:0 0 0 2024-04-26T04:52:56.648 INFO:tasks.workunit.client.1.smithi078.stdout:6/627: creat d2/de/d27/da9/da/d36/db5/fc0 x:0 0 0 2024-04-26T04:52:56.650 INFO:tasks.workunit.client.1.smithi078.stdout:3/570: link d16/d26/d2b/d54/d5a/f82 d16/d26/d2b/d31/d71/d8a/fc3 0 2024-04-26T04:52:56.651 INFO:tasks.workunit.client.1.smithi078.stdout:3/571: stat d16/d26/d2b/d54/f6c 0 2024-04-26T04:52:56.653 INFO:tasks.workunit.client.1.smithi078.stdout:5/477: rename d7/c9 to d7/d21/c9f 0 2024-04-26T04:52:56.653 INFO:tasks.workunit.client.1.smithi078.stdout:5/478: readlink d7/d48/d57/d65/d46/l49 0 2024-04-26T04:52:56.654 INFO:tasks.workunit.client.1.smithi078.stdout:7/528: sync 2024-04-26T04:52:56.654 INFO:tasks.workunit.client.1.smithi078.stdout:9/489: sync 2024-04-26T04:52:56.654 INFO:tasks.workunit.client.1.smithi078.stdout:2/521: link d8/d11/d14/d69/d8f/f5c d8/d11/d18/d5d/d61/fb8 0 2024-04-26T04:52:56.655 INFO:tasks.workunit.client.1.smithi078.stdout:9/490: chown dc/df/d17 11 1 2024-04-26T04:52:56.655 INFO:tasks.workunit.client.1.smithi078.stdout:2/522: write d8/d53/f7a [707771,13840] 0 2024-04-26T04:52:56.656 INFO:tasks.workunit.client.1.smithi078.stdout:0/614: rename d2/d39/f42 to d2/d9/d31/fce 0 2024-04-26T04:52:56.656 INFO:tasks.workunit.client.1.smithi078.stdout:3/572: read d16/d26/d2b/d54/d56/f7d [554281,39105] 0 2024-04-26T04:52:56.657 INFO:tasks.workunit.client.1.smithi078.stdout:2/523: chown d8/d11/d6e/cae 623789 1 2024-04-26T04:52:56.657 INFO:tasks.workunit.client.1.smithi078.stdout:7/529: dread - d9/da/d33/f56 zero size 2024-04-26T04:52:56.664 INFO:tasks.workunit.client.1.smithi078.stdout:5/479: write d7/f8 [4436191,84250] 0 2024-04-26T04:52:56.664 INFO:tasks.workunit.client.1.smithi078.stdout:1/581: getdents d7/d1e/d9e/db/d37/d58/d5c/d68 0 2024-04-26T04:52:56.670 INFO:tasks.workunit.client.1.smithi078.stdout:1/582: write d7/d1e/d9e/dd/d20/f7b [87490,57185] 0 2024-04-26T04:52:56.670 INFO:tasks.workunit.client.1.smithi078.stdout:2/524: mknod d8/d11/d18/d5d/d8b/db0/db4/cb9 0 2024-04-26T04:52:56.670 INFO:tasks.workunit.client.1.smithi078.stdout:9/491: dwrite dc/df/d27/f76 [0,4194304] 0 2024-04-26T04:52:56.672 INFO:tasks.workunit.client.1.smithi078.stdout:5/480: mknod d7/d18/d32/d37/d79/ca0 0 2024-04-26T04:52:56.672 INFO:tasks.workunit.client.1.smithi078.stdout:2/525: write d8/f25 [5068091,58682] 0 2024-04-26T04:52:56.673 INFO:tasks.workunit.client.1.smithi078.stdout:7/530: mkdir d9/da/d11/d13/d1a/d52/d5a/d28/dab 0 2024-04-26T04:52:56.673 INFO:tasks.workunit.client.1.smithi078.stdout:1/583: chown d7/d1e/d9e/db/d37/d58 1087 1 2024-04-26T04:52:56.674 INFO:tasks.workunit.client.1.smithi078.stdout:3/573: mknod d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/cc4 0 2024-04-26T04:52:56.674 INFO:tasks.workunit.client.1.smithi078.stdout:1/584: readlink d7/d1e/d9e/dd/d20/l29 0 2024-04-26T04:52:56.675 INFO:tasks.workunit.client.0.smithi077.stdout:6/213: write f4 [395159,25735] 0 2024-04-26T04:52:56.676 INFO:tasks.workunit.client.0.smithi077.stdout:6/214: write d8/d2a/f2e [1641779,124319] 0 2024-04-26T04:52:56.679 INFO:tasks.workunit.client.1.smithi078.stdout:5/481: creat d7/d18/d32/d37/d79/fa1 x:0 0 0 2024-04-26T04:52:56.680 INFO:tasks.workunit.client.1.smithi078.stdout:5/482: fsync d7/d18/d32/d37/f6d 0 2024-04-26T04:52:56.683 INFO:tasks.workunit.client.1.smithi078.stdout:7/531: creat d9/da/d33/d4b/d4c/d5b/d83/fac x:0 0 0 2024-04-26T04:52:56.683 INFO:tasks.workunit.client.1.smithi078.stdout:8/531: truncate d0/d19/f2a 3295230 0 2024-04-26T04:52:56.685 INFO:tasks.workunit.client.0.smithi077.stdout:6/215: unlink d8/d1c/l46 0 2024-04-26T04:52:56.685 INFO:tasks.workunit.client.1.smithi078.stdout:4/583: write d0/de/f1e [3215730,129678] 0 2024-04-26T04:52:56.689 INFO:tasks.workunit.client.0.smithi077.stdout:6/216: mkdir d8/d1c/d4a 0 2024-04-26T04:52:56.689 INFO:tasks.workunit.client.1.smithi078.stdout:7/532: dwrite d9/da/d33/d4b/d4c/d6c/f81 [0,4194304] 0 2024-04-26T04:52:56.693 INFO:tasks.workunit.client.1.smithi078.stdout:1/585: rename d7/d1e/d9e/d17 to d7/d1e/d9e/db/d37/d58/d5c/dc0 0 2024-04-26T04:52:56.694 INFO:tasks.workunit.client.1.smithi078.stdout:0/615: dread d2/d9/d18/fd [0,4194304] 0 2024-04-26T04:52:56.695 INFO:tasks.workunit.client.1.smithi078.stdout:0/616: dread - d2/d9/d31/d4a/fb9 zero size 2024-04-26T04:52:56.696 INFO:tasks.workunit.client.1.smithi078.stdout:9/492: creat dc/f94 x:0 0 0 2024-04-26T04:52:56.698 INFO:tasks.workunit.client.0.smithi077.stdout:6/217: fsync d8/f37 0 2024-04-26T04:52:56.699 INFO:tasks.workunit.client.1.smithi078.stdout:8/532: readlink d0/d19/d25/l2e 0 2024-04-26T04:52:56.703 INFO:tasks.workunit.client.1.smithi078.stdout:4/584: dread - d0/d25/d20/d31/d47/db2/d44/d55/f95 zero size 2024-04-26T04:52:56.705 INFO:tasks.workunit.client.0.smithi077.stdout:6/218: mknod d8/c4b 0 2024-04-26T04:52:56.707 INFO:tasks.workunit.client.1.smithi078.stdout:2/526: creat d8/d11/d14/d69/d8f/fba x:0 0 0 2024-04-26T04:52:56.709 INFO:tasks.workunit.client.1.smithi078.stdout:5/483: rename d7/d18/d32/d37/d8f/c61 to d7/d48/d57/d65/d70/d72/d85/ca2 0 2024-04-26T04:52:56.711 INFO:tasks.workunit.client.0.smithi077.stdout:6/219: creat d8/d1c/f4c x:0 0 0 2024-04-26T04:52:56.712 INFO:tasks.workunit.client.1.smithi078.stdout:1/586: creat d7/d1e/d9e/db/d49/dba/d87/fc1 x:0 0 0 2024-04-26T04:52:56.713 INFO:tasks.workunit.client.1.smithi078.stdout:0/617: creat d2/d9/d18/d7d/fcf x:0 0 0 2024-04-26T04:52:56.714 INFO:tasks.workunit.client.0.smithi077.stdout:6/220: chown d8/d2a/l47 630721 1 2024-04-26T04:52:56.716 INFO:tasks.workunit.client.1.smithi078.stdout:8/533: mkdir d0/d19/da8 0 2024-04-26T04:52:56.717 INFO:tasks.workunit.client.1.smithi078.stdout:3/574: write d16/d26/d2e/f43 [432088,36095] 0 2024-04-26T04:52:56.719 INFO:tasks.workunit.client.0.smithi077.stdout:6/221: creat d8/d2a/d38/f4d x:0 0 0 2024-04-26T04:52:56.721 INFO:tasks.workunit.client.1.smithi078.stdout:4/585: fsync d0/d25/d20/d31/f6f 0 2024-04-26T04:52:56.726 INFO:tasks.workunit.client.1.smithi078.stdout:2/527: mkdir d8/d11/d18/dbb 0 2024-04-26T04:52:56.729 INFO:tasks.workunit.client.1.smithi078.stdout:9/493: rename dc/d55/f82 to dc/df/d4b/d3e/d80/f95 0 2024-04-26T04:52:56.731 INFO:tasks.workunit.client.1.smithi078.stdout:8/534: mknod d0/d19/d25/d32/d8a/d92/ca9 0 2024-04-26T04:52:56.731 INFO:tasks.workunit.client.1.smithi078.stdout:8/535: fsync d0/d19/d25/f87 0 2024-04-26T04:52:56.732 INFO:tasks.workunit.client.1.smithi078.stdout:7/533: creat d9/da/d11/fad x:0 0 0 2024-04-26T04:52:56.734 INFO:tasks.workunit.client.1.smithi078.stdout:4/586: mkdir d0/d25/d20/d31/d47/db2/d4c/d62/dd7 0 2024-04-26T04:52:56.737 INFO:tasks.workunit.client.1.smithi078.stdout:1/587: symlink d7/d1e/d9e/db/d37/d58/d86/dab/lc2 0 2024-04-26T04:52:56.738 INFO:tasks.workunit.client.1.smithi078.stdout:9/494: creat dc/df/d4b/d22/d2a/d57/d69/f96 x:0 0 0 2024-04-26T04:52:56.738 INFO:tasks.workunit.client.1.smithi078.stdout:7/534: truncate d9/da/d33/f56 455503 0 2024-04-26T04:52:56.739 INFO:tasks.workunit.client.1.smithi078.stdout:2/528: symlink d8/d11/d18/dbb/lbc 0 2024-04-26T04:52:56.739 INFO:tasks.workunit.client.1.smithi078.stdout:9/495: chown dc/df/d4b/d22/l6d 58 1 2024-04-26T04:52:56.741 INFO:tasks.workunit.client.0.smithi077.stdout:6/222: dwrite d8/f37 [0,4194304] 0 2024-04-26T04:52:56.741 INFO:tasks.workunit.client.1.smithi078.stdout:5/484: creat d7/d48/fa3 x:0 0 0 2024-04-26T04:52:56.742 INFO:tasks.workunit.client.0.smithi077.stdout:6/223: stat d8/f3f 0 2024-04-26T04:52:56.742 INFO:tasks.workunit.client.1.smithi078.stdout:5/485: fdatasync d7/d18/d1c/d5a/f7c 0 2024-04-26T04:52:56.743 INFO:tasks.workunit.client.1.smithi078.stdout:4/587: mkdir d0/d25/dd8 0 2024-04-26T04:52:56.744 INFO:tasks.workunit.client.1.smithi078.stdout:9/496: dwrite dc/d55/f52 [0,4194304] 0 2024-04-26T04:52:56.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/529: creat d8/fbd x:0 0 0 2024-04-26T04:52:56.746 INFO:tasks.workunit.client.1.smithi078.stdout:9/497: dread - dc/df/d27/d81/f87 zero size 2024-04-26T04:52:56.748 INFO:tasks.workunit.client.1.smithi078.stdout:7/535: mknod d9/da/d11/d13/d1a/d52/cae 0 2024-04-26T04:52:56.748 INFO:tasks.workunit.client.0.smithi077.stdout:6/224: read d8/f39 [1772828,70404] 0 2024-04-26T04:52:56.751 INFO:tasks.workunit.client.1.smithi078.stdout:3/575: rename d16/d26/d2b/d66/f6f to d16/d26/d2b/d31/fc5 0 2024-04-26T04:52:56.762 INFO:tasks.workunit.client.1.smithi078.stdout:4/588: creat d0/d25/d20/d31/d47/db2/d4c/d8e/fd9 x:0 0 0 2024-04-26T04:52:56.764 INFO:tasks.workunit.client.1.smithi078.stdout:7/536: rmdir d9/da/d33 39 2024-04-26T04:52:56.765 INFO:tasks.workunit.client.1.smithi078.stdout:7/537: dread - d9/da/d11/fad zero size 2024-04-26T04:52:56.766 INFO:tasks.workunit.client.1.smithi078.stdout:3/576: unlink d16/d26/d2b/d54/d5a/f60 0 2024-04-26T04:52:56.767 INFO:tasks.workunit.client.1.smithi078.stdout:4/589: symlink d0/de/d22/d71/dae/lda 0 2024-04-26T04:52:56.768 INFO:tasks.workunit.client.1.smithi078.stdout:1/588: getdents d7/d2e 0 2024-04-26T04:52:56.769 INFO:tasks.workunit.client.1.smithi078.stdout:1/589: chown d7/d1e/d9e/db/d49/ca7 6194 1 2024-04-26T04:52:56.770 INFO:tasks.workunit.client.1.smithi078.stdout:4/590: readlink d0/de/l63 0 2024-04-26T04:52:56.772 INFO:tasks.workunit.client.1.smithi078.stdout:7/538: rename d9/da/d11/d13/d1a/d52/d5a/d28/f98 to d9/da/faf 0 2024-04-26T04:52:56.773 INFO:tasks.workunit.client.1.smithi078.stdout:1/590: mknod d7/d1e/d9e/db/d47/cc3 0 2024-04-26T04:52:56.777 INFO:tasks.workunit.client.1.smithi078.stdout:1/591: fdatasync d7/d1e/f54 0 2024-04-26T04:52:56.781 INFO:tasks.workunit.client.1.smithi078.stdout:7/539: rename d9/da/d33/d4b/d4c/d5b/d83/f97 to d9/da/d33/fb0 0 2024-04-26T04:52:56.782 INFO:tasks.workunit.client.1.smithi078.stdout:5/486: write d7/d48/d57/d65/f3a [1717115,4161] 0 2024-04-26T04:52:56.783 INFO:tasks.workunit.client.1.smithi078.stdout:5/487: truncate d7/d48/fa3 834374 0 2024-04-26T04:52:56.784 INFO:tasks.workunit.client.1.smithi078.stdout:9/498: write dc/df/d27/f13 [3205671,107340] 0 2024-04-26T04:52:56.785 INFO:tasks.workunit.client.1.smithi078.stdout:2/530: write d8/d11/d18/d5d/d61/f9b [722564,6685] 0 2024-04-26T04:52:56.785 INFO:tasks.workunit.client.1.smithi078.stdout:9/499: write dc/df/d27/f76 [2737483,16201] 0 2024-04-26T04:52:56.787 INFO:tasks.workunit.client.0.smithi077.stdout:6/225: dwrite d8/d1c/f28 [0,4194304] 0 2024-04-26T04:52:56.788 INFO:tasks.workunit.client.1.smithi078.stdout:5/488: mkdir d7/da/d1e/da4 0 2024-04-26T04:52:56.789 INFO:tasks.workunit.client.1.smithi078.stdout:9/500: dread dc/d55/f52 [0,4194304] 0 2024-04-26T04:52:56.790 INFO:tasks.workunit.client.0.smithi077.stdout:6/226: fdatasync d8/d2a/f2e 0 2024-04-26T04:52:56.790 INFO:tasks.workunit.client.1.smithi078.stdout:9/501: chown dc/df/d4b/f1c 531519594 1 2024-04-26T04:52:56.792 INFO:tasks.workunit.client.1.smithi078.stdout:3/577: write d16/d26/d2b/d54/f6c [244118,124695] 0 2024-04-26T04:52:56.798 INFO:tasks.workunit.client.0.smithi077.stdout:6/227: unlink d8/d2a/f40 0 2024-04-26T04:52:56.799 INFO:tasks.workunit.client.1.smithi078.stdout:1/592: link d7/d1e/d9e/dd/lad d7/lc4 0 2024-04-26T04:52:56.802 INFO:tasks.workunit.client.0.smithi077.stdout:6/228: symlink d8/d1c/l4e 0 2024-04-26T04:52:56.806 INFO:tasks.workunit.client.1.smithi078.stdout:9/502: unlink dc/df/d17/c25 0 2024-04-26T04:52:56.806 INFO:tasks.workunit.client.1.smithi078.stdout:6/628: sync 2024-04-26T04:52:56.807 INFO:tasks.workunit.client.1.smithi078.stdout:7/540: getdents d9/da/d33/d4b/d4c/d5b 0 2024-04-26T04:52:56.807 INFO:tasks.workunit.client.1.smithi078.stdout:6/629: stat d2/d1a/l5d 0 2024-04-26T04:52:56.807 INFO:tasks.workunit.client.1.smithi078.stdout:9/503: truncate dc/df/d4b/d3e/d80/f95 609746 0 2024-04-26T04:52:56.807 INFO:tasks.workunit.client.1.smithi078.stdout:3/578: creat d16/d26/d2e/d5f/d49/d77/fc6 x:0 0 0 2024-04-26T04:52:56.808 INFO:tasks.workunit.client.0.smithi077.stdout:6/229: dwrite d8/f43 [0,4194304] 0 2024-04-26T04:52:56.816 INFO:tasks.workunit.client.1.smithi078.stdout:5/489: truncate d7/f35 1273310 0 2024-04-26T04:52:56.817 INFO:tasks.workunit.client.1.smithi078.stdout:5/490: chown d7/d48/d57/f89 571127 1 2024-04-26T04:52:56.821 INFO:tasks.workunit.client.1.smithi078.stdout:6/630: mknod d2/de/d27/da9/da/d36/cc1 0 2024-04-26T04:52:56.824 INFO:tasks.workunit.client.1.smithi078.stdout:2/531: creat d8/d11/fbe x:0 0 0 2024-04-26T04:52:56.825 INFO:tasks.workunit.client.1.smithi078.stdout:2/532: stat d8/d11/d18/f47 0 2024-04-26T04:52:56.830 INFO:tasks.workunit.client.0.smithi077.stdout:6/230: write d8/fe [3326128,97907] 0 2024-04-26T04:52:56.834 INFO:tasks.workunit.client.1.smithi078.stdout:1/593: write d7/f13 [1643308,108950] 0 2024-04-26T04:52:56.834 INFO:tasks.workunit.client.0.smithi077.stdout:6/231: fsync d8/d2a/f2e 0 2024-04-26T04:52:56.837 INFO:tasks.workunit.client.1.smithi078.stdout:6/631: dread d2/de/d27/da9/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:52:56.838 INFO:tasks.workunit.client.1.smithi078.stdout:3/579: creat d16/d26/d2e/d5f/d49/d77/dba/fc7 x:0 0 0 2024-04-26T04:52:56.838 INFO:tasks.workunit.client.1.smithi078.stdout:6/632: read - d2/de/fac zero size 2024-04-26T04:52:56.840 INFO:tasks.workunit.client.1.smithi078.stdout:1/594: mkdir d7/d1e/d9e/db/d49/dba/dc5 0 2024-04-26T04:52:56.840 INFO:tasks.workunit.client.1.smithi078.stdout:7/541: dread d9/fe [0,4194304] 0 2024-04-26T04:52:56.841 INFO:tasks.workunit.client.1.smithi078.stdout:9/504: rmdir dc/df/d89/d8b 0 2024-04-26T04:52:56.841 INFO:tasks.workunit.client.1.smithi078.stdout:1/595: fsync d7/d1e/d9e/db/d37/f88 0 2024-04-26T04:52:56.842 INFO:tasks.workunit.client.1.smithi078.stdout:5/491: link d7/d18/d1c/c56 d7/d18/d32/d84/ca5 0 2024-04-26T04:52:56.842 INFO:tasks.workunit.client.1.smithi078.stdout:6/633: rmdir d2/de/d27/da9/d3a/d69/daa/dba 39 2024-04-26T04:52:56.843 INFO:tasks.workunit.client.1.smithi078.stdout:1/596: fdatasync d7/d1e/d9e/db/d49/fb7 0 2024-04-26T04:52:56.844 INFO:tasks.workunit.client.1.smithi078.stdout:2/533: creat d8/d11/d14/fbf x:0 0 0 2024-04-26T04:52:56.846 INFO:tasks.workunit.client.1.smithi078.stdout:5/492: read d7/da/f73 [1230615,129956] 0 2024-04-26T04:52:56.847 INFO:tasks.workunit.client.1.smithi078.stdout:9/505: dread dc/f26 [0,4194304] 0 2024-04-26T04:52:56.848 INFO:tasks.workunit.client.1.smithi078.stdout:7/542: symlink d9/da/d11/d13/d1a/d52/d5a/d28/d6d/lb1 0 2024-04-26T04:52:56.848 INFO:tasks.workunit.client.1.smithi078.stdout:5/493: write d7/d11/f47 [3778111,82898] 0 2024-04-26T04:52:56.852 INFO:tasks.workunit.client.0.smithi077.stdout:6/232: dread d8/d2a/f2e [0,4194304] 0 2024-04-26T04:52:56.853 INFO:tasks.workunit.client.1.smithi078.stdout:6/634: rename d2/de/d27/da9/d2a/d4d/f61 to d2/de/d27/da9/d11/fc2 0 2024-04-26T04:52:56.854 INFO:tasks.workunit.client.1.smithi078.stdout:6/635: readlink d2/de/d27/d58/lb4 0 2024-04-26T04:52:56.855 INFO:tasks.workunit.client.0.smithi077.stdout:6/233: write d8/f33 [4143499,47268] 0 2024-04-26T04:52:56.855 INFO:tasks.workunit.client.1.smithi078.stdout:1/597: dread d7/d1e/d9e/dd/d20/fb3 [0,4194304] 0 2024-04-26T04:52:56.856 INFO:tasks.workunit.client.1.smithi078.stdout:3/580: dwrite d16/d26/d2b/f72 [0,4194304] 0 2024-04-26T04:52:56.859 INFO:tasks.workunit.client.0.smithi077.stdout:6/234: creat d8/d1c/f4f x:0 0 0 2024-04-26T04:52:56.863 INFO:tasks.workunit.client.1.smithi078.stdout:6/636: dwrite d2/de/d27/da9/d3a/d42/f84 [0,4194304] 0 2024-04-26T04:52:56.863 INFO:tasks.workunit.client.1.smithi078.stdout:7/543: mknod d9/da/d11/cb2 0 2024-04-26T04:52:56.864 INFO:tasks.workunit.client.1.smithi078.stdout:5/494: mknod d7/d48/ca6 0 2024-04-26T04:52:56.868 INFO:tasks.workunit.client.1.smithi078.stdout:3/581: creat d16/d26/d2e/d74/fc8 x:0 0 0 2024-04-26T04:52:56.868 INFO:tasks.workunit.client.0.smithi077.stdout:6/235: getdents d8/d2a 0 2024-04-26T04:52:56.869 INFO:tasks.workunit.client.0.smithi077.stdout:6/236: fdatasync d8/d1c/f3e 0 2024-04-26T04:52:56.872 INFO:tasks.workunit.client.1.smithi078.stdout:0/618: sync 2024-04-26T04:52:56.873 INFO:tasks.workunit.client.1.smithi078.stdout:5/495: mkdir d7/d18/d32/d37/d79/da7 0 2024-04-26T04:52:56.874 INFO:tasks.workunit.client.1.smithi078.stdout:5/496: dread - d7/d18/d1c/f7d zero size 2024-04-26T04:52:56.874 INFO:tasks.workunit.client.1.smithi078.stdout:9/506: creat dc/df/d4b/d22/f97 x:0 0 0 2024-04-26T04:52:56.876 INFO:tasks.workunit.client.0.smithi077.stdout:6/237: creat d8/f50 x:0 0 0 2024-04-26T04:52:56.876 INFO:tasks.workunit.client.1.smithi078.stdout:2/534: link d8/d11/d18/dbb/lbc d8/d11/d18/d5d/d61/lc0 0 2024-04-26T04:52:56.878 INFO:tasks.workunit.client.1.smithi078.stdout:7/544: mknod d9/cb3 0 2024-04-26T04:52:56.878 INFO:tasks.workunit.client.1.smithi078.stdout:9/507: dread dc/f26 [0,4194304] 0 2024-04-26T04:52:56.879 INFO:tasks.workunit.client.1.smithi078.stdout:0/619: mknod d2/d9/d31/cd0 0 2024-04-26T04:52:56.880 INFO:tasks.workunit.client.1.smithi078.stdout:0/620: chown d2/d9/d31/d73 812 1 2024-04-26T04:52:56.881 INFO:tasks.workunit.client.0.smithi077.stdout:6/238: dread d8/f37 [0,4194304] 0 2024-04-26T04:52:56.882 INFO:tasks.workunit.client.1.smithi078.stdout:1/598: rename d7/d1e/d9e/db/d37/c43 to d7/d1e/d9e/d4c/d3d/cc6 0 2024-04-26T04:52:56.882 INFO:tasks.workunit.client.0.smithi077.stdout:6/239: dread - d8/d1c/f4c zero size 2024-04-26T04:52:56.882 INFO:tasks.workunit.client.0.smithi077.stdout:6/240: readlink l6 0 2024-04-26T04:52:56.883 INFO:tasks.workunit.client.1.smithi078.stdout:5/497: mknod d7/d18/d32/d37/d79/ca8 0 2024-04-26T04:52:56.884 INFO:tasks.workunit.client.1.smithi078.stdout:3/582: truncate d16/d26/d2b/d54/d5a/f82 2337742 0 2024-04-26T04:52:56.887 INFO:tasks.workunit.client.1.smithi078.stdout:6/637: dwrite d2/de/d27/da9/f6b [0,4194304] 0 2024-04-26T04:52:56.888 INFO:tasks.workunit.client.1.smithi078.stdout:1/599: dread - d7/d1e/d9e/db/d49/dba/f83 zero size 2024-04-26T04:52:56.894 INFO:tasks.workunit.client.1.smithi078.stdout:6/638: mkdir d2/de/d27/da9/da/d36/db9/d97/dc3 0 2024-04-26T04:52:56.896 INFO:tasks.workunit.client.1.smithi078.stdout:5/498: mkdir d7/d48/da9 0 2024-04-26T04:52:56.897 INFO:tasks.workunit.client.0.smithi077.stdout:6/241: write d8/f37 [2053930,100157] 0 2024-04-26T04:52:56.897 INFO:tasks.workunit.client.1.smithi078.stdout:5/499: chown d7/d48/d57/d65/d46/c7f 0 1 2024-04-26T04:52:56.900 INFO:tasks.workunit.client.1.smithi078.stdout:9/508: mkdir dc/df/d4b/d22/d2d/d98 0 2024-04-26T04:52:56.900 INFO:tasks.workunit.client.1.smithi078.stdout:9/509: fdatasync f1 0 2024-04-26T04:52:56.901 INFO:tasks.workunit.client.0.smithi077.stdout:6/242: rmdir d8/d2a/d38 39 2024-04-26T04:52:56.902 INFO:tasks.workunit.client.1.smithi078.stdout:3/583: truncate d16/d26/d2b/d31/fc5 4981905 0 2024-04-26T04:52:56.904 INFO:tasks.workunit.client.1.smithi078.stdout:9/510: dread dc/df/d4b/d22/d2a/d92/f71 [0,4194304] 0 2024-04-26T04:52:56.905 INFO:tasks.workunit.client.1.smithi078.stdout:6/639: fdatasync d2/de/d27/da9/da/d36/db9/f47 0 2024-04-26T04:52:56.908 INFO:tasks.workunit.client.1.smithi078.stdout:5/500: creat d7/d48/d57/faa x:0 0 0 2024-04-26T04:52:56.909 INFO:tasks.workunit.client.0.smithi077.stdout:6/243: rmdir d8/d2a/d38 39 2024-04-26T04:52:56.912 INFO:tasks.workunit.client.1.smithi078.stdout:1/600: dwrite d7/d1e/d9e/db/d49/f84 [0,4194304] 0 2024-04-26T04:52:56.914 INFO:tasks.workunit.client.1.smithi078.stdout:2/535: link d8/d11/d14/f1c d8/db7/fc1 0 2024-04-26T04:52:56.916 INFO:tasks.workunit.client.0.smithi077.stdout:6/244: rename d8/d2a/d38/c3d to d8/d2a/d38/c51 0 2024-04-26T04:52:56.918 INFO:tasks.workunit.client.1.smithi078.stdout:9/511: mkdir dc/df/d4b/d22/d2a/d57/d99 0 2024-04-26T04:52:56.919 INFO:tasks.workunit.client.1.smithi078.stdout:3/584: write f9 [763872,31424] 0 2024-04-26T04:52:56.920 INFO:tasks.workunit.client.0.smithi077.stdout:6/245: getdents d8/d1c/d4a 0 2024-04-26T04:52:56.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/501: rename d7/da/f24 to d7/d48/d57/fab 0 2024-04-26T04:52:56.925 INFO:tasks.workunit.client.1.smithi078.stdout:2/536: creat d8/d11/d6e/fc2 x:0 0 0 2024-04-26T04:52:56.926 INFO:tasks.workunit.client.0.smithi077.stdout:6/246: rmdir d8/d1c/d4a 0 2024-04-26T04:52:56.928 INFO:tasks.workunit.client.1.smithi078.stdout:0/621: link d2/d9/d18/f13 d2/d9/d18/db7/fd1 0 2024-04-26T04:52:56.929 INFO:tasks.workunit.client.0.smithi077.stdout:9/168: sync 2024-04-26T04:52:56.929 INFO:tasks.workunit.client.0.smithi077.stdout:2/197: sync 2024-04-26T04:52:56.929 INFO:tasks.workunit.client.1.smithi078.stdout:9/512: symlink dc/df/d27/l9a 0 2024-04-26T04:52:56.930 INFO:tasks.workunit.client.1.smithi078.stdout:2/537: dwrite d8/d11/d18/d5d/f8a [0,4194304] 0 2024-04-26T04:52:56.931 INFO:tasks.workunit.client.0.smithi077.stdout:2/198: write d9/d15/f2f [1248573,109320] 0 2024-04-26T04:52:56.931 INFO:tasks.workunit.client.1.smithi078.stdout:2/538: dread - d8/d11/d14/fbf zero size 2024-04-26T04:52:56.932 INFO:tasks.workunit.client.0.smithi077.stdout:2/199: dread - d9/d15/f37 zero size 2024-04-26T04:52:56.933 INFO:tasks.workunit.client.0.smithi077.stdout:6/247: dwrite d8/f39 [4194304,4194304] 0 2024-04-26T04:52:56.936 INFO:tasks.workunit.client.1.smithi078.stdout:6/640: mkdir d2/de/d27/da9/d3a/d69/daa/dba/dc4 0 2024-04-26T04:52:56.937 INFO:tasks.workunit.client.1.smithi078.stdout:3/585: mknod d16/d9b/cc9 0 2024-04-26T04:52:56.938 INFO:tasks.workunit.client.1.smithi078.stdout:6/641: chown d2/de/d27/d58/d5b/c8a 1 1 2024-04-26T04:52:56.938 INFO:tasks.workunit.client.1.smithi078.stdout:9/513: dwrite dc/df/d4b/d22/d2a/f7d [0,4194304] 0 2024-04-26T04:52:56.940 INFO:tasks.workunit.client.0.smithi077.stdout:9/169: creat d9/dd/d3d/f3e x:0 0 0 2024-04-26T04:52:56.940 INFO:tasks.workunit.client.1.smithi078.stdout:1/601: symlink d7/d1e/d9e/db/d49/dba/dc5/lc7 0 2024-04-26T04:52:56.942 INFO:tasks.workunit.client.1.smithi078.stdout:1/602: chown d7/d1e/d9e/db/d47/f90 6154806 1 2024-04-26T04:52:56.946 INFO:tasks.workunit.client.1.smithi078.stdout:6/642: dwrite d2/de/d27/f37 [0,4194304] 0 2024-04-26T04:52:56.946 INFO:tasks.workunit.client.1.smithi078.stdout:8/536: sync 2024-04-26T04:52:56.946 INFO:tasks.workunit.client.1.smithi078.stdout:4/591: sync 2024-04-26T04:52:56.946 INFO:tasks.workunit.client.0.smithi077.stdout:2/200: rename d9/dc/f3a to d9/d15/d1a/d2b/f46 0 2024-04-26T04:52:56.948 INFO:tasks.workunit.client.1.smithi078.stdout:6/643: dread d2/de/f8d [0,4194304] 0 2024-04-26T04:52:56.948 INFO:tasks.workunit.client.1.smithi078.stdout:8/537: chown d0/d19/d25/d36/c98 64406132 1 2024-04-26T04:52:56.950 INFO:tasks.workunit.client.1.smithi078.stdout:9/514: creat dc/df/d27/f9b x:0 0 0 2024-04-26T04:52:56.953 INFO:tasks.workunit.client.1.smithi078.stdout:1/603: creat d7/d1e/d9e/db/d49/dba/fc8 x:0 0 0 2024-04-26T04:52:56.954 INFO:tasks.workunit.client.0.smithi077.stdout:9/170: rename d9/dd/d1b/l25 to d9/dd/d1b/d2b/d32/d36/l3f 0 2024-04-26T04:52:56.955 INFO:tasks.workunit.client.1.smithi078.stdout:1/604: write d7/d1e/d9e/d4c/d3d/f91 [4863986,23249] 0 2024-04-26T04:52:56.957 INFO:tasks.workunit.client.1.smithi078.stdout:4/592: creat d0/de/d22/d27/d7b/fdb x:0 0 0 2024-04-26T04:52:56.958 INFO:tasks.workunit.client.1.smithi078.stdout:6/644: unlink d2/d1a/d38/c95 0 2024-04-26T04:52:56.959 INFO:tasks.workunit.client.1.smithi078.stdout:9/515: dwrite dc/df/d27/d81/f91 [0,4194304] 0 2024-04-26T04:52:56.961 INFO:tasks.workunit.client.1.smithi078.stdout:3/586: link d16/d26/d2e/d5f/d49/f4b d16/d26/d2b/d31/d71/d8a/fca 0 2024-04-26T04:52:56.962 INFO:tasks.workunit.client.0.smithi077.stdout:2/201: creat d9/dc/d1d/f47 x:0 0 0 2024-04-26T04:52:56.963 INFO:tasks.workunit.client.0.smithi077.stdout:2/202: write d9/fd [4208177,12285] 0 2024-04-26T04:52:56.964 INFO:tasks.workunit.client.1.smithi078.stdout:2/539: dread d8/db7/fc1 [0,4194304] 0 2024-04-26T04:52:56.965 INFO:tasks.workunit.client.0.smithi077.stdout:2/203: truncate d9/d15/d1a/d2b/f42 77769 0 2024-04-26T04:52:56.966 INFO:tasks.workunit.client.1.smithi078.stdout:1/605: rename d7/d1e/d9e/db/d47/cc3 to d7/d1e/d9e/dd/d20/d53/cc9 0 2024-04-26T04:52:56.966 INFO:tasks.workunit.client.0.smithi077.stdout:2/204: write d9/dc/d1d/d1e/f3b [132685,104305] 0 2024-04-26T04:52:56.968 INFO:tasks.workunit.client.1.smithi078.stdout:9/516: dwrite dc/df/f2f [0,4194304] 0 2024-04-26T04:52:56.971 INFO:tasks.workunit.client.0.smithi077.stdout:2/205: symlink d9/dc/d1d/d45/l48 0 2024-04-26T04:52:56.972 INFO:tasks.workunit.client.1.smithi078.stdout:6/645: link d2/de/d27/da9/d2a/dbf/d5f/laf d2/de/d27/da9/d3a/d42/lc5 0 2024-04-26T04:52:56.974 INFO:tasks.workunit.client.1.smithi078.stdout:1/606: dwrite d7/d1e/d9e/dd/fa8 [0,4194304] 0 2024-04-26T04:52:56.976 INFO:tasks.workunit.client.1.smithi078.stdout:3/587: rename d16/d3c/lb3 to d16/d26/d2b/lcb 0 2024-04-26T04:52:56.977 INFO:tasks.workunit.client.0.smithi077.stdout:2/206: mknod d9/dc/d1d/d1e/c49 0 2024-04-26T04:52:56.979 INFO:tasks.workunit.client.1.smithi078.stdout:1/607: mkdir d7/d1e/d9e/dd/d20/dca 0 2024-04-26T04:52:56.982 INFO:tasks.workunit.client.1.smithi078.stdout:3/588: mkdir d16/d26/d2e/d5f/d49/dcc 0 2024-04-26T04:52:56.984 INFO:tasks.workunit.client.0.smithi077.stdout:6/248: write d8/f27 [1266401,57123] 0 2024-04-26T04:52:56.986 INFO:tasks.workunit.client.1.smithi078.stdout:6/646: link d2/d1a/fa6 d2/de/d27/da9/d3a/d69/daa/fc6 0 2024-04-26T04:52:56.986 INFO:tasks.workunit.client.0.smithi077.stdout:2/207: creat d9/dc/d1d/f4a x:0 0 0 2024-04-26T04:52:56.989 INFO:tasks.workunit.client.1.smithi078.stdout:3/589: rename d16/d26/d2e/fa6 to d16/d26/d2b/d54/d56/da8/fcd 0 2024-04-26T04:52:56.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:56 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:56.994 INFO:tasks.workunit.client.0.smithi077.stdout:6/249: rename d8/d2a/d38 to d8/d52 0 2024-04-26T04:52:56.997 INFO:tasks.workunit.client.0.smithi077.stdout:9/171: write d9/dd/d1b/f23 [721747,101316] 0 2024-04-26T04:52:56.998 INFO:tasks.workunit.client.1.smithi078.stdout:6/647: getdents d2/de/d27/da9/d2a/dbf 0 2024-04-26T04:52:57.000 INFO:tasks.workunit.client.1.smithi078.stdout:6/648: truncate d2/de/d27/da9/da/f4a 1842741 0 2024-04-26T04:52:57.001 INFO:tasks.workunit.client.1.smithi078.stdout:9/517: dread dc/df/f1d [0,4194304] 0 2024-04-26T04:52:57.003 INFO:tasks.workunit.client.0.smithi077.stdout:6/250: dwrite d8/ff [0,4194304] 0 2024-04-26T04:52:57.003 INFO:tasks.workunit.client.0.smithi077.stdout:9/172: rename d9/c12 to d9/dd/d28/c40 0 2024-04-26T04:52:57.003 INFO:tasks.workunit.client.0.smithi077.stdout:6/251: read d8/f49 [790024,93152] 0 2024-04-26T04:52:57.006 INFO:tasks.workunit.client.1.smithi078.stdout:4/593: truncate d0/d25/d20/d31/fb9 2572620 0 2024-04-26T04:52:57.011 INFO:tasks.workunit.client.0.smithi077.stdout:6/252: dwrite d8/d1c/f28 [0,4194304] 0 2024-04-26T04:52:57.011 INFO:tasks.workunit.client.0.smithi077.stdout:6/253: chown d8/f22 105992544 1 2024-04-26T04:52:57.012 INFO:tasks.workunit.client.1.smithi078.stdout:6/649: unlink d2/de/d27/da9/da/d36/db9/c80 0 2024-04-26T04:52:57.014 INFO:tasks.workunit.client.1.smithi078.stdout:4/594: readlink d0/d25/d20/l8c 0 2024-04-26T04:52:57.015 INFO:tasks.workunit.client.1.smithi078.stdout:9/518: dwrite dc/df/d4b/d22/d2a/d57/f90 [4194304,4194304] 0 2024-04-26T04:52:57.016 INFO:tasks.workunit.client.1.smithi078.stdout:1/608: dread d7/d1e/d9e/d4c/f26 [0,4194304] 0 2024-04-26T04:52:57.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:56 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:57.019 INFO:tasks.workunit.client.1.smithi078.stdout:6/650: creat d2/de/d27/da9/da/fc7 x:0 0 0 2024-04-26T04:52:57.026 INFO:tasks.workunit.client.0.smithi077.stdout:6/254: dwrite d8/fb [0,4194304] 0 2024-04-26T04:52:57.027 INFO:tasks.workunit.client.1.smithi078.stdout:3/590: write d16/d26/d2b/d31/fc5 [3103369,115171] 0 2024-04-26T04:52:57.028 INFO:tasks.workunit.client.1.smithi078.stdout:4/595: mkdir d0/de/d22/d27/d56/d5a/ddc 0 2024-04-26T04:52:57.028 INFO:tasks.workunit.client.0.smithi077.stdout:6/255: write d8/fe [4795103,122739] 0 2024-04-26T04:52:57.033 INFO:tasks.workunit.client.1.smithi078.stdout:6/651: rmdir d2/de/d27/da9/d3a 39 2024-04-26T04:52:57.034 INFO:tasks.workunit.client.1.smithi078.stdout:7/545: sync 2024-04-26T04:52:57.036 INFO:tasks.workunit.client.0.smithi077.stdout:6/256: mkdir d8/d52/d53 0 2024-04-26T04:52:57.037 INFO:tasks.workunit.client.0.smithi077.stdout:9/173: getdents d9/d3a 0 2024-04-26T04:52:57.038 INFO:tasks.workunit.client.1.smithi078.stdout:3/591: symlink d16/d26/d2e/d5f/d49/dad/db0/lce 0 2024-04-26T04:52:57.042 INFO:tasks.workunit.client.0.smithi077.stdout:9/174: rename d9/c18 to d9/d35/c41 0 2024-04-26T04:52:57.042 INFO:tasks.workunit.client.0.smithi077.stdout:9/175: readlink d9/l1d 0 2024-04-26T04:52:57.043 INFO:tasks.workunit.client.0.smithi077.stdout:9/176: fdatasync f2 0 2024-04-26T04:52:57.043 INFO:tasks.workunit.client.0.smithi077.stdout:6/257: dread d8/d2a/f32 [0,4194304] 0 2024-04-26T04:52:57.043 INFO:tasks.workunit.client.1.smithi078.stdout:1/609: symlink d7/d1e/d9e/db/lcb 0 2024-04-26T04:52:57.045 INFO:tasks.workunit.client.1.smithi078.stdout:6/652: mkdir d2/de/d27/da9/da8/dc8 0 2024-04-26T04:52:57.046 INFO:tasks.workunit.client.1.smithi078.stdout:6/653: write d2/d52/fbd [818391,130578] 0 2024-04-26T04:52:57.046 INFO:tasks.workunit.client.1.smithi078.stdout:7/546: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4 0 2024-04-26T04:52:57.047 INFO:tasks.workunit.client.1.smithi078.stdout:9/519: unlink dc/df/d4b/d22/d2d/c43 0 2024-04-26T04:52:57.050 INFO:tasks.workunit.client.1.smithi078.stdout:3/592: unlink d16/f25 0 2024-04-26T04:52:57.051 INFO:tasks.workunit.client.1.smithi078.stdout:3/593: chown d16/d26/d2b/d31/f59 409068523 1 2024-04-26T04:52:57.053 INFO:tasks.workunit.client.1.smithi078.stdout:9/520: dwrite dc/f39 [0,4194304] 0 2024-04-26T04:52:57.054 INFO:tasks.workunit.client.1.smithi078.stdout:9/521: chown dc/df/d4b/d22/d2a/d57/l8e 0 1 2024-04-26T04:52:57.062 INFO:tasks.workunit.client.0.smithi077.stdout:6/258: getdents d8/d1c 0 2024-04-26T04:52:57.064 INFO:tasks.workunit.client.1.smithi078.stdout:3/594: dread d16/d26/d2b/d31/f89 [0,4194304] 0 2024-04-26T04:52:57.064 INFO:tasks.workunit.client.1.smithi078.stdout:4/596: truncate d0/d25/d20/d31/f73 1769145 0 2024-04-26T04:52:57.066 INFO:tasks.workunit.client.0.smithi077.stdout:9/177: dwrite d9/f10 [4194304,4194304] 0 2024-04-26T04:52:57.067 INFO:tasks.workunit.client.1.smithi078.stdout:9/522: symlink dc/d55/l9c 0 2024-04-26T04:52:57.068 INFO:tasks.workunit.client.1.smithi078.stdout:9/523: write dc/df/d4b/d3e/f5d [260950,680] 0 2024-04-26T04:52:57.069 INFO:tasks.workunit.client.0.smithi077.stdout:6/259: mknod d8/c54 0 2024-04-26T04:52:57.071 INFO:tasks.workunit.client.0.smithi077.stdout:9/178: creat d9/dd/f42 x:0 0 0 2024-04-26T04:52:57.074 INFO:tasks.workunit.client.0.smithi077.stdout:9/179: mkdir d9/d43 0 2024-04-26T04:52:57.075 INFO:tasks.workunit.client.0.smithi077.stdout:9/180: fdatasync d9/dd/d28/f33 0 2024-04-26T04:52:57.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/181: dread - d9/dd/d1b/d2b/f2f zero size 2024-04-26T04:52:57.080 INFO:tasks.workunit.client.0.smithi077.stdout:9/182: symlink d9/d3a/l44 0 2024-04-26T04:52:57.080 INFO:tasks.workunit.client.1.smithi078.stdout:4/597: mknod d0/d25/d20/d31/d47/db2/d4c/d62/dd7/cdd 0 2024-04-26T04:52:57.081 INFO:tasks.workunit.client.0.smithi077.stdout:9/183: read - d9/f2e zero size 2024-04-26T04:52:57.083 INFO:tasks.workunit.client.1.smithi078.stdout:3/595: creat d16/d26/d2e/d5f/fcf x:0 0 0 2024-04-26T04:52:57.083 INFO:tasks.workunit.client.1.smithi078.stdout:1/610: link d7/d1e/d9e/dd/d20/d53/l79 d7/d1e/d9e/db/d49/dba/lcc 0 2024-04-26T04:52:57.083 INFO:tasks.workunit.client.1.smithi078.stdout:4/598: write d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/fbe [129188,87911] 0 2024-04-26T04:52:57.084 INFO:tasks.workunit.client.1.smithi078.stdout:9/524: dread dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:52:57.085 INFO:tasks.workunit.client.1.smithi078.stdout:4/599: chown d0/d25/d20/d31 1414983 1 2024-04-26T04:52:57.086 INFO:tasks.workunit.client.1.smithi078.stdout:1/611: dread - d7/d1e/d9e/dd/d20/d53/f76 zero size 2024-04-26T04:52:57.086 INFO:tasks.workunit.client.1.smithi078.stdout:4/600: readlink d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/lb3 0 2024-04-26T04:52:57.087 INFO:tasks.workunit.client.0.smithi077.stdout:9/184: dwrite f3 [0,4194304] 0 2024-04-26T04:52:57.087 INFO:tasks.workunit.client.1.smithi078.stdout:3/596: mknod d16/cd0 0 2024-04-26T04:52:57.087 INFO:tasks.workunit.client.1.smithi078.stdout:4/601: fsync d0/d25/d20/d31/f6b 0 2024-04-26T04:52:57.088 INFO:tasks.workunit.client.1.smithi078.stdout:7/547: truncate f4 10377414 0 2024-04-26T04:52:57.088 INFO:tasks.workunit.client.1.smithi078.stdout:7/548: stat d9/da/d33/fb0 0 2024-04-26T04:52:57.089 INFO:tasks.workunit.client.1.smithi078.stdout:4/602: readlink d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/lb3 0 2024-04-26T04:52:57.090 INFO:tasks.workunit.client.1.smithi078.stdout:7/549: chown d9/da/d11/d13/d1a/d52/l5f 0 1 2024-04-26T04:52:57.090 INFO:tasks.workunit.client.1.smithi078.stdout:7/550: stat d9/da/d11/d13/d1a/d52/d5a/c35 0 2024-04-26T04:52:57.091 INFO:tasks.workunit.client.1.smithi078.stdout:7/551: write d9/da/d11/fad [953797,70159] 0 2024-04-26T04:52:57.092 INFO:tasks.workunit.client.0.smithi077.stdout:9/185: dread d9/f17 [4194304,4194304] 0 2024-04-26T04:52:57.092 INFO:tasks.workunit.client.0.smithi077.stdout:6/260: write d8/f22 [200507,48369] 0 2024-04-26T04:52:57.099 INFO:tasks.workunit.client.0.smithi077.stdout:9/186: rmdir d9/dd/d3d 39 2024-04-26T04:52:57.100 INFO:tasks.workunit.client.0.smithi077.stdout:9/187: chown d9/d35 182639 1 2024-04-26T04:52:57.100 INFO:tasks.workunit.client.1.smithi078.stdout:0/622: sync 2024-04-26T04:52:57.100 INFO:tasks.workunit.client.1.smithi078.stdout:5/502: sync 2024-04-26T04:52:57.100 INFO:tasks.workunit.client.1.smithi078.stdout:0/623: chown d2/d9/d31 118618 1 2024-04-26T04:52:57.101 INFO:tasks.workunit.client.1.smithi078.stdout:3/597: rename d16/d26/d2b/db6 to d16/d26/d2b/d31/d78/dd1 0 2024-04-26T04:52:57.102 INFO:tasks.workunit.client.1.smithi078.stdout:3/598: write f9 [1100985,71767] 0 2024-04-26T04:52:57.104 INFO:tasks.workunit.client.1.smithi078.stdout:4/603: creat d0/d25/d20/d31/d47/db2/d44/d69/fde x:0 0 0 2024-04-26T04:52:57.105 INFO:tasks.workunit.client.0.smithi077.stdout:9/188: dwrite d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:52:57.106 INFO:tasks.workunit.client.0.smithi077.stdout:9/189: write d9/dd/f30 [1013467,109758] 0 2024-04-26T04:52:57.109 INFO:tasks.workunit.client.1.smithi078.stdout:5/503: chown d7/d21/c9f 5880 1 2024-04-26T04:52:57.109 INFO:tasks.workunit.client.1.smithi078.stdout:9/525: dread dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:52:57.110 INFO:tasks.workunit.client.0.smithi077.stdout:6/261: mkdir d8/d55 0 2024-04-26T04:52:57.115 INFO:tasks.workunit.client.1.smithi078.stdout:9/526: dwrite dc/df/d27/f53 [0,4194304] 0 2024-04-26T04:52:57.115 INFO:tasks.workunit.client.0.smithi077.stdout:9/190: mkdir d9/d45 0 2024-04-26T04:52:57.117 INFO:tasks.workunit.client.0.smithi077.stdout:5/246: sync 2024-04-26T04:52:57.117 INFO:tasks.workunit.client.0.smithi077.stdout:1/334: sync 2024-04-26T04:52:57.120 INFO:tasks.workunit.client.1.smithi078.stdout:1/612: dwrite d7/d1e/f54 [0,4194304] 0 2024-04-26T04:52:57.123 INFO:tasks.workunit.client.0.smithi077.stdout:9/191: dwrite f3 [4194304,4194304] 0 2024-04-26T04:52:57.123 INFO:tasks.workunit.client.1.smithi078.stdout:7/552: rename d9/da/d20/f6f to d9/da/d11/d13/d1a/d52/fb5 0 2024-04-26T04:52:57.123 INFO:tasks.workunit.client.1.smithi078.stdout:9/527: dread dc/df/d4b/d22/d2a/d57/f90 [4194304,4194304] 0 2024-04-26T04:52:57.128 INFO:tasks.workunit.client.1.smithi078.stdout:4/604: symlink d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/ldf 0 2024-04-26T04:52:57.129 INFO:tasks.workunit.client.1.smithi078.stdout:5/504: stat d7/d18/d1c/f83 0 2024-04-26T04:52:57.130 INFO:tasks.workunit.client.1.smithi078.stdout:9/528: dread dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:52:57.130 INFO:tasks.workunit.client.1.smithi078.stdout:8/538: sync 2024-04-26T04:52:57.130 INFO:tasks.workunit.client.1.smithi078.stdout:6/654: sync 2024-04-26T04:52:57.130 INFO:tasks.workunit.client.1.smithi078.stdout:2/540: sync 2024-04-26T04:52:57.131 INFO:tasks.workunit.client.1.smithi078.stdout:5/505: read - d7/d18/d1c/f95 zero size 2024-04-26T04:52:57.132 INFO:tasks.workunit.client.1.smithi078.stdout:4/605: read - d0/d25/d20/d31/d47/db2/d4c/d8e/fd9 zero size 2024-04-26T04:52:57.132 INFO:tasks.workunit.client.0.smithi077.stdout:9/192: unlink d9/d3a/l3c 0 2024-04-26T04:52:57.133 INFO:tasks.workunit.client.1.smithi078.stdout:0/624: mknod d2/d9/cd2 0 2024-04-26T04:52:57.133 INFO:tasks.workunit.client.1.smithi078.stdout:4/606: fdatasync d0/de/d22/fb7 0 2024-04-26T04:52:57.136 INFO:tasks.workunit.client.0.smithi077.stdout:6/262: link d8/d52/c51 d8/d52/d53/c56 0 2024-04-26T04:52:57.138 INFO:tasks.workunit.client.0.smithi077.stdout:5/247: rename d1/d12/c5d to d1/d12/d21/d25/d31/c5f 0 2024-04-26T04:52:57.140 INFO:tasks.workunit.client.1.smithi078.stdout:4/607: dwrite d0/f93 [0,4194304] 0 2024-04-26T04:52:57.140 INFO:tasks.workunit.client.1.smithi078.stdout:3/599: truncate d16/f69 891738 0 2024-04-26T04:52:57.142 INFO:tasks.workunit.client.1.smithi078.stdout:9/529: rmdir dc/d55 39 2024-04-26T04:52:57.142 INFO:tasks.workunit.client.1.smithi078.stdout:3/600: write d16/d26/d2e/d74/f4f [1186239,117136] 0 2024-04-26T04:52:57.143 INFO:tasks.workunit.client.0.smithi077.stdout:6/263: mkdir d8/d2a/d57 0 2024-04-26T04:52:57.144 INFO:tasks.workunit.client.1.smithi078.stdout:9/530: read dc/df/f59 [3035436,117875] 0 2024-04-26T04:52:57.146 INFO:tasks.workunit.client.0.smithi077.stdout:1/335: write d9/f21 [8836950,31070] 0 2024-04-26T04:52:57.146 INFO:tasks.workunit.client.1.smithi078.stdout:2/541: read - d8/d11/d14/f8d zero size 2024-04-26T04:52:57.147 INFO:tasks.workunit.client.1.smithi078.stdout:6/655: symlink d2/d1a/d38/lc9 0 2024-04-26T04:52:57.147 INFO:tasks.workunit.client.1.smithi078.stdout:2/542: fdatasync d8/d11/d14/fbf 0 2024-04-26T04:52:57.150 INFO:tasks.workunit.client.0.smithi077.stdout:5/248: creat d1/d12/f60 x:0 0 0 2024-04-26T04:52:57.151 INFO:tasks.workunit.client.0.smithi077.stdout:5/249: truncate d1/d12/d45/f5e 797455 0 2024-04-26T04:52:57.153 INFO:tasks.workunit.client.0.smithi077.stdout:1/336: creat d9/db/d2b/f61 x:0 0 0 2024-04-26T04:52:57.153 INFO:tasks.workunit.client.0.smithi077.stdout:1/337: readlink d9/db/dd/d34/l35 0 2024-04-26T04:52:57.161 INFO:tasks.workunit.client.0.smithi077.stdout:6/264: creat d8/d2a/f58 x:0 0 0 2024-04-26T04:52:57.164 INFO:tasks.workunit.client.1.smithi078.stdout:1/613: truncate d7/d1e/d9e/dd/fa8 953180 0 2024-04-26T04:52:57.166 INFO:tasks.workunit.client.1.smithi078.stdout:2/543: dread d8/d3e/f43 [0,4194304] 0 2024-04-26T04:52:57.167 INFO:tasks.workunit.client.0.smithi077.stdout:5/250: truncate d1/d8/ff 1813707 0 2024-04-26T04:52:57.168 INFO:tasks.workunit.client.0.smithi077.stdout:5/251: write d1/d12/d21/f17 [2718590,98937] 0 2024-04-26T04:52:57.172 INFO:tasks.workunit.client.1.smithi078.stdout:4/608: mkdir d0/d25/d20/d31/d47/db2/d4c/d62/daa/d2e/de0 0 2024-04-26T04:52:57.172 INFO:tasks.workunit.client.0.smithi077.stdout:5/252: dread d1/d30/f53 [0,4194304] 0 2024-04-26T04:52:57.174 INFO:tasks.workunit.client.0.smithi077.stdout:5/253: truncate d1/f5 4324694 0 2024-04-26T04:52:57.174 INFO:tasks.workunit.client.1.smithi078.stdout:4/609: fdatasync d0/d25/d20/d31/d47/db2/d4c/dba/fc1 0 2024-04-26T04:52:57.174 INFO:tasks.workunit.client.1.smithi078.stdout:4/610: readlink d0/d25/d20/l8c 0 2024-04-26T04:52:57.175 INFO:tasks.workunit.client.0.smithi077.stdout:5/254: write d1/d12/d21/d25/d31/d3c/f4e [555398,31715] 0 2024-04-26T04:52:57.177 INFO:tasks.workunit.client.1.smithi078.stdout:3/601: symlink d16/d26/d2e/ld2 0 2024-04-26T04:52:57.178 INFO:tasks.workunit.client.0.smithi077.stdout:1/338: rmdir d9/db/d2b/d5e 0 2024-04-26T04:52:57.180 INFO:tasks.workunit.client.0.smithi077.stdout:6/265: rename d8/d2a/l30 to d8/d55/l59 0 2024-04-26T04:52:57.182 INFO:tasks.workunit.client.0.smithi077.stdout:5/255: symlink d1/d30/d35/l61 0 2024-04-26T04:52:57.183 INFO:tasks.workunit.client.0.smithi077.stdout:1/339: mknod d9/d53/c62 0 2024-04-26T04:52:57.187 INFO:tasks.workunit.client.1.smithi078.stdout:0/625: unlink d2/d9/d18/f62 0 2024-04-26T04:52:57.192 INFO:tasks.workunit.client.0.smithi077.stdout:5/256: mkdir d1/d42/d62 0 2024-04-26T04:52:57.194 INFO:tasks.workunit.client.0.smithi077.stdout:6/266: creat d8/f5a x:0 0 0 2024-04-26T04:52:57.194 INFO:tasks.workunit.client.1.smithi078.stdout:6/656: write d2/de/d27/da9/da/d36/db9/f47 [294566,59133] 0 2024-04-26T04:52:57.195 INFO:tasks.workunit.client.1.smithi078.stdout:6/657: chown d2/de/d27/da9/d2a/d4d 1581617 1 2024-04-26T04:52:57.196 INFO:tasks.workunit.client.0.smithi077.stdout:5/257: unlink d1/d12/d21/f41 0 2024-04-26T04:52:57.197 INFO:tasks.workunit.client.1.smithi078.stdout:5/506: write d7/da/f10 [3732329,69494] 0 2024-04-26T04:52:57.197 INFO:tasks.workunit.client.0.smithi077.stdout:5/258: fsync d1/d12/d21/d25/d31/d3c/f4e 0 2024-04-26T04:52:57.198 INFO:tasks.workunit.client.0.smithi077.stdout:6/267: mknod d8/d55/c5b 0 2024-04-26T04:52:57.199 INFO:tasks.workunit.client.1.smithi078.stdout:6/658: dread d2/de/d27/d58/f76 [0,4194304] 0 2024-04-26T04:52:57.200 INFO:tasks.workunit.client.1.smithi078.stdout:9/531: getdents dc/df/d4b/d22/d2a/d57/d93 0 2024-04-26T04:52:57.201 INFO:tasks.workunit.client.0.smithi077.stdout:5/259: creat d1/d12/d45/f63 x:0 0 0 2024-04-26T04:52:57.202 INFO:tasks.workunit.client.0.smithi077.stdout:5/260: dread - d1/d12/f1f zero size 2024-04-26T04:52:57.202 INFO:tasks.workunit.client.0.smithi077.stdout:6/268: creat d8/d1c/f5c x:0 0 0 2024-04-26T04:52:57.204 INFO:tasks.workunit.client.1.smithi078.stdout:1/614: dwrite d7/d1e/f7f [0,4194304] 0 2024-04-26T04:52:57.204 INFO:tasks.workunit.client.1.smithi078.stdout:8/539: rename d0/d19/d52/d35/d44/f59 to d0/d19/d52/faa 0 2024-04-26T04:52:57.205 INFO:tasks.workunit.client.0.smithi077.stdout:5/261: creat d1/d12/d21/f64 x:0 0 0 2024-04-26T04:52:57.205 INFO:tasks.workunit.client.1.smithi078.stdout:7/553: getdents d9/da/d33/d4b 0 2024-04-26T04:52:57.206 INFO:tasks.workunit.client.0.smithi077.stdout:5/262: mknod d1/d8/c65 0 2024-04-26T04:52:57.207 INFO:tasks.workunit.client.1.smithi078.stdout:2/544: read - d8/d11/d14/d69/d8f/d34/f51 zero size 2024-04-26T04:52:57.207 INFO:tasks.workunit.client.0.smithi077.stdout:6/269: mknod d8/d2a/c5d 0 2024-04-26T04:52:57.209 INFO:tasks.workunit.client.0.smithi077.stdout:5/263: mkdir d1/d30/d33/d66 0 2024-04-26T04:52:57.212 INFO:tasks.workunit.client.1.smithi078.stdout:6/659: creat d2/de/fca x:0 0 0 2024-04-26T04:52:57.212 INFO:tasks.workunit.client.1.smithi078.stdout:6/660: chown d2/de/fac 6 1 2024-04-26T04:52:57.214 INFO:tasks.workunit.client.1.smithi078.stdout:9/532: mknod dc/df/d4b/d22/d2a/d92/c9d 0 2024-04-26T04:52:57.215 INFO:tasks.workunit.client.0.smithi077.stdout:6/270: mkdir d8/d52/d53/d5e 0 2024-04-26T04:52:57.217 INFO:tasks.workunit.client.0.smithi077.stdout:5/264: mknod d1/d12/d28/d38/c67 0 2024-04-26T04:52:57.218 INFO:tasks.workunit.client.0.smithi077.stdout:5/265: fsync d1/d30/f36 0 2024-04-26T04:52:57.221 INFO:tasks.workunit.client.0.smithi077.stdout:6/271: unlink d8/d2a/f58 0 2024-04-26T04:52:57.221 INFO:tasks.workunit.client.0.smithi077.stdout:6/272: fsync f3 0 2024-04-26T04:52:57.222 INFO:tasks.workunit.client.0.smithi077.stdout:6/273: chown d8/d1c/f4f 1 1 2024-04-26T04:52:57.223 INFO:tasks.workunit.client.0.smithi077.stdout:6/274: write f4 [3226353,96825] 0 2024-04-26T04:52:57.224 INFO:tasks.workunit.client.0.smithi077.stdout:5/266: creat d1/d12/d21/d25/d31/d3c/f68 x:0 0 0 2024-04-26T04:52:57.225 INFO:tasks.workunit.client.0.smithi077.stdout:6/275: write d8/fe [2292617,1812] 0 2024-04-26T04:52:57.225 INFO:tasks.workunit.client.0.smithi077.stdout:6/276: stat d8/c16 0 2024-04-26T04:52:57.226 INFO:tasks.workunit.client.0.smithi077.stdout:5/267: write d1/d12/d21/d25/f4b [930461,85152] 0 2024-04-26T04:52:57.232 INFO:tasks.workunit.client.1.smithi078.stdout:4/611: rename d0/d25 to d0/d1d/de1 0 2024-04-26T04:52:57.235 INFO:tasks.workunit.client.1.smithi078.stdout:8/540: symlink d0/d19/d52/d35/d73/lab 0 2024-04-26T04:52:57.236 INFO:tasks.workunit.client.1.smithi078.stdout:8/541: fsync d0/d19/d52/d35/d44/d6b/d6c/f86 0 2024-04-26T04:52:57.237 INFO:tasks.workunit.client.1.smithi078.stdout:7/554: creat d9/da/d33/d4b/d4c/d6c/fb6 x:0 0 0 2024-04-26T04:52:57.238 INFO:tasks.workunit.client.1.smithi078.stdout:3/602: sync 2024-04-26T04:52:57.239 INFO:tasks.workunit.client.0.smithi077.stdout:6/277: mknod d8/d52/c5f 0 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:2/208: sync 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:4/246: sync 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:0/184: sync 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:3/204: sync 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:8/249: sync 2024-04-26T04:52:57.242 INFO:tasks.workunit.client.0.smithi077.stdout:7/265: sync 2024-04-26T04:52:57.243 INFO:tasks.workunit.client.0.smithi077.stdout:7/266: fdatasync da/f11 0 2024-04-26T04:52:57.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/545: dwrite d8/d11/d14/f3b [0,4194304] 0 2024-04-26T04:52:57.243 INFO:tasks.workunit.client.1.smithi078.stdout:5/507: dwrite d7/d18/d1c/f83 [0,4194304] 0 2024-04-26T04:52:57.244 INFO:tasks.workunit.client.1.smithi078.stdout:5/508: stat d7/d18/f27 0 2024-04-26T04:52:57.246 INFO:tasks.workunit.client.0.smithi077.stdout:5/268: dwrite d1/d30/f53 [0,4194304] 0 2024-04-26T04:52:57.248 INFO:tasks.workunit.client.0.smithi077.stdout:5/269: write d1/d12/d21/d25/d31/d3c/f68 [957052,13531] 0 2024-04-26T04:52:57.254 INFO:tasks.workunit.client.1.smithi078.stdout:1/615: truncate d7/ff 2066895 0 2024-04-26T04:52:57.254 INFO:tasks.workunit.client.1.smithi078.stdout:0/626: truncate d2/d9/d31/f85 3933853 0 2024-04-26T04:52:57.256 INFO:tasks.workunit.client.0.smithi077.stdout:4/247: dwrite d4/f8 [8388608,4194304] 0 2024-04-26T04:52:57.258 INFO:tasks.workunit.client.0.smithi077.stdout:4/248: truncate d4/de/f47 455601 0 2024-04-26T04:52:57.259 INFO:tasks.workunit.client.1.smithi078.stdout:4/612: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/le2 0 2024-04-26T04:52:57.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/250: chown d4/df/d22/d2d/d2b/d33/f39 1174 1 2024-04-26T04:52:57.265 INFO:tasks.workunit.client.0.smithi077.stdout:3/205: creat d0/d9/d32/d29/f41 x:0 0 0 2024-04-26T04:52:57.265 INFO:tasks.workunit.client.1.smithi078.stdout:4/613: dwrite d0/de/f91 [0,4194304] 0 2024-04-26T04:52:57.267 INFO:tasks.workunit.client.0.smithi077.stdout:5/270: creat d1/d42/f69 x:0 0 0 2024-04-26T04:52:57.268 INFO:tasks.workunit.client.0.smithi077.stdout:0/185: symlink d0/db/l3f 0 2024-04-26T04:52:57.270 INFO:tasks.workunit.client.1.smithi078.stdout:2/546: symlink d8/lc3 0 2024-04-26T04:52:57.271 INFO:tasks.workunit.client.1.smithi078.stdout:5/509: creat d7/d48/d57/d65/d7a/fac x:0 0 0 2024-04-26T04:52:57.273 INFO:tasks.workunit.client.0.smithi077.stdout:4/249: mknod d4/de/d15/d1f/d25/c55 0 2024-04-26T04:52:57.276 INFO:tasks.workunit.client.1.smithi078.stdout:9/533: unlink dc/df/d4b/l1f 0 2024-04-26T04:52:57.277 INFO:tasks.workunit.client.0.smithi077.stdout:6/278: rename f3 to d8/f60 0 2024-04-26T04:52:57.280 INFO:tasks.workunit.client.1.smithi078.stdout:9/534: read dc/f3d [881794,109397] 0 2024-04-26T04:52:57.283 INFO:tasks.workunit.client.0.smithi077.stdout:2/209: write d9/fe [628375,47642] 0 2024-04-26T04:52:57.283 INFO:tasks.workunit.client.1.smithi078.stdout:6/661: dread d2/de/d27/da9/da/d35/f70 [0,4194304] 0 2024-04-26T04:52:57.285 INFO:tasks.workunit.client.0.smithi077.stdout:5/271: mknod d1/d8/c6a 0 2024-04-26T04:52:57.286 INFO:tasks.workunit.client.0.smithi077.stdout:5/272: dread - d1/d30/f36 zero size 2024-04-26T04:52:57.287 INFO:tasks.workunit.client.1.smithi078.stdout:8/542: unlink d0/d19/d25/d36/c66 0 2024-04-26T04:52:57.288 INFO:tasks.workunit.client.0.smithi077.stdout:0/186: mkdir d0/d28/d40 0 2024-04-26T04:52:57.291 INFO:tasks.workunit.client.1.smithi078.stdout:4/614: dread - d0/d1d/d46/f77 zero size 2024-04-26T04:52:57.293 INFO:tasks.workunit.client.0.smithi077.stdout:7/267: rename da/d15/f2a to da/d13/d22/d5d/f60 0 2024-04-26T04:52:57.294 INFO:tasks.workunit.client.1.smithi078.stdout:7/555: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d95 0 2024-04-26T04:52:57.295 INFO:tasks.workunit.client.1.smithi078.stdout:7/556: chown d9/da/d33/d4b/d4c/d6c/fb6 8168653 1 2024-04-26T04:52:57.295 INFO:tasks.workunit.client.0.smithi077.stdout:7/268: truncate da/d15/d1a/d39/f41 4280863 0 2024-04-26T04:52:57.296 INFO:tasks.workunit.client.0.smithi077.stdout:6/279: mknod d8/d1c/c61 0 2024-04-26T04:52:57.297 INFO:tasks.workunit.client.0.smithi077.stdout:6/280: truncate d8/f3f 118666 0 2024-04-26T04:52:57.299 INFO:tasks.workunit.client.0.smithi077.stdout:8/251: fsync d4/df/d22/d2d/f4a 0 2024-04-26T04:52:57.301 INFO:tasks.workunit.client.1.smithi078.stdout:5/510: chown d7/f38 76 1 2024-04-26T04:52:57.307 INFO:tasks.workunit.client.1.smithi078.stdout:1/616: symlink d7/d1e/d9e/lcd 0 2024-04-26T04:52:57.309 INFO:tasks.workunit.client.0.smithi077.stdout:2/210: creat d9/d15/d1a/d2b/f4b x:0 0 0 2024-04-26T04:52:57.310 INFO:tasks.workunit.client.0.smithi077.stdout:2/211: dread - d9/dc/d1d/f1f zero size 2024-04-26T04:52:57.310 INFO:tasks.workunit.client.1.smithi078.stdout:1/617: stat d7/d1e/d9e/db/d37 0 2024-04-26T04:52:57.312 INFO:tasks.workunit.client.1.smithi078.stdout:4/615: dread d0/d1d/de1/d20/d31/d47/db2/d44/d69/fa4 [0,4194304] 0 2024-04-26T04:52:57.313 INFO:tasks.workunit.client.1.smithi078.stdout:0/627: dwrite d2/d9/d31/d4a/f64 [0,4194304] 0 2024-04-26T04:52:57.314 INFO:tasks.workunit.client.0.smithi077.stdout:0/187: creat d0/d19/d1e/d2f/f41 x:0 0 0 2024-04-26T04:52:57.314 INFO:tasks.workunit.client.1.smithi078.stdout:1/618: stat d7/d1e/d9e/db/d37/d58/d5c 0 2024-04-26T04:52:57.315 INFO:tasks.workunit.client.1.smithi078.stdout:9/535: read dc/df/d4b/d22/d2d/f4d [4126532,36764] 0 2024-04-26T04:52:57.316 INFO:tasks.workunit.client.1.smithi078.stdout:0/628: dread d2/d9/d18/d7d/fa0 [0,4194304] 0 2024-04-26T04:52:57.318 INFO:tasks.workunit.client.0.smithi077.stdout:0/188: dread d0/f2 [0,4194304] 0 2024-04-26T04:52:57.319 INFO:tasks.workunit.client.0.smithi077.stdout:4/250: rename d4/de/d19 to d4/d17/d18/d43/d53/d56 0 2024-04-26T04:52:57.319 INFO:tasks.workunit.client.0.smithi077.stdout:0/189: dread - d0/d19/d1e/f34 zero size 2024-04-26T04:52:57.320 INFO:tasks.workunit.client.0.smithi077.stdout:4/251: chown d4/de/d15/d1f/d25 35187485 1 2024-04-26T04:52:57.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/281: creat d8/d55/f62 x:0 0 0 2024-04-26T04:52:57.323 INFO:tasks.workunit.client.1.smithi078.stdout:3/603: dwrite d16/f36 [0,4194304] 0 2024-04-26T04:52:57.326 INFO:tasks.workunit.client.0.smithi077.stdout:8/252: mknod d4/df/d18/c5a 0 2024-04-26T04:52:57.328 INFO:tasks.workunit.client.0.smithi077.stdout:5/273: mkdir d1/d42/d62/d6b 0 2024-04-26T04:52:57.329 INFO:tasks.workunit.client.0.smithi077.stdout:5/274: chown d1/d30/d35/f57 202 1 2024-04-26T04:52:57.329 INFO:tasks.workunit.client.1.smithi078.stdout:2/547: symlink d8/d11/d14/d69/d8f/d34/d70/lc4 0 2024-04-26T04:52:57.330 INFO:tasks.workunit.client.0.smithi077.stdout:5/275: truncate d1/d42/f69 979163 0 2024-04-26T04:52:57.335 INFO:tasks.workunit.client.0.smithi077.stdout:3/206: truncate d0/f1 671436 0 2024-04-26T04:52:57.337 INFO:tasks.workunit.client.0.smithi077.stdout:0/190: truncate d0/f15 2791 0 2024-04-26T04:52:57.339 INFO:tasks.workunit.client.0.smithi077.stdout:4/252: fsync d4/de/d15/f16 0 2024-04-26T04:52:57.342 INFO:tasks.workunit.client.1.smithi078.stdout:4/616: creat d0/de/d22/d27/d56/fe3 x:0 0 0 2024-04-26T04:52:57.346 INFO:tasks.workunit.client.0.smithi077.stdout:2/212: mknod d9/c4c 0 2024-04-26T04:52:57.346 INFO:tasks.workunit.client.0.smithi077.stdout:4/253: dwrite d4/de/d15/d1f/f34 [0,4194304] 0 2024-04-26T04:52:57.347 INFO:tasks.workunit.client.1.smithi078.stdout:1/619: rmdir d7/d1e/d9e/db/d49/dba/dc5 39 2024-04-26T04:52:57.347 INFO:tasks.workunit.client.1.smithi078.stdout:6/662: dwrite d2/de/d27/da9/da/d35/f3e [0,4194304] 0 2024-04-26T04:52:57.348 INFO:tasks.workunit.client.0.smithi077.stdout:5/276: creat d1/d30/d35/f6c x:0 0 0 2024-04-26T04:52:57.349 INFO:tasks.workunit.client.1.smithi078.stdout:6/663: read d2/d1a/d38/f79 [1395463,110402] 0 2024-04-26T04:52:57.349 INFO:tasks.workunit.client.1.smithi078.stdout:6/664: fdatasync d2/f71 0 2024-04-26T04:52:57.350 INFO:tasks.workunit.client.1.smithi078.stdout:9/536: rmdir dc/df/d4b/d22/d2d 39 2024-04-26T04:52:57.353 INFO:tasks.workunit.client.1.smithi078.stdout:0/629: fdatasync d2/d9/d31/d4a/f7a 0 2024-04-26T04:52:57.357 INFO:tasks.workunit.client.1.smithi078.stdout:5/511: write f5 [3354881,32908] 0 2024-04-26T04:52:57.363 INFO:tasks.workunit.client.1.smithi078.stdout:5/512: dwrite d7/f8 [4194304,4194304] 0 2024-04-26T04:52:57.366 INFO:tasks.workunit.client.0.smithi077.stdout:4/254: readlink d4/de/d15/l22 0 2024-04-26T04:52:57.367 INFO:tasks.workunit.client.0.smithi077.stdout:4/255: chown d4/d17/d18/d43/d51 7 1 2024-04-26T04:52:57.370 INFO:tasks.workunit.client.1.smithi078.stdout:2/548: rmdir d8/d3e 39 2024-04-26T04:52:57.373 INFO:tasks.workunit.client.0.smithi077.stdout:6/282: creat d8/d2a/f63 x:0 0 0 2024-04-26T04:52:57.373 INFO:tasks.workunit.client.1.smithi078.stdout:4/617: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/fa6 [3942227,120509] 0 2024-04-26T04:52:57.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/191: write d0/db/ff [370241,88108] 0 2024-04-26T04:52:57.381 INFO:tasks.workunit.client.1.smithi078.stdout:7/557: write f4 [1389972,11399] 0 2024-04-26T04:52:57.382 INFO:tasks.workunit.client.1.smithi078.stdout:7/558: stat d9/da/d33/fb0 0 2024-04-26T04:52:57.383 INFO:tasks.workunit.client.0.smithi077.stdout:5/277: dread d1/d8/f22 [0,4194304] 0 2024-04-26T04:52:57.384 INFO:tasks.workunit.client.1.smithi078.stdout:1/620: dwrite d7/fc [4194304,4194304] 0 2024-04-26T04:52:57.384 INFO:tasks.workunit.client.0.smithi077.stdout:8/253: getdents d4 0 2024-04-26T04:52:57.390 INFO:tasks.workunit.client.1.smithi078.stdout:8/543: link d0/d19/d52/c15 d0/d19/d52/d35/d73/cac 0 2024-04-26T04:52:57.390 INFO:tasks.workunit.client.1.smithi078.stdout:5/513: creat d7/d18/d32/d37/d79/fad x:0 0 0 2024-04-26T04:52:57.391 INFO:tasks.workunit.client.1.smithi078.stdout:3/604: symlink d16/d26/d2b/d31/d71/ld3 0 2024-04-26T04:52:57.391 INFO:tasks.workunit.client.1.smithi078.stdout:6/665: truncate d2/de/d27/da9/d3a/d69/daa/fc6 310200 0 2024-04-26T04:52:57.392 INFO:tasks.workunit.client.0.smithi077.stdout:8/254: rmdir d4/df 39 2024-04-26T04:52:57.393 INFO:tasks.workunit.client.1.smithi078.stdout:7/559: dwrite d9/da/d11/d13/f91 [0,4194304] 0 2024-04-26T04:52:57.393 INFO:tasks.workunit.client.1.smithi078.stdout:7/560: fdatasync f4 0 2024-04-26T04:52:57.394 INFO:tasks.workunit.client.0.smithi077.stdout:0/192: truncate d0/f1 3937059 0 2024-04-26T04:52:57.395 INFO:tasks.workunit.client.0.smithi077.stdout:0/193: read - d0/d19/d1e/f34 zero size 2024-04-26T04:52:57.395 INFO:tasks.workunit.client.1.smithi078.stdout:7/561: stat d9/da/d11/d13/d1a/d52 0 2024-04-26T04:52:57.395 INFO:tasks.workunit.client.1.smithi078.stdout:2/549: mkdir d8/d11/d14/d69/dc5 0 2024-04-26T04:52:57.396 INFO:tasks.workunit.client.1.smithi078.stdout:7/562: write d9/da/d33/d4b/d4c/d6c/f81 [904034,34277] 0 2024-04-26T04:52:57.397 INFO:tasks.workunit.client.0.smithi077.stdout:2/213: getdents d9/dc/d1d/d45 0 2024-04-26T04:52:57.401 INFO:tasks.workunit.client.0.smithi077.stdout:6/283: dread f4 [0,4194304] 0 2024-04-26T04:52:57.404 INFO:tasks.workunit.client.0.smithi077.stdout:0/194: mknod d0/c42 0 2024-04-26T04:52:57.406 INFO:tasks.workunit.client.0.smithi077.stdout:2/214: mknod d9/dc/d1d/d1e/c4d 0 2024-04-26T04:52:57.407 INFO:tasks.workunit.client.0.smithi077.stdout:4/256: truncate d4/f39 1450069 0 2024-04-26T04:52:57.408 INFO:tasks.workunit.client.1.smithi078.stdout:0/630: creat d2/d9/d18/db7/d92/fd3 x:0 0 0 2024-04-26T04:52:57.410 INFO:tasks.workunit.client.1.smithi078.stdout:3/605: mknod d16/d9b/cd4 0 2024-04-26T04:52:57.411 INFO:tasks.workunit.client.1.smithi078.stdout:0/631: dread - d2/d9/d60/d6a/fc4 zero size 2024-04-26T04:52:57.411 INFO:tasks.workunit.client.1.smithi078.stdout:9/537: dread dc/df/f8f [0,4194304] 0 2024-04-26T04:52:57.411 INFO:tasks.workunit.client.0.smithi077.stdout:8/255: chown d4/df/d22/d2d/d2b/l31 5 1 2024-04-26T04:52:57.413 INFO:tasks.workunit.client.1.smithi078.stdout:4/618: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/d55/f95 [0,4194304] 0 2024-04-26T04:52:57.413 INFO:tasks.workunit.client.1.smithi078.stdout:8/544: rename d0/d19/d52/d95/c9e to d0/cad 0 2024-04-26T04:52:57.414 INFO:tasks.workunit.client.1.smithi078.stdout:8/545: readlink d0/d19/d52/d95/l9f 0 2024-04-26T04:52:57.416 INFO:tasks.workunit.client.1.smithi078.stdout:4/619: fdatasync d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f50 0 2024-04-26T04:52:57.422 INFO:tasks.workunit.client.0.smithi077.stdout:5/278: getdents d1/d12/d21/d25 0 2024-04-26T04:52:57.424 INFO:tasks.workunit.client.0.smithi077.stdout:6/284: write d8/d1c/f25 [618764,41363] 0 2024-04-26T04:52:57.426 INFO:tasks.workunit.client.1.smithi078.stdout:7/563: mknod d9/da/d11/d13/d50/d8a/cb7 0 2024-04-26T04:52:57.427 INFO:tasks.workunit.client.1.smithi078.stdout:7/564: dread - d9/f58 zero size 2024-04-26T04:52:57.427 INFO:tasks.workunit.client.1.smithi078.stdout:7/565: fsync d9/da/d11/d13/d50/f7c 0 2024-04-26T04:52:57.431 INFO:tasks.workunit.client.0.smithi077.stdout:6/285: dwrite d8/f17 [0,4194304] 0 2024-04-26T04:52:57.432 INFO:tasks.workunit.client.1.smithi078.stdout:7/566: dwrite d9/da/d11/fad [0,4194304] 0 2024-04-26T04:52:57.433 INFO:tasks.workunit.client.1.smithi078.stdout:7/567: stat d9/da/d11/d13/d1a/d52/fb5 0 2024-04-26T04:52:57.434 INFO:tasks.workunit.client.1.smithi078.stdout:9/538: dread - dc/df/d4b/d22/d2a/f4a zero size 2024-04-26T04:52:57.436 INFO:tasks.workunit.client.0.smithi077.stdout:0/195: creat d0/d2c/f43 x:0 0 0 2024-04-26T04:52:57.437 INFO:tasks.workunit.client.1.smithi078.stdout:0/632: write d2/d9/d18/f13 [1329026,13817] 0 2024-04-26T04:52:57.440 INFO:tasks.workunit.client.0.smithi077.stdout:5/279: symlink d1/d12/l6d 0 2024-04-26T04:52:57.441 INFO:tasks.workunit.client.1.smithi078.stdout:9/539: dwrite dc/df/d27/f9b [0,4194304] 0 2024-04-26T04:52:57.445 INFO:tasks.workunit.client.0.smithi077.stdout:0/196: fdatasync d0/d19/d1e/f20 0 2024-04-26T04:52:57.446 INFO:tasks.workunit.client.1.smithi078.stdout:9/540: dwrite dc/df/d4b/d22/d2a/d57/d69/f96 [0,4194304] 0 2024-04-26T04:52:57.447 INFO:tasks.workunit.client.0.smithi077.stdout:2/215: getdents d9/d15 0 2024-04-26T04:52:57.447 INFO:tasks.workunit.client.0.smithi077.stdout:5/280: symlink d1/d30/d33/l6e 0 2024-04-26T04:52:57.451 INFO:tasks.workunit.client.0.smithi077.stdout:5/281: write d1/d42/f69 [560303,22377] 0 2024-04-26T04:52:57.453 INFO:tasks.workunit.client.0.smithi077.stdout:6/286: dwrite d8/f60 [0,4194304] 0 2024-04-26T04:52:57.454 INFO:tasks.workunit.client.0.smithi077.stdout:2/216: chown d9/dc/d1d/d1e/c28 0 1 2024-04-26T04:52:57.455 INFO:tasks.workunit.client.0.smithi077.stdout:6/287: fdatasync d8/d1c/f4f 0 2024-04-26T04:52:57.455 INFO:tasks.workunit.client.0.smithi077.stdout:6/288: dread - d8/d1c/f4f zero size 2024-04-26T04:52:57.458 INFO:tasks.workunit.client.0.smithi077.stdout:6/289: dread d8/f3f [0,4194304] 0 2024-04-26T04:52:57.460 INFO:tasks.workunit.client.1.smithi078.stdout:3/606: getdents d16/d3c/d88 0 2024-04-26T04:52:57.461 INFO:tasks.workunit.client.0.smithi077.stdout:5/282: readlink d1/d30/d35/l54 0 2024-04-26T04:52:57.462 INFO:tasks.workunit.client.0.smithi077.stdout:2/217: dread d9/d15/d1a/f1c [0,4194304] 0 2024-04-26T04:52:57.463 INFO:tasks.workunit.client.1.smithi078.stdout:5/514: truncate d7/d18/f44 527685 0 2024-04-26T04:52:57.463 INFO:tasks.workunit.client.0.smithi077.stdout:5/283: write d1/d12/d21/d25/f4b [246077,55402] 0 2024-04-26T04:52:57.465 INFO:tasks.workunit.client.1.smithi078.stdout:6/666: truncate d2/de/d27/f64 3540518 0 2024-04-26T04:52:57.466 INFO:tasks.workunit.client.0.smithi077.stdout:0/197: dread d0/f9 [0,4194304] 0 2024-04-26T04:52:57.468 INFO:tasks.workunit.client.0.smithi077.stdout:2/218: dwrite f7 [0,4194304] 0 2024-04-26T04:52:57.468 INFO:tasks.workunit.client.0.smithi077.stdout:6/290: stat d8/cd 0 2024-04-26T04:52:57.470 INFO:tasks.workunit.client.0.smithi077.stdout:2/219: write d9/dc/d1d/d1e/f21 [2032541,117830] 0 2024-04-26T04:52:57.476 INFO:tasks.workunit.client.1.smithi078.stdout:4/620: mkdir d0/d1d/de1/d20/dd5/de4 0 2024-04-26T04:52:57.480 INFO:tasks.workunit.client.0.smithi077.stdout:5/284: creat d1/d42/d62/f6f x:0 0 0 2024-04-26T04:52:57.481 INFO:tasks.workunit.client.1.smithi078.stdout:9/541: truncate dc/df/d4b/d22/d2a/f4a 350750 0 2024-04-26T04:52:57.483 INFO:tasks.workunit.client.1.smithi078.stdout:9/542: write dc/df/d4b/d22/f97 [605556,54022] 0 2024-04-26T04:52:57.484 INFO:tasks.workunit.client.1.smithi078.stdout:2/550: creat d8/fc6 x:0 0 0 2024-04-26T04:52:57.485 INFO:tasks.workunit.client.0.smithi077.stdout:0/198: creat d0/d19/d1e/d2f/f44 x:0 0 0 2024-04-26T04:52:57.488 INFO:tasks.workunit.client.1.smithi078.stdout:1/621: getdents d7/d1e/d9e/dd/d20 0 2024-04-26T04:52:57.488 INFO:tasks.workunit.client.0.smithi077.stdout:6/291: truncate d8/d2a/f34 545028 0 2024-04-26T04:52:57.489 INFO:tasks.workunit.client.1.smithi078.stdout:3/607: symlink d16/d26/d2b/ld5 0 2024-04-26T04:52:57.490 INFO:tasks.workunit.client.1.smithi078.stdout:0/633: dwrite d2/d1e/f56 [0,4194304] 0 2024-04-26T04:52:57.491 INFO:tasks.workunit.client.1.smithi078.stdout:5/515: symlink d7/d48/d57/d65/d46/lae 0 2024-04-26T04:52:57.492 INFO:tasks.workunit.client.1.smithi078.stdout:5/516: chown d7/d11/f47 14275 1 2024-04-26T04:52:57.493 INFO:tasks.workunit.client.1.smithi078.stdout:7/568: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fb8 x:0 0 0 2024-04-26T04:52:57.494 INFO:tasks.workunit.client.1.smithi078.stdout:5/517: chown d7/d18/d1c/f9e 61057292 1 2024-04-26T04:52:57.494 INFO:tasks.workunit.client.0.smithi077.stdout:5/285: rmdir d1/d42/d62 39 2024-04-26T04:52:57.496 INFO:tasks.workunit.client.0.smithi077.stdout:5/286: write d1/d12/d21/d25/d31/d3c/f68 [1992593,94862] 0 2024-04-26T04:52:57.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/199: fdatasync d0/db/f16 0 2024-04-26T04:52:57.497 INFO:tasks.workunit.client.1.smithi078.stdout:4/621: dread d0/de/d22/d27/d7b/f8f [0,4194304] 0 2024-04-26T04:52:57.498 INFO:tasks.workunit.client.1.smithi078.stdout:8/546: rmdir d0/d19/da8 0 2024-04-26T04:52:57.498 INFO:tasks.workunit.client.0.smithi077.stdout:0/200: write d0/db/f16 [1663077,125958] 0 2024-04-26T04:52:57.499 INFO:tasks.workunit.client.0.smithi077.stdout:0/201: dread - d0/d19/d1e/f34 zero size 2024-04-26T04:52:57.500 INFO:tasks.workunit.client.0.smithi077.stdout:6/292: creat d8/d1c/f64 x:0 0 0 2024-04-26T04:52:57.502 INFO:tasks.workunit.client.0.smithi077.stdout:2/220: mkdir d9/d15/d44/d4e 0 2024-04-26T04:52:57.502 INFO:tasks.workunit.client.0.smithi077.stdout:2/221: dread - d9/dc/f41 zero size 2024-04-26T04:52:57.505 INFO:tasks.workunit.client.1.smithi078.stdout:9/543: creat dc/df/d27/d81/f9e x:0 0 0 2024-04-26T04:52:57.506 INFO:tasks.workunit.client.1.smithi078.stdout:2/551: chown d8/d11/d14/d69/d8f/f6c 14299 1 2024-04-26T04:52:57.508 INFO:tasks.workunit.client.0.smithi077.stdout:5/287: fdatasync d1/d8/f49 0 2024-04-26T04:52:57.512 INFO:tasks.workunit.client.1.smithi078.stdout:1/622: symlink d7/d1e/d9e/db/d49/d93/lce 0 2024-04-26T04:52:57.513 INFO:tasks.workunit.client.0.smithi077.stdout:6/293: creat d8/d52/d53/f65 x:0 0 0 2024-04-26T04:52:57.518 INFO:tasks.workunit.client.1.smithi078.stdout:7/569: chown d9/da/d11/d13/d1a/d2d/f3f 6946 1 2024-04-26T04:52:57.518 INFO:tasks.workunit.client.1.smithi078.stdout:5/518: symlink d7/d48/d57/d65/d70/d72/laf 0 2024-04-26T04:52:57.519 INFO:tasks.workunit.client.0.smithi077.stdout:2/222: truncate d9/d15/d1a/d2b/f46 1380591 0 2024-04-26T04:52:57.519 INFO:tasks.workunit.client.1.smithi078.stdout:5/519: truncate d7/d48/fa3 1670776 0 2024-04-26T04:52:57.520 INFO:tasks.workunit.client.0.smithi077.stdout:0/202: unlink d0/d28/l2b 0 2024-04-26T04:52:57.520 INFO:tasks.workunit.client.1.smithi078.stdout:5/520: read d7/f8 [3689391,127028] 0 2024-04-26T04:52:57.522 INFO:tasks.workunit.client.1.smithi078.stdout:5/521: truncate d7/d48/d57/faa 615234 0 2024-04-26T04:52:57.522 INFO:tasks.workunit.client.1.smithi078.stdout:8/547: rename d0/d19/d25/f87 to d0/fae 0 2024-04-26T04:52:57.524 INFO:tasks.workunit.client.0.smithi077.stdout:6/294: creat d8/d2a/f66 x:0 0 0 2024-04-26T04:52:57.525 INFO:tasks.workunit.client.1.smithi078.stdout:2/552: symlink d8/d11/d18/d5d/d8b/db0/db4/lc7 0 2024-04-26T04:52:57.527 INFO:tasks.workunit.client.1.smithi078.stdout:1/623: rmdir d7/d1e/d9e/db/d37/d58/d86 39 2024-04-26T04:52:57.527 INFO:tasks.workunit.client.0.smithi077.stdout:2/223: link d9/c2a d9/d15/d44/c4f 0 2024-04-26T04:52:57.529 INFO:tasks.workunit.client.1.smithi078.stdout:3/608: unlink d16/d26/d2b/lcb 0 2024-04-26T04:52:57.532 INFO:tasks.workunit.client.0.smithi077.stdout:2/224: creat d9/dc/d1d/d34/f50 x:0 0 0 2024-04-26T04:52:57.539 INFO:tasks.workunit.client.0.smithi077.stdout:2/225: mknod d9/dc/d1d/c51 0 2024-04-26T04:52:57.539 INFO:tasks.workunit.client.0.smithi077.stdout:2/226: fsync d9/fe 0 2024-04-26T04:52:57.548 INFO:tasks.workunit.client.0.smithi077.stdout:0/203: dwrite d0/f25 [0,4194304] 0 2024-04-26T04:52:57.549 INFO:tasks.workunit.client.1.smithi078.stdout:2/553: rmdir d8/d11 39 2024-04-26T04:52:57.550 INFO:tasks.workunit.client.1.smithi078.stdout:2/554: write d8/f84 [517000,70775] 0 2024-04-26T04:52:57.551 INFO:tasks.workunit.client.0.smithi077.stdout:2/227: mknod d9/dc/d1d/d45/c52 0 2024-04-26T04:52:57.552 INFO:tasks.workunit.client.1.smithi078.stdout:1/624: stat d7/d1e/d9e/dd/d20/d53/c75 0 2024-04-26T04:52:57.556 INFO:tasks.workunit.client.1.smithi078.stdout:3/609: mkdir d16/d26/d2b/dd6 0 2024-04-26T04:52:57.557 INFO:tasks.workunit.client.1.smithi078.stdout:3/610: stat d16/d26/d2b/d31/f93 0 2024-04-26T04:52:57.558 INFO:tasks.workunit.client.1.smithi078.stdout:6/667: getdents d2/d1a 0 2024-04-26T04:52:57.558 INFO:tasks.workunit.client.0.smithi077.stdout:2/228: creat d9/d15/d1a/d2b/f53 x:0 0 0 2024-04-26T04:52:57.560 INFO:tasks.workunit.client.0.smithi077.stdout:0/204: symlink d0/db/l45 0 2024-04-26T04:52:57.562 INFO:tasks.workunit.client.0.smithi077.stdout:0/205: mknod d0/d2c/c46 0 2024-04-26T04:52:57.567 INFO:tasks.workunit.client.1.smithi078.stdout:9/544: link dc/f94 dc/df/f9f 0 2024-04-26T04:52:57.567 INFO:tasks.workunit.client.1.smithi078.stdout:9/545: stat dc/df/d27/l34 0 2024-04-26T04:52:57.569 INFO:tasks.workunit.client.1.smithi078.stdout:2/555: chown d8/d11/d6e/f80 15387 1 2024-04-26T04:52:57.570 INFO:tasks.workunit.client.1.smithi078.stdout:9/546: dread dc/df/f8f [0,4194304] 0 2024-04-26T04:52:57.572 INFO:tasks.workunit.client.1.smithi078.stdout:3/611: rename d16/d26/d2b/d31/f89 to d16/d3c/d81/fd7 0 2024-04-26T04:52:57.573 INFO:tasks.workunit.client.1.smithi078.stdout:5/522: truncate d7/d21/f42 3072975 0 2024-04-26T04:52:57.574 INFO:tasks.workunit.client.1.smithi078.stdout:4/622: getdents d0/de/d22/d27/d56/d5a 0 2024-04-26T04:52:57.577 INFO:tasks.workunit.client.0.smithi077.stdout:0/206: rename d0/db to d0/d47 0 2024-04-26T04:52:57.579 INFO:tasks.workunit.client.1.smithi078.stdout:8/548: link d0/d19/d52/d35/d44/d6b/c79 d0/d19/d25/d36/d58/d5b/d61/d67/caf 0 2024-04-26T04:52:57.579 INFO:tasks.workunit.client.1.smithi078.stdout:1/625: mkdir d7/d1e/d9e/db/d49/dba/dc5/dcf 0 2024-04-26T04:52:57.581 INFO:tasks.workunit.client.1.smithi078.stdout:9/547: creat dc/df/d4b/d22/d2a/d57/d69/fa0 x:0 0 0 2024-04-26T04:52:57.584 INFO:tasks.workunit.client.0.smithi077.stdout:0/207: write d0/d19/d1e/d2f/f41 [15660,2586] 0 2024-04-26T04:52:57.585 INFO:tasks.workunit.client.1.smithi078.stdout:3/612: creat d16/d26/d2e/d5f/d49/dad/fd8 x:0 0 0 2024-04-26T04:52:57.586 INFO:tasks.workunit.client.0.smithi077.stdout:9/193: sync 2024-04-26T04:52:57.586 INFO:tasks.workunit.client.0.smithi077.stdout:1/340: sync 2024-04-26T04:52:57.586 INFO:tasks.workunit.client.0.smithi077.stdout:7/269: sync 2024-04-26T04:52:57.587 INFO:tasks.workunit.client.0.smithi077.stdout:9/194: fsync d9/dd/d1b/f23 0 2024-04-26T04:52:57.589 INFO:tasks.workunit.client.0.smithi077.stdout:7/270: mkdir da/d15/d61 0 2024-04-26T04:52:57.590 INFO:tasks.workunit.client.0.smithi077.stdout:7/271: write da/d13/d22/f2e [2605712,48343] 0 2024-04-26T04:52:57.592 INFO:tasks.workunit.client.0.smithi077.stdout:1/341: mkdir d9/db/dd/d34/d63 0 2024-04-26T04:52:57.605 INFO:tasks.workunit.client.1.smithi078.stdout:3/613: symlink d16/d26/d2b/d66/ld9 0 2024-04-26T04:52:57.605 INFO:tasks.workunit.client.1.smithi078.stdout:4/623: fsync d0/f37 0 2024-04-26T04:52:57.607 INFO:tasks.workunit.client.1.smithi078.stdout:2/556: rename d8/d11/d18/d5d/d61/c1d to d8/d11/d18/d5d/cc8 0 2024-04-26T04:52:57.607 INFO:tasks.workunit.client.1.smithi078.stdout:8/549: dwrite d0/d19/d52/d35/d44/f5d [0,4194304] 0 2024-04-26T04:52:57.609 INFO:tasks.workunit.client.1.smithi078.stdout:9/548: readlink dc/df/d4b/d3e/l65 0 2024-04-26T04:52:57.609 INFO:tasks.workunit.client.1.smithi078.stdout:3/614: mknod d16/d26/d2b/d54/cda 0 2024-04-26T04:52:57.610 INFO:tasks.workunit.client.1.smithi078.stdout:1/626: link d7/d1e/d9e/lcd d7/d1e/d9e/d92/ld0 0 2024-04-26T04:52:57.612 INFO:tasks.workunit.client.1.smithi078.stdout:9/549: fsync dc/df/d27/d81/f66 0 2024-04-26T04:52:57.613 INFO:tasks.workunit.client.1.smithi078.stdout:3/615: readlink d16/d26/d2e/d74/l6b 0 2024-04-26T04:52:57.613 INFO:tasks.workunit.client.1.smithi078.stdout:1/627: write d7/d1e/d9e/f9f [9103896,48029] 0 2024-04-26T04:52:57.616 INFO:tasks.workunit.client.1.smithi078.stdout:2/557: dread d8/d11/d14/f1c [0,4194304] 0 2024-04-26T04:52:57.617 INFO:tasks.workunit.client.1.smithi078.stdout:2/558: write d8/f1f [936860,15287] 0 2024-04-26T04:52:57.617 INFO:tasks.workunit.client.1.smithi078.stdout:4/624: creat d0/d1d/de1/d20/d31/d47/db2/d44/d55/fe5 x:0 0 0 2024-04-26T04:52:57.618 INFO:tasks.workunit.client.1.smithi078.stdout:3/616: read fd [528286,96859] 0 2024-04-26T04:52:57.619 INFO:tasks.workunit.client.1.smithi078.stdout:1/628: dread d7/d1e/d9e/d4c/f26 [0,4194304] 0 2024-04-26T04:52:57.620 INFO:tasks.workunit.client.1.smithi078.stdout:5/523: rename d7/d11/f2b to d7/d48/d57/d65/fb0 0 2024-04-26T04:52:57.620 INFO:tasks.workunit.client.1.smithi078.stdout:5/524: fdatasync d7/d11/f6a 0 2024-04-26T04:52:57.621 INFO:tasks.workunit.client.1.smithi078.stdout:1/629: write d7/d1e/d9e/dd/d20/f50 [4418324,26663] 0 2024-04-26T04:52:57.626 INFO:tasks.workunit.client.1.smithi078.stdout:3/617: mknod d16/d7e/cdb 0 2024-04-26T04:52:57.628 INFO:tasks.workunit.client.1.smithi078.stdout:5/525: rmdir d7/da/d1e 39 2024-04-26T04:52:57.629 INFO:tasks.workunit.client.1.smithi078.stdout:2/559: dwrite d8/d3e/f58 [0,4194304] 0 2024-04-26T04:52:57.631 INFO:tasks.workunit.client.1.smithi078.stdout:5/526: truncate d7/d48/d57/d65/d7a/fac 515823 0 2024-04-26T04:52:57.632 INFO:tasks.workunit.client.1.smithi078.stdout:3/618: mkdir d16/d26/d2b/d54/d56/ddc 0 2024-04-26T04:52:57.632 INFO:tasks.workunit.client.0.smithi077.stdout:0/208: mknod d0/d28/d40/c48 0 2024-04-26T04:52:57.634 INFO:tasks.workunit.client.1.smithi078.stdout:1/630: rename d7/d1e/d9e/dd/d20/d53/l79 to d7/d1e/d9e/d92/ld1 0 2024-04-26T04:52:57.635 INFO:tasks.workunit.client.1.smithi078.stdout:1/631: chown d7/d1e/d9e/dd/d20/l64 0 1 2024-04-26T04:52:57.635 INFO:tasks.workunit.client.0.smithi077.stdout:9/195: chown d9/dd/d1b/d2b/d32/d36/l3f 865429 1 2024-04-26T04:52:57.635 INFO:tasks.workunit.client.0.smithi077.stdout:0/209: write d0/d47/d11/f14 [1426745,31806] 0 2024-04-26T04:52:57.637 INFO:tasks.workunit.client.1.smithi078.stdout:2/560: fsync d8/d11/d18/d5d/d61/fb8 0 2024-04-26T04:52:57.637 INFO:tasks.workunit.client.0.smithi077.stdout:7/272: dwrite da/d13/f5c [0,4194304] 0 2024-04-26T04:52:57.641 INFO:tasks.workunit.client.1.smithi078.stdout:5/527: mkdir d7/d18/d32/d37/db1 0 2024-04-26T04:52:57.643 INFO:tasks.workunit.client.0.smithi077.stdout:1/342: mknod d9/db/d2b/d5c/c64 0 2024-04-26T04:52:57.645 INFO:tasks.workunit.client.1.smithi078.stdout:8/550: write d0/d19/fa1 [1298729,44594] 0 2024-04-26T04:52:57.649 INFO:tasks.workunit.client.0.smithi077.stdout:0/210: rename d0/f25 to d0/d19/f49 0 2024-04-26T04:52:57.650 INFO:tasks.workunit.client.1.smithi078.stdout:9/550: write dc/df/d4b/d22/d2a/d57/f90 [1963965,25902] 0 2024-04-26T04:52:57.650 INFO:tasks.workunit.client.1.smithi078.stdout:1/632: mknod d7/d1e/d9e/db/d49/dba/d87/cd2 0 2024-04-26T04:52:57.652 INFO:tasks.workunit.client.0.smithi077.stdout:7/273: symlink da/d15/d1a/d39/l62 0 2024-04-26T04:52:57.654 INFO:tasks.workunit.client.1.smithi078.stdout:2/561: mknod d8/d3e/d8e/d91/cc9 0 2024-04-26T04:52:57.654 INFO:tasks.workunit.client.1.smithi078.stdout:4/625: write d0/d1d/de1/d20/f23 [3780152,103281] 0 2024-04-26T04:52:57.658 INFO:tasks.workunit.client.1.smithi078.stdout:8/551: read - d0/d19/d52/f5a zero size 2024-04-26T04:52:57.658 INFO:tasks.workunit.client.1.smithi078.stdout:7/570: sync 2024-04-26T04:52:57.658 INFO:tasks.workunit.client.1.smithi078.stdout:0/634: sync 2024-04-26T04:52:57.661 INFO:tasks.workunit.client.0.smithi077.stdout:9/196: write d9/f17 [1869926,62980] 0 2024-04-26T04:52:57.662 INFO:tasks.workunit.client.0.smithi077.stdout:0/211: dread d0/f3 [4194304,4194304] 0 2024-04-26T04:52:57.664 INFO:tasks.workunit.client.1.smithi078.stdout:9/551: creat dc/df/d27/fa1 x:0 0 0 2024-04-26T04:52:57.668 INFO:tasks.workunit.client.0.smithi077.stdout:9/197: dwrite f2 [8388608,4194304] 0 2024-04-26T04:52:57.668 INFO:tasks.workunit.client.0.smithi077.stdout:7/274: mkdir da/d15/d61/d63 0 2024-04-26T04:52:57.670 INFO:tasks.workunit.client.0.smithi077.stdout:1/343: link d9/db/d2b/d5c/c64 d9/db/d2b/c65 0 2024-04-26T04:52:57.673 INFO:tasks.workunit.client.1.smithi078.stdout:5/528: truncate d7/d18/d1c/d5a/f7c 2224698 0 2024-04-26T04:52:57.676 INFO:tasks.workunit.client.1.smithi078.stdout:1/633: write d7/d1e/d9e/dd/d20/f40 [478040,52394] 0 2024-04-26T04:52:57.678 INFO:tasks.workunit.client.1.smithi078.stdout:1/634: write d7/d1e/d9e/db/d49/f6c [2649414,36785] 0 2024-04-26T04:52:57.679 INFO:tasks.workunit.client.0.smithi077.stdout:7/275: rmdir da 39 2024-04-26T04:52:57.680 INFO:tasks.workunit.client.1.smithi078.stdout:4/626: truncate d0/d1d/de1/d20/d31/d47/f49 1863754 0 2024-04-26T04:52:57.682 INFO:tasks.workunit.client.0.smithi077.stdout:9/198: creat d9/f46 x:0 0 0 2024-04-26T04:52:57.684 INFO:tasks.workunit.client.1.smithi078.stdout:7/571: fsync d9/da/faf 0 2024-04-26T04:52:57.685 INFO:tasks.workunit.client.1.smithi078.stdout:7/572: write d9/da/f15 [950214,11523] 0 2024-04-26T04:52:57.687 INFO:tasks.workunit.client.1.smithi078.stdout:2/562: dwrite d8/d11/d14/f1c [0,4194304] 0 2024-04-26T04:52:57.687 INFO:tasks.workunit.client.1.smithi078.stdout:3/619: rmdir d16/d26/d2b/d54/d56/ddc 0 2024-04-26T04:52:57.689 INFO:tasks.workunit.client.0.smithi077.stdout:1/344: mknod d9/db/dd/d34/d63/c66 0 2024-04-26T04:52:57.690 INFO:tasks.workunit.client.0.smithi077.stdout:1/345: fsync d9/db/f60 0 2024-04-26T04:52:57.691 INFO:tasks.workunit.client.1.smithi078.stdout:9/552: creat dc/df/d4b/d22/d2a/d92/fa2 x:0 0 0 2024-04-26T04:52:57.694 INFO:tasks.workunit.client.0.smithi077.stdout:9/199: creat d9/dd/d3d/f47 x:0 0 0 2024-04-26T04:52:57.696 INFO:tasks.workunit.client.1.smithi078.stdout:0/635: dwrite d2/d9/d31/d73/f83 [0,4194304] 0 2024-04-26T04:52:57.697 INFO:tasks.workunit.client.0.smithi077.stdout:7/276: mkdir da/d15/d61/d63/d64 0 2024-04-26T04:52:57.697 INFO:tasks.workunit.client.0.smithi077.stdout:9/200: mknod d9/dd/d1b/c48 0 2024-04-26T04:52:57.698 INFO:tasks.workunit.client.1.smithi078.stdout:9/553: dwrite dc/df/d27/d81/f91 [4194304,4194304] 0 2024-04-26T04:52:57.700 INFO:tasks.workunit.client.0.smithi077.stdout:1/346: fdatasync d9/db/d27/f28 0 2024-04-26T04:52:57.705 INFO:tasks.workunit.client.1.smithi078.stdout:8/552: mknod d0/d19/d52/d35/cb0 0 2024-04-26T04:52:57.706 INFO:tasks.workunit.client.0.smithi077.stdout:9/201: dwrite d9/f1c [0,4194304] 0 2024-04-26T04:52:57.708 INFO:tasks.workunit.client.1.smithi078.stdout:0/636: dwrite d2/d9/d18/d7d/da9/dab/dc6/f79 [0,4194304] 0 2024-04-26T04:52:57.708 INFO:tasks.workunit.client.0.smithi077.stdout:9/202: read d9/f10 [5754082,82113] 0 2024-04-26T04:52:57.709 INFO:tasks.workunit.client.1.smithi078.stdout:1/635: dread d7/d2e/f33 [4194304,4194304] 0 2024-04-26T04:52:57.710 INFO:tasks.workunit.client.1.smithi078.stdout:2/563: creat d8/d3e/d52/fca x:0 0 0 2024-04-26T04:52:57.710 INFO:tasks.workunit.client.1.smithi078.stdout:1/636: fsync d7/f13 0 2024-04-26T04:52:57.713 INFO:tasks.workunit.client.1.smithi078.stdout:3/620: creat d16/d26/d2e/d5f/d49/dad/db0/fdd x:0 0 0 2024-04-26T04:52:57.715 INFO:tasks.workunit.client.0.smithi077.stdout:9/203: symlink d9/dd/d28/l49 0 2024-04-26T04:52:57.727 INFO:tasks.workunit.client.1.smithi078.stdout:4/627: unlink d0/d1d/d46/cbd 0 2024-04-26T04:52:57.727 INFO:tasks.workunit.client.0.smithi077.stdout:7/277: fsync da/d13/f5b 0 2024-04-26T04:52:57.730 INFO:tasks.workunit.client.0.smithi077.stdout:9/204: dwrite d9/f10 [0,4194304] 0 2024-04-26T04:52:57.735 INFO:tasks.workunit.client.0.smithi077.stdout:7/278: rename da/d15/d1a/f56 to da/d15/d1a/f65 0 2024-04-26T04:52:57.736 INFO:tasks.workunit.client.1.smithi078.stdout:2/564: dread - d8/d11/d14/d69/d8f/d34/fad zero size 2024-04-26T04:52:57.737 INFO:tasks.workunit.client.1.smithi078.stdout:1/637: creat d7/d1e/d9e/db/d49/fd3 x:0 0 0 2024-04-26T04:52:57.739 INFO:tasks.workunit.client.0.smithi077.stdout:9/205: dwrite d9/f2e [0,4194304] 0 2024-04-26T04:52:57.739 INFO:tasks.workunit.client.0.smithi077.stdout:7/279: creat da/d13/d22/d5d/f66 x:0 0 0 2024-04-26T04:52:57.741 INFO:tasks.workunit.client.1.smithi078.stdout:4/628: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fe6 x:0 0 0 2024-04-26T04:52:57.741 INFO:tasks.workunit.client.0.smithi077.stdout:1/347: dread d9/db/d27/f30 [0,4194304] 0 2024-04-26T04:52:57.742 INFO:tasks.workunit.client.0.smithi077.stdout:9/206: rmdir d9/dd/d1b 39 2024-04-26T04:52:57.746 INFO:tasks.workunit.client.1.smithi078.stdout:2/565: rmdir d8/d11/d14 39 2024-04-26T04:52:57.748 INFO:tasks.workunit.client.1.smithi078.stdout:2/566: write d8/d3e/d52/fca [911893,46080] 0 2024-04-26T04:52:57.748 INFO:tasks.workunit.client.0.smithi077.stdout:1/348: symlink d9/db/d2b/d5c/l67 0 2024-04-26T04:52:57.751 INFO:tasks.workunit.client.1.smithi078.stdout:5/529: truncate d7/d11/f47 3323536 0 2024-04-26T04:52:57.751 INFO:tasks.workunit.client.1.smithi078.stdout:5/530: chown d7/d18/d1c/d5a 1146791 1 2024-04-26T04:52:57.752 INFO:tasks.workunit.client.1.smithi078.stdout:5/531: chown d7/d18/d1c/f92 76 1 2024-04-26T04:52:57.752 INFO:tasks.workunit.client.1.smithi078.stdout:1/638: rename d7/d1e/d9e/d4c/d3d/l8f to d7/d1e/d9e/db/d47/ld4 0 2024-04-26T04:52:57.753 INFO:tasks.workunit.client.0.smithi077.stdout:7/280: creat da/d15/d61/d63/d64/f67 x:0 0 0 2024-04-26T04:52:57.754 INFO:tasks.workunit.client.1.smithi078.stdout:5/532: chown d7/f39 340475925 1 2024-04-26T04:52:57.754 INFO:tasks.workunit.client.1.smithi078.stdout:1/639: chown d7/d1e/d9e/db/d37/d58/d5c/dc0/l44 12619 1 2024-04-26T04:52:57.754 INFO:tasks.workunit.client.1.smithi078.stdout:5/533: chown d7/d21/c78 4311081 1 2024-04-26T04:52:57.754 INFO:tasks.workunit.client.1.smithi078.stdout:8/553: write d0/d19/d25/d32/f76 [380269,73128] 0 2024-04-26T04:52:57.755 INFO:tasks.workunit.client.1.smithi078.stdout:8/554: readlink d0/d19/l22 0 2024-04-26T04:52:57.755 INFO:tasks.workunit.client.0.smithi077.stdout:1/349: unlink d9/d53/f5f 0 2024-04-26T04:52:57.756 INFO:tasks.workunit.client.1.smithi078.stdout:0/637: write d2/f21 [3588346,40164] 0 2024-04-26T04:52:57.756 INFO:tasks.workunit.client.0.smithi077.stdout:9/207: getdents d9/d43 0 2024-04-26T04:52:57.757 INFO:tasks.workunit.client.0.smithi077.stdout:9/208: readlink d9/dd/l24 0 2024-04-26T04:52:57.757 INFO:tasks.workunit.client.1.smithi078.stdout:0/638: chown d2/d9/d60/dae 0 1 2024-04-26T04:52:57.758 INFO:tasks.workunit.client.0.smithi077.stdout:7/281: creat da/f68 x:0 0 0 2024-04-26T04:52:57.759 INFO:tasks.workunit.client.1.smithi078.stdout:5/534: dwrite d7/da/f10 [4194304,4194304] 0 2024-04-26T04:52:57.759 INFO:tasks.workunit.client.1.smithi078.stdout:5/535: readlink d7/d48/d57/d65/d70/l7e 0 2024-04-26T04:52:57.763 INFO:tasks.workunit.client.1.smithi078.stdout:6/668: sync 2024-04-26T04:52:57.763 INFO:tasks.workunit.client.1.smithi078.stdout:8/555: rmdir d0/d19/d25/d32 39 2024-04-26T04:52:57.764 INFO:tasks.workunit.client.0.smithi077.stdout:9/209: dwrite d9/dd/d28/f33 [8388608,4194304] 0 2024-04-26T04:52:57.766 INFO:tasks.workunit.client.0.smithi077.stdout:7/282: mknod da/d15/d61/d63/d64/c69 0 2024-04-26T04:52:57.768 INFO:tasks.workunit.client.1.smithi078.stdout:3/621: truncate d16/d26/d2e/d5f/d49/fb4 3346544 0 2024-04-26T04:52:57.769 INFO:tasks.workunit.client.1.smithi078.stdout:3/622: fsync d16/d26/d2e/d5f/d49/f8f 0 2024-04-26T04:52:57.769 INFO:tasks.workunit.client.0.smithi077.stdout:1/350: mkdir d9/db/dd/d68 0 2024-04-26T04:52:57.773 INFO:tasks.workunit.client.1.smithi078.stdout:2/567: write d8/d3e/f58 [5038672,41859] 0 2024-04-26T04:52:57.773 INFO:tasks.workunit.client.0.smithi077.stdout:7/283: rename da/d15/d1a/l53 to da/d15/d1a/l6a 0 2024-04-26T04:52:57.776 INFO:tasks.workunit.client.1.smithi078.stdout:5/536: rename d7/f35 to d7/d18/d32/d84/fb2 0 2024-04-26T04:52:57.777 INFO:tasks.workunit.client.1.smithi078.stdout:2/568: read d8/d3e/f93 [378260,85525] 0 2024-04-26T04:52:57.777 INFO:tasks.workunit.client.1.smithi078.stdout:8/556: dwrite d0/d19/d25/d32/f76 [0,4194304] 0 2024-04-26T04:52:57.781 INFO:tasks.workunit.client.0.smithi077.stdout:7/284: truncate da/d13/d22/f2b 241616 0 2024-04-26T04:52:57.781 INFO:tasks.workunit.client.1.smithi078.stdout:3/623: unlink d16/d26/f62 0 2024-04-26T04:52:57.782 INFO:tasks.workunit.client.0.smithi077.stdout:7/285: write da/d15/f32 [537458,42494] 0 2024-04-26T04:52:57.783 INFO:tasks.workunit.client.1.smithi078.stdout:3/624: dread - d16/d26/d2b/fa9 zero size 2024-04-26T04:52:57.784 INFO:tasks.workunit.client.1.smithi078.stdout:3/625: chown d16/cd0 0 1 2024-04-26T04:52:57.785 INFO:tasks.workunit.client.1.smithi078.stdout:1/640: dwrite d7/d1e/d9e/dd/d20/f45 [0,4194304] 0 2024-04-26T04:52:57.788 INFO:tasks.workunit.client.0.smithi077.stdout:7/286: rmdir da/d15/d1a/d39 39 2024-04-26T04:52:57.790 INFO:tasks.workunit.client.0.smithi077.stdout:1/351: creat d9/db/dd/f69 x:0 0 0 2024-04-26T04:52:57.793 INFO:tasks.workunit.client.0.smithi077.stdout:9/210: getdents d9/dd/d3d 0 2024-04-26T04:52:57.794 INFO:tasks.workunit.client.0.smithi077.stdout:9/211: stat d9/f2e 0 2024-04-26T04:52:57.797 INFO:tasks.workunit.client.0.smithi077.stdout:7/287: rename da/d15/d61 to da/d6b 0 2024-04-26T04:52:57.798 INFO:tasks.workunit.client.0.smithi077.stdout:1/352: dwrite d9/f20 [4194304,4194304] 0 2024-04-26T04:52:57.799 INFO:tasks.workunit.client.0.smithi077.stdout:1/353: dread - d9/db/d2b/d3f/f4b zero size 2024-04-26T04:52:57.801 INFO:tasks.workunit.client.1.smithi078.stdout:2/569: symlink d8/d11/d18/dbb/lcb 0 2024-04-26T04:52:57.805 INFO:tasks.workunit.client.1.smithi078.stdout:6/669: truncate d2/de/d27/d58/f76 3202484 0 2024-04-26T04:52:57.808 INFO:tasks.workunit.client.1.smithi078.stdout:8/557: rename d0/d19/d25/d36/d58/d5b/d61/d67 to d0/d19/d25/d32/d8a/db1 0 2024-04-26T04:52:57.809 INFO:tasks.workunit.client.1.smithi078.stdout:3/626: dread d16/d26/d2e/f51 [0,4194304] 0 2024-04-26T04:52:57.825 INFO:tasks.workunit.client.0.smithi077.stdout:3/207: sync 2024-04-26T04:52:57.826 INFO:tasks.workunit.client.1.smithi078.stdout:1/641: rmdir d7/d1e/d9e/d92 39 2024-04-26T04:52:57.826 INFO:tasks.workunit.client.0.smithi077.stdout:9/212: rmdir d9/dd/d1b/d2b 39 2024-04-26T04:52:57.832 INFO:tasks.workunit.client.0.smithi077.stdout:1/354: symlink d9/db/dd/l6a 0 2024-04-26T04:52:57.833 INFO:tasks.workunit.client.0.smithi077.stdout:9/213: dwrite d9/dd/f42 [0,4194304] 0 2024-04-26T04:52:57.837 INFO:tasks.workunit.client.0.smithi077.stdout:1/355: dread d9/db/dd/d34/d4a/f59 [0,4194304] 0 2024-04-26T04:52:57.837 INFO:tasks.workunit.client.0.smithi077.stdout:7/288: link da/d13/c38 da/d6b/d63/c6c 0 2024-04-26T04:52:57.839 INFO:tasks.workunit.client.0.smithi077.stdout:7/289: truncate da/d13/d22/d34/f51 468284 0 2024-04-26T04:52:57.839 INFO:tasks.workunit.client.1.smithi078.stdout:6/670: symlink d2/d1a/lcb 0 2024-04-26T04:52:57.841 INFO:tasks.workunit.client.1.smithi078.stdout:2/570: dwrite d8/d11/d14/d69/d8f/f6c [0,4194304] 0 2024-04-26T04:52:57.843 INFO:tasks.workunit.client.1.smithi078.stdout:8/558: symlink d0/d19/d25/d32/d8a/d92/lb2 0 2024-04-26T04:52:57.844 INFO:tasks.workunit.client.1.smithi078.stdout:3/627: creat d16/d3c/d81/fde x:0 0 0 2024-04-26T04:52:57.847 INFO:tasks.workunit.client.0.smithi077.stdout:9/214: write d9/dd/d1b/d2b/f38 [397046,71138] 0 2024-04-26T04:52:57.849 INFO:tasks.workunit.client.1.smithi078.stdout:7/573: sync 2024-04-26T04:52:57.851 INFO:tasks.workunit.client.1.smithi078.stdout:7/574: chown d9/da/d11/d13/d1a/d65 6514365 1 2024-04-26T04:52:57.853 INFO:tasks.workunit.client.1.smithi078.stdout:6/671: symlink d2/de/d27/da9/d3a/d69/lcc 0 2024-04-26T04:52:57.854 INFO:tasks.workunit.client.0.smithi077.stdout:3/208: creat d0/f42 x:0 0 0 2024-04-26T04:52:57.855 INFO:tasks.workunit.client.1.smithi078.stdout:2/571: mknod d8/d11/d14/d69/d8f/d34/d70/ccc 0 2024-04-26T04:52:57.855 INFO:tasks.workunit.client.0.smithi077.stdout:9/215: dwrite d9/f1c [4194304,4194304] 0 2024-04-26T04:52:57.856 INFO:tasks.workunit.client.0.smithi077.stdout:3/209: chown d0/d7/d15/c17 12 1 2024-04-26T04:52:57.856 INFO:tasks.workunit.client.1.smithi078.stdout:8/559: readlink d0/d19/d52/d35/d44/d6b/d6c/l9a 0 2024-04-26T04:52:57.856 INFO:tasks.workunit.client.0.smithi077.stdout:3/210: fdatasync d0/d9/f2b 0 2024-04-26T04:52:57.859 INFO:tasks.workunit.client.1.smithi078.stdout:3/628: creat d16/d26/d2b/d31/d71/d8a/dbc/fdf x:0 0 0 2024-04-26T04:52:57.859 INFO:tasks.workunit.client.0.smithi077.stdout:3/211: write d0/d9/d32/d2a/f40 [1017447,12887] 0 2024-04-26T04:52:57.861 INFO:tasks.workunit.client.0.smithi077.stdout:9/216: dread f2 [8388608,4194304] 0 2024-04-26T04:52:57.863 INFO:tasks.workunit.client.1.smithi078.stdout:5/537: getdents d7 0 2024-04-26T04:52:57.863 INFO:tasks.workunit.client.0.smithi077.stdout:7/290: rmdir da/d15/d42/d4e 0 2024-04-26T04:52:57.863 INFO:tasks.workunit.client.1.smithi078.stdout:2/572: dwrite d8/d11/d14/d69/d8f/d34/d70/f75 [0,4194304] 0 2024-04-26T04:52:57.865 INFO:tasks.workunit.client.1.smithi078.stdout:5/538: write d7/d18/d1c/f9e [287402,114053] 0 2024-04-26T04:52:57.867 INFO:tasks.workunit.client.0.smithi077.stdout:1/356: dread d9/db/dd/d34/f4d [0,4194304] 0 2024-04-26T04:52:57.867 INFO:tasks.workunit.client.0.smithi077.stdout:3/212: write d0/d7/f3e [3953821,11221] 0 2024-04-26T04:52:57.868 INFO:tasks.workunit.client.1.smithi078.stdout:6/672: mkdir d2/de/dcd 0 2024-04-26T04:52:57.868 INFO:tasks.workunit.client.0.smithi077.stdout:3/213: fsync d0/d9/f2b 0 2024-04-26T04:52:57.869 INFO:tasks.workunit.client.0.smithi077.stdout:3/214: chown d0/d7/d15/f20 191139 1 2024-04-26T04:52:57.874 INFO:tasks.workunit.client.1.smithi078.stdout:1/642: creat d7/d1e/d9e/dd/fd5 x:0 0 0 2024-04-26T04:52:57.874 INFO:tasks.workunit.client.0.smithi077.stdout:9/217: creat d9/d45/f4a x:0 0 0 2024-04-26T04:52:57.874 INFO:tasks.workunit.client.1.smithi078.stdout:2/573: dwrite d8/d11/d14/d69/d8f/d34/d70/fac [0,4194304] 0 2024-04-26T04:52:57.875 INFO:tasks.workunit.client.1.smithi078.stdout:1/643: write d7/fc [3040661,59656] 0 2024-04-26T04:52:57.876 INFO:tasks.workunit.client.0.smithi077.stdout:1/357: mknod d9/db/d27/c6b 0 2024-04-26T04:52:57.878 INFO:tasks.workunit.client.0.smithi077.stdout:3/215: dread d0/d9/f25 [0,4194304] 0 2024-04-26T04:52:57.879 INFO:tasks.workunit.client.0.smithi077.stdout:9/218: dread d9/f17 [4194304,4194304] 0 2024-04-26T04:52:57.882 INFO:tasks.workunit.client.0.smithi077.stdout:1/358: dread d9/f20 [0,4194304] 0 2024-04-26T04:52:57.882 INFO:tasks.workunit.client.0.smithi077.stdout:7/291: rename da/d15/d1a/d39/f41 to da/d13/f6d 0 2024-04-26T04:52:57.883 INFO:tasks.workunit.client.1.smithi078.stdout:7/575: dwrite d9/da/d11/d13/d1a/d2d/f3f [0,4194304] 0 2024-04-26T04:52:57.884 INFO:tasks.workunit.client.0.smithi077.stdout:7/292: write da/f57 [170260,112410] 0 2024-04-26T04:52:57.886 INFO:tasks.workunit.client.1.smithi078.stdout:6/673: creat d2/de/d27/da9/d2a/dbf/fce x:0 0 0 2024-04-26T04:52:57.889 INFO:tasks.workunit.client.1.smithi078.stdout:9/554: sync 2024-04-26T04:52:57.889 INFO:tasks.workunit.client.1.smithi078.stdout:0/639: sync 2024-04-26T04:52:57.889 INFO:tasks.workunit.client.1.smithi078.stdout:4/629: sync 2024-04-26T04:52:57.893 INFO:tasks.workunit.client.0.smithi077.stdout:9/219: mknod d9/dd/d1b/d2b/d32/d36/c4b 0 2024-04-26T04:52:57.895 INFO:tasks.workunit.client.1.smithi078.stdout:9/555: dwrite dc/df/d27/f76 [0,4194304] 0 2024-04-26T04:52:57.897 INFO:tasks.workunit.client.1.smithi078.stdout:1/644: dread d7/d1e/d9e/dd/fa8 [0,4194304] 0 2024-04-26T04:52:57.898 INFO:tasks.workunit.client.1.smithi078.stdout:5/539: fsync d7/d21/f42 0 2024-04-26T04:52:57.899 INFO:tasks.workunit.client.0.smithi077.stdout:7/293: symlink da/d15/d42/l6e 0 2024-04-26T04:52:57.901 INFO:tasks.workunit.client.1.smithi078.stdout:8/560: creat d0/d19/d52/d5c/fb3 x:0 0 0 2024-04-26T04:52:57.906 INFO:tasks.workunit.client.0.smithi077.stdout:3/216: fsync d0/f19 0 2024-04-26T04:52:57.909 INFO:tasks.workunit.client.0.smithi077.stdout:9/220: rename d9/f10 to d9/d35/f4c 0 2024-04-26T04:52:57.911 INFO:tasks.workunit.client.1.smithi078.stdout:3/629: truncate d16/d26/d2b/d54/f58 553476 0 2024-04-26T04:52:57.916 INFO:tasks.workunit.client.1.smithi078.stdout:2/574: rename d8/d3e/d8e/caf to d8/d11/d18/dbb/ccd 0 2024-04-26T04:52:57.917 INFO:tasks.workunit.client.1.smithi078.stdout:0/640: dread d2/f12 [0,4194304] 0 2024-04-26T04:52:57.917 INFO:tasks.workunit.client.0.smithi077.stdout:3/217: creat d0/d7/d1f/f43 x:0 0 0 2024-04-26T04:52:57.918 INFO:tasks.workunit.client.0.smithi077.stdout:3/218: write d0/d9/d32/f21 [483261,46603] 0 2024-04-26T04:52:57.919 INFO:tasks.workunit.client.0.smithi077.stdout:1/359: truncate d9/f21 1310694 0 2024-04-26T04:52:57.921 INFO:tasks.workunit.client.0.smithi077.stdout:7/294: mkdir da/d15/d1a/d6f 0 2024-04-26T04:52:57.921 INFO:tasks.workunit.client.1.smithi078.stdout:5/540: stat d7/d21/d2f/l80 0 2024-04-26T04:52:57.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/541: fsync d7/d18/d32/d37/f51 0 2024-04-26T04:52:57.922 INFO:tasks.workunit.client.0.smithi077.stdout:3/219: creat d0/d7/d15/f44 x:0 0 0 2024-04-26T04:52:57.923 INFO:tasks.workunit.client.1.smithi078.stdout:8/561: write d0/fae [976807,80364] 0 2024-04-26T04:52:57.923 INFO:tasks.workunit.client.0.smithi077.stdout:3/220: truncate d0/fb 1596945 0 2024-04-26T04:52:57.923 INFO:tasks.workunit.client.0.smithi077.stdout:9/221: unlink d9/dd/c1e 0 2024-04-26T04:52:57.925 INFO:tasks.workunit.client.1.smithi078.stdout:7/576: symlink d9/da/d11/d13/d1a/d52/d5a/lb9 0 2024-04-26T04:52:57.925 INFO:tasks.workunit.client.1.smithi078.stdout:8/562: fdatasync d0/fae 0 2024-04-26T04:52:57.926 INFO:tasks.workunit.client.0.smithi077.stdout:1/360: mkdir d9/db/dd/d34/d4a/d6c 0 2024-04-26T04:52:57.927 INFO:tasks.workunit.client.1.smithi078.stdout:6/674: stat d2/d1a/f9b 0 2024-04-26T04:52:57.928 INFO:tasks.workunit.client.0.smithi077.stdout:7/295: creat da/d6b/d63/f70 x:0 0 0 2024-04-26T04:52:57.928 INFO:tasks.workunit.client.1.smithi078.stdout:5/542: dwrite d7/d18/d1c/f7d [0,4194304] 0 2024-04-26T04:52:57.929 INFO:tasks.workunit.client.0.smithi077.stdout:9/222: write f3 [13254619,111270] 0 2024-04-26T04:52:57.929 INFO:tasks.workunit.client.1.smithi078.stdout:4/630: mkdir d0/de/d22/d27/de7 0 2024-04-26T04:52:57.930 INFO:tasks.workunit.client.0.smithi077.stdout:9/223: dread - d9/dd/d1b/d2b/f2f zero size 2024-04-26T04:52:57.931 INFO:tasks.workunit.client.0.smithi077.stdout:1/361: chown d9/db/d2b/d5c/c64 236478366 1 2024-04-26T04:52:57.932 INFO:tasks.workunit.client.1.smithi078.stdout:3/630: creat d16/d3c/d81/fe0 x:0 0 0 2024-04-26T04:52:57.932 INFO:tasks.workunit.client.1.smithi078.stdout:2/575: mkdir d8/d3e/dce 0 2024-04-26T04:52:57.933 INFO:tasks.workunit.client.0.smithi077.stdout:7/296: dread da/d13/d22/d34/f5a [0,4194304] 0 2024-04-26T04:52:57.934 INFO:tasks.workunit.client.1.smithi078.stdout:9/556: link dc/f39 dc/df/d4b/d22/d2a/d92/fa3 0 2024-04-26T04:52:57.936 INFO:tasks.workunit.client.0.smithi077.stdout:1/362: dread d9/db/dd/f5b [0,4194304] 0 2024-04-26T04:52:57.938 INFO:tasks.workunit.client.0.smithi077.stdout:1/363: write d9/f1d [660738,21102] 0 2024-04-26T04:52:57.940 INFO:tasks.workunit.client.0.smithi077.stdout:9/224: dwrite d9/f22 [0,4194304] 0 2024-04-26T04:52:57.948 INFO:tasks.workunit.client.1.smithi078.stdout:7/577: dread d9/da/f23 [4194304,4194304] 0 2024-04-26T04:52:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:57 smithi077 ceph-mon[26311]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 177 MiB/s wr, 279 op/s 2024-04-26T04:52:57.995 INFO:tasks.workunit.client.0.smithi077.stdout:8/256: sync 2024-04-26T04:52:57.995 INFO:tasks.workunit.client.0.smithi077.stdout:4/257: sync 2024-04-26T04:52:57.998 INFO:tasks.workunit.client.0.smithi077.stdout:4/258: dread d4/d17/d18/d43/d53/d56/f1b [0,4194304] 0 2024-04-26T04:52:58.008 INFO:tasks.workunit.client.0.smithi077.stdout:7/297: creat da/d15/d1a/d39/f71 x:0 0 0 2024-04-26T04:52:58.009 INFO:tasks.workunit.client.1.smithi078.stdout:1/645: symlink d7/d1e/d9e/d4c/ld6 0 2024-04-26T04:52:58.010 INFO:tasks.workunit.client.1.smithi078.stdout:1/646: readlink d7/d1e/d9e/db/d47/l74 0 2024-04-26T04:52:58.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:57 smithi078 ceph-mon[33746]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 177 MiB/s wr, 279 op/s 2024-04-26T04:52:58.019 INFO:tasks.workunit.client.1.smithi078.stdout:8/563: mkdir d0/d19/d52/d35/d44/d6b/d6c/db4 0 2024-04-26T04:52:58.022 INFO:tasks.workunit.client.1.smithi078.stdout:6/675: rename d2/de/d27/da9/da/dab/cbb to d2/de/d27/da9/d2a/dbf/d5f/ccf 0 2024-04-26T04:52:58.026 INFO:tasks.workunit.client.1.smithi078.stdout:8/564: dwrite d0/d19/d25/d36/d58/d5b/d61/f8c [4194304,4194304] 0 2024-04-26T04:52:58.027 INFO:tasks.workunit.client.0.smithi077.stdout:1/364: creat d9/db/d2b/d5c/f6d x:0 0 0 2024-04-26T04:52:58.028 INFO:tasks.workunit.client.1.smithi078.stdout:8/565: write d0/d19/d25/d36/f7f [101029,104794] 0 2024-04-26T04:52:58.028 INFO:tasks.workunit.client.1.smithi078.stdout:4/631: symlink d0/d1d/de1/d20/d31/d47/db2/le8 0 2024-04-26T04:52:58.030 INFO:tasks.workunit.client.0.smithi077.stdout:9/225: symlink d9/d45/l4d 0 2024-04-26T04:52:58.030 INFO:tasks.workunit.client.0.smithi077.stdout:7/298: dread da/d13/f30 [0,4194304] 0 2024-04-26T04:52:58.031 INFO:tasks.workunit.client.0.smithi077.stdout:9/226: chown d9/dd/d1b/f23 133534198 1 2024-04-26T04:52:58.031 INFO:tasks.workunit.client.1.smithi078.stdout:3/631: rmdir d16/d26/d2b/d31/d78 39 2024-04-26T04:52:58.032 INFO:tasks.workunit.client.1.smithi078.stdout:4/632: chown d0/d1d/de1/d20/d31/d47/db2/d44/d55/d5e/d75/fd1 1828 1 2024-04-26T04:52:58.034 INFO:tasks.workunit.client.1.smithi078.stdout:2/576: creat d8/d11/d14/d69/d8f/d34/d70/fcf x:0 0 0 2024-04-26T04:52:58.036 INFO:tasks.workunit.client.0.smithi077.stdout:8/257: dread d4/f40 [0,4194304] 0 2024-04-26T04:52:58.039 INFO:tasks.workunit.client.1.smithi078.stdout:7/578: mknod d9/da/d33/d4b/cba 0 2024-04-26T04:52:58.040 INFO:tasks.workunit.client.0.smithi077.stdout:7/299: dread da/d15/d1a/f20 [0,4194304] 0 2024-04-26T04:52:58.043 INFO:tasks.workunit.client.0.smithi077.stdout:8/258: dwrite d4/df/d22/f34 [0,4194304] 0 2024-04-26T04:52:58.045 INFO:tasks.workunit.client.1.smithi078.stdout:0/641: dwrite d2/d9/d31/d73/f90 [0,4194304] 0 2024-04-26T04:52:58.051 INFO:tasks.workunit.client.1.smithi078.stdout:9/557: rename dc/f94 to dc/df/d4b/d22/d2a/d57/fa4 0 2024-04-26T04:52:58.054 INFO:tasks.workunit.client.0.smithi077.stdout:9/227: truncate d9/f17 5770743 0 2024-04-26T04:52:58.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/676: dread d2/de/d27/da9/da/d35/f3e [0,4194304] 0 2024-04-26T04:52:58.057 INFO:tasks.workunit.client.0.smithi077.stdout:7/300: mknod da/d13/c72 0 2024-04-26T04:52:58.060 INFO:tasks.workunit.client.1.smithi078.stdout:4/633: mknod d0/d1d/ce9 0 2024-04-26T04:52:58.063 INFO:tasks.workunit.client.0.smithi077.stdout:4/259: creat d4/d17/f57 x:0 0 0 2024-04-26T04:52:58.064 INFO:tasks.workunit.client.1.smithi078.stdout:7/579: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/fbb x:0 0 0 2024-04-26T04:52:58.066 INFO:tasks.workunit.client.0.smithi077.stdout:7/301: symlink da/d13/l73 0 2024-04-26T04:52:58.069 INFO:tasks.workunit.client.0.smithi077.stdout:8/259: mkdir d4/d17/d35/d5b 0 2024-04-26T04:52:58.069 INFO:tasks.workunit.client.1.smithi078.stdout:7/580: dwrite d9/f14 [0,4194304] 0 2024-04-26T04:52:58.070 INFO:tasks.workunit.client.1.smithi078.stdout:0/642: symlink d2/d9/d78/d84/ld4 0 2024-04-26T04:52:58.070 INFO:tasks.workunit.client.0.smithi077.stdout:8/260: write d4/df/d22/f50 [164593,76019] 0 2024-04-26T04:52:58.074 INFO:tasks.workunit.client.1.smithi078.stdout:1/647: dwrite d7/d1e/d9e/d4c/f4e [0,4194304] 0 2024-04-26T04:52:58.075 INFO:tasks.workunit.client.1.smithi078.stdout:5/543: rename d7/d48/d57/d65/d46/l49 to d7/da/d1e/lb3 0 2024-04-26T04:52:58.076 INFO:tasks.workunit.client.1.smithi078.stdout:5/544: write d7/d48/d57/fab [2222967,16618] 0 2024-04-26T04:52:58.077 INFO:tasks.workunit.client.0.smithi077.stdout:4/260: mkdir d4/d17/d18/d43/d53/d58 0 2024-04-26T04:52:58.084 INFO:tasks.workunit.client.0.smithi077.stdout:8/261: mknod d4/d17/c5c 0 2024-04-26T04:52:58.084 INFO:tasks.workunit.client.0.smithi077.stdout:8/262: fsync d4/d17/f3c 0 2024-04-26T04:52:58.088 INFO:tasks.workunit.client.1.smithi078.stdout:6/677: creat d2/de/d27/da9/da/d36/db9/fd0 x:0 0 0 2024-04-26T04:52:58.090 INFO:tasks.workunit.client.1.smithi078.stdout:3/632: mknod d16/d26/d2b/d31/d78/dd1/ce1 0 2024-04-26T04:52:58.090 INFO:tasks.workunit.client.1.smithi078.stdout:3/633: readlink d16/d26/l9c 0 2024-04-26T04:52:58.092 INFO:tasks.workunit.client.0.smithi077.stdout:4/261: mknod d4/c59 0 2024-04-26T04:52:58.092 INFO:tasks.workunit.client.0.smithi077.stdout:8/263: dwrite d4/df/d22/d2d/d2b/d33/f4e [0,4194304] 0 2024-04-26T04:52:58.093 INFO:tasks.workunit.client.1.smithi078.stdout:4/634: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/lea 0 2024-04-26T04:52:58.093 INFO:tasks.workunit.client.0.smithi077.stdout:4/262: chown d4/de/d15/d1f/d25/c55 903117075 1 2024-04-26T04:52:58.097 INFO:tasks.workunit.client.1.smithi078.stdout:2/577: creat d8/d11/d14/d5b/fd0 x:0 0 0 2024-04-26T04:52:58.100 INFO:tasks.workunit.client.0.smithi077.stdout:4/263: fsync d4/de/f2b 0 2024-04-26T04:52:58.105 INFO:tasks.workunit.client.1.smithi078.stdout:9/558: dwrite dc/df/d4b/d22/d2a/d92/f71 [0,4194304] 0 2024-04-26T04:52:58.109 INFO:tasks.workunit.client.0.smithi077.stdout:8/264: rename d4/df/d22/f34 to d4/df/d22/d2d/d2b/f5d 0 2024-04-26T04:52:58.109 INFO:tasks.workunit.client.0.smithi077.stdout:9/228: dwrite d9/d35/f4c [0,4194304] 0 2024-04-26T04:52:58.114 INFO:tasks.workunit.client.0.smithi077.stdout:4/264: creat d4/d4e/f5a x:0 0 0 2024-04-26T04:52:58.118 INFO:tasks.workunit.client.0.smithi077.stdout:9/229: dwrite d9/dd/d3d/f47 [0,4194304] 0 2024-04-26T04:52:58.118 INFO:tasks.workunit.client.0.smithi077.stdout:9/230: fdatasync d9/f1c 0 2024-04-26T04:52:58.120 INFO:tasks.workunit.client.0.smithi077.stdout:9/231: write f3 [6778670,22190] 0 2024-04-26T04:52:58.187 INFO:tasks.workunit.client.0.smithi077.stdout:5/288: sync 2024-04-26T04:52:58.187 INFO:tasks.workunit.client.0.smithi077.stdout:5/289: chown d1/d12/d21/d25/l26 3067 1 2024-04-26T04:52:58.188 INFO:tasks.workunit.client.1.smithi078.stdout:8/566: creat d0/d19/d52/d5c/fb5 x:0 0 0 2024-04-26T04:52:58.188 INFO:tasks.workunit.client.1.smithi078.stdout:8/567: fdatasync d0/d19/d52/f69 0 2024-04-26T04:52:58.195 INFO:tasks.workunit.client.0.smithi077.stdout:5/290: dwrite d1/d8/fc [4194304,4194304] 0 2024-04-26T04:52:58.197 INFO:tasks.workunit.client.1.smithi078.stdout:5/545: dwrite d7/d11/f66 [0,4194304] 0 2024-04-26T04:52:58.198 INFO:tasks.workunit.client.1.smithi078.stdout:5/546: chown d7/d48/d57/f89 197502 1 2024-04-26T04:52:58.199 INFO:tasks.workunit.client.1.smithi078.stdout:1/648: mkdir d7/dd7 0 2024-04-26T04:52:58.203 INFO:tasks.workunit.client.0.smithi077.stdout:5/291: creat d1/d8/f70 x:0 0 0 2024-04-26T04:52:58.204 INFO:tasks.workunit.client.0.smithi077.stdout:9/232: mkdir d9/dd/d1b/d4e 0 2024-04-26T04:52:58.206 INFO:tasks.workunit.client.0.smithi077.stdout:9/233: write d9/f2e [3189916,115070] 0 2024-04-26T04:52:58.209 INFO:tasks.workunit.client.0.smithi077.stdout:4/265: link d4/d17/f3d d4/de/d15/f5b 0 2024-04-26T04:52:58.210 INFO:tasks.workunit.client.1.smithi078.stdout:8/568: read d0/f23 [40370,113791] 0 2024-04-26T04:52:58.210 INFO:tasks.workunit.client.1.smithi078.stdout:3/634: mkdir d16/d26/d2e/d5f/d49/dcc/de2 0 2024-04-26T04:52:58.213 INFO:tasks.workunit.client.0.smithi077.stdout:8/265: link d4/c20 d4/d17/c5e 0 2024-04-26T04:52:58.217 INFO:tasks.workunit.client.0.smithi077.stdout:9/234: rename d9/dd/d1b/d2b/d32/d36/c4b to d9/dd/d1b/d2b/c4f 0 2024-04-26T04:52:58.219 INFO:tasks.workunit.client.1.smithi078.stdout:5/547: unlink d7/f39 0 2024-04-26T04:52:58.221 INFO:tasks.workunit.client.1.smithi078.stdout:1/649: creat d7/d1e/d9e/db/d49/fd8 x:0 0 0 2024-04-26T04:52:58.223 INFO:tasks.workunit.client.0.smithi077.stdout:8/266: creat d4/df/d22/d2d/f5f x:0 0 0 2024-04-26T04:52:58.224 INFO:tasks.workunit.client.1.smithi078.stdout:7/581: rename d9/da/d11/d13/d1a/f4e to d9/fbc 0 2024-04-26T04:52:58.228 INFO:tasks.workunit.client.1.smithi078.stdout:9/559: unlink dc/df/d4b/d3e/d80/l8d 0 2024-04-26T04:52:58.228 INFO:tasks.workunit.client.1.smithi078.stdout:4/635: write d0/de/d22/d27/d56/f5c [2750909,39403] 0 2024-04-26T04:52:58.229 INFO:tasks.workunit.client.0.smithi077.stdout:8/267: dwrite d4/d17/d35/f52 [0,4194304] 0 2024-04-26T04:52:58.229 INFO:tasks.workunit.client.1.smithi078.stdout:4/636: write d0/f13 [2286839,18432] 0 2024-04-26T04:52:58.230 INFO:tasks.workunit.client.0.smithi077.stdout:5/292: creat d1/d12/d21/d25/d31/f71 x:0 0 0 2024-04-26T04:52:58.236 INFO:tasks.workunit.client.1.smithi078.stdout:6/678: link d2/de/d27/da9/d3a/f62 d2/de/d27/da9/d2a/d4d/fd1 0 2024-04-26T04:52:58.236 INFO:tasks.workunit.client.0.smithi077.stdout:4/266: dwrite d4/d17/f3d [0,4194304] 0 2024-04-26T04:52:58.238 INFO:tasks.workunit.client.1.smithi078.stdout:9/560: dwrite dc/df/f1e [0,4194304] 0 2024-04-26T04:52:58.242 INFO:tasks.workunit.client.0.smithi077.stdout:4/267: mkdir d4/de/d15/d1f/d35/d5c 0 2024-04-26T04:52:58.243 INFO:tasks.workunit.client.1.smithi078.stdout:0/643: getdents d2/d9/d18/db7/d92 0 2024-04-26T04:52:58.245 INFO:tasks.workunit.client.0.smithi077.stdout:9/235: mkdir d9/dd/d28/d50 0 2024-04-26T04:52:58.246 INFO:tasks.workunit.client.0.smithi077.stdout:9/236: chown d9/dd/d1b/d2b/d32/d36 1803344 1 2024-04-26T04:52:58.249 INFO:tasks.workunit.client.0.smithi077.stdout:4/268: rename d4/de/f26 to d4/d17/d18/f5d 0 2024-04-26T04:52:58.250 INFO:tasks.workunit.client.1.smithi078.stdout:1/650: creat d7/d1e/d9e/db/d49/d93/fd9 x:0 0 0 2024-04-26T04:52:58.251 INFO:tasks.workunit.client.0.smithi077.stdout:9/237: mknod d9/c51 0 2024-04-26T04:52:58.256 INFO:tasks.workunit.client.0.smithi077.stdout:9/238: fdatasync d9/dd/d28/f33 0 2024-04-26T04:52:58.259 INFO:tasks.workunit.client.0.smithi077.stdout:4/269: dwrite d4/f14 [0,4194304] 0 2024-04-26T04:52:58.259 INFO:tasks.workunit.client.1.smithi078.stdout:9/561: creat dc/df/d27/d81/fa5 x:0 0 0 2024-04-26T04:52:58.261 INFO:tasks.workunit.client.0.smithi077.stdout:9/239: fsync d9/dd/d1b/d2b/d32/f39 0 2024-04-26T04:52:58.261 INFO:tasks.workunit.client.1.smithi078.stdout:2/578: getdents d8/d11/d18/d5d/d8b 0 2024-04-26T04:52:58.267 INFO:tasks.workunit.client.0.smithi077.stdout:4/270: dread d4/f27 [0,4194304] 0 2024-04-26T04:52:58.271 INFO:tasks.workunit.client.1.smithi078.stdout:4/637: mkdir d0/de/d22/deb 0 2024-04-26T04:52:58.272 INFO:tasks.workunit.client.1.smithi078.stdout:3/635: write d16/f3f [6461820,103846] 0 2024-04-26T04:52:58.273 INFO:tasks.workunit.client.0.smithi077.stdout:4/271: dwrite d4/de/d15/d1f/f32 [4194304,4194304] 0 2024-04-26T04:52:58.275 INFO:tasks.workunit.client.0.smithi077.stdout:9/240: rename d9/dd/f42 to d9/dd/f52 0 2024-04-26T04:52:58.280 INFO:tasks.workunit.client.0.smithi077.stdout:4/272: creat d4/de/d15/d1f/f5e x:0 0 0 2024-04-26T04:52:58.283 INFO:tasks.workunit.client.1.smithi078.stdout:5/548: fdatasync d7/f38 0 2024-04-26T04:52:58.284 INFO:tasks.workunit.client.1.smithi078.stdout:6/679: dread d2/de/d27/f64 [0,4194304] 0 2024-04-26T04:52:58.285 INFO:tasks.workunit.client.1.smithi078.stdout:1/651: symlink d7/d1e/d9e/d4c/d3d/lda 0 2024-04-26T04:52:58.286 INFO:tasks.workunit.client.1.smithi078.stdout:9/562: write dc/df/f9f [850941,20805] 0 2024-04-26T04:52:58.287 INFO:tasks.workunit.client.1.smithi078.stdout:1/652: readlink d7/d1e/d9e/db/d37/d58/d5c/dc0/l27 0 2024-04-26T04:52:58.293 INFO:tasks.workunit.client.1.smithi078.stdout:3/636: rename d16/d26/d2b/f72 to d16/d26/d2b/dd6/fe3 0 2024-04-26T04:52:58.294 INFO:tasks.workunit.client.1.smithi078.stdout:3/637: readlink d16/d26/d2e/d74/lb5 0 2024-04-26T04:52:58.296 INFO:tasks.workunit.client.0.smithi077.stdout:4/273: rename d4/l4a to d4/l5f 0 2024-04-26T04:52:58.299 INFO:tasks.workunit.client.0.smithi077.stdout:9/241: truncate f3 5340251 0 2024-04-26T04:52:58.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/274: mknod d4/de/c60 0 2024-04-26T04:52:58.303 INFO:tasks.workunit.client.1.smithi078.stdout:9/563: rmdir dc 39 2024-04-26T04:52:58.304 INFO:tasks.workunit.client.1.smithi078.stdout:1/653: creat d7/d1e/d9e/db/d49/d93/fdb x:0 0 0 2024-04-26T04:52:58.305 INFO:tasks.workunit.client.1.smithi078.stdout:7/582: link d9/da/d11/d13/c48 d9/da/d11/d13/cbd 0 2024-04-26T04:52:58.307 INFO:tasks.workunit.client.1.smithi078.stdout:2/579: truncate d8/f1f 1200236 0 2024-04-26T04:52:58.308 INFO:tasks.workunit.client.0.smithi077.stdout:9/242: link d9/f46 d9/d35/f53 0 2024-04-26T04:52:58.316 INFO:tasks.workunit.client.0.smithi077.stdout:9/243: creat d9/d3a/f54 x:0 0 0 2024-04-26T04:52:58.318 INFO:tasks.workunit.client.1.smithi078.stdout:0/644: getdents d2/d9/d18 0 2024-04-26T04:52:58.320 INFO:tasks.workunit.client.0.smithi077.stdout:9/244: rename d9/c51 to d9/d43/c55 0 2024-04-26T04:52:58.321 INFO:tasks.workunit.client.0.smithi077.stdout:9/245: fdatasync d9/d35/f4c 0 2024-04-26T04:52:58.322 INFO:tasks.workunit.client.1.smithi078.stdout:9/564: chown dc/df/d4b/d22/l51 1523 1 2024-04-26T04:52:58.323 INFO:tasks.workunit.client.0.smithi077.stdout:4/275: truncate d4/f14 3697301 0 2024-04-26T04:52:58.325 INFO:tasks.workunit.client.0.smithi077.stdout:9/246: creat d9/d43/f56 x:0 0 0 2024-04-26T04:52:58.326 INFO:tasks.workunit.client.0.smithi077.stdout:9/247: stat d9/dd/d1b/c48 0 2024-04-26T04:52:58.327 INFO:tasks.workunit.client.0.smithi077.stdout:9/248: truncate d9/f11 4813062 0 2024-04-26T04:52:58.327 INFO:tasks.workunit.client.1.smithi078.stdout:4/638: creat d0/d1d/de1/d20/d31/d47/db2/d44/d55/fec x:0 0 0 2024-04-26T04:52:58.328 INFO:tasks.workunit.client.0.smithi077.stdout:9/249: fdatasync d9/d45/f4a 0 2024-04-26T04:52:58.329 INFO:tasks.workunit.client.1.smithi078.stdout:2/580: symlink d8/d3e/ld1 0 2024-04-26T04:52:58.329 INFO:tasks.workunit.client.1.smithi078.stdout:2/581: write d8/d11/f31 [8277918,57874] 0 2024-04-26T04:52:58.331 INFO:tasks.workunit.client.1.smithi078.stdout:2/582: write d8/d53/f7a [384518,104503] 0 2024-04-26T04:52:58.331 INFO:tasks.workunit.client.0.smithi077.stdout:4/276: fsync d4/f39 0 2024-04-26T04:52:58.333 INFO:tasks.workunit.client.0.smithi077.stdout:9/250: creat d9/dd/d1b/d2b/f57 x:0 0 0 2024-04-26T04:52:58.334 INFO:tasks.workunit.client.1.smithi078.stdout:0/645: dread d2/d9/d31/d4a/f82 [0,4194304] 0 2024-04-26T04:52:58.334 INFO:tasks.workunit.client.1.smithi078.stdout:5/549: mknod d7/d48/da9/cb4 0 2024-04-26T04:52:58.336 INFO:tasks.workunit.client.1.smithi078.stdout:9/565: unlink dc/df/d27/fa1 0 2024-04-26T04:52:58.337 INFO:tasks.workunit.client.1.smithi078.stdout:9/566: chown dc/df/d4b/d22 182568450 1 2024-04-26T04:52:58.337 INFO:tasks.workunit.client.0.smithi077.stdout:4/277: chown d4/cc 1964124 1 2024-04-26T04:52:58.337 INFO:tasks.workunit.client.1.smithi078.stdout:9/567: dread - dc/df/d17/f88 zero size 2024-04-26T04:52:58.338 INFO:tasks.workunit.client.0.smithi077.stdout:4/278: chown d4/cc 2 1 2024-04-26T04:52:58.338 INFO:tasks.workunit.client.1.smithi078.stdout:9/568: write dc/df/f2f [671179,17897] 0 2024-04-26T04:52:58.343 INFO:tasks.workunit.client.1.smithi078.stdout:7/583: mknod d9/da/d11/d13/d1a/d52/d5a/d41/d95/cbe 0 2024-04-26T04:52:58.345 INFO:tasks.workunit.client.1.smithi078.stdout:3/638: write d16/d26/d2b/d54/f58 [1377671,68939] 0 2024-04-26T04:52:58.347 INFO:tasks.workunit.client.1.smithi078.stdout:4/639: creat d0/de/d22/d71/fed x:0 0 0 2024-04-26T04:52:58.352 INFO:tasks.workunit.client.1.smithi078.stdout:2/583: fdatasync d8/d11/d14/d69/d8f/f5c 0 2024-04-26T04:52:58.352 INFO:tasks.workunit.client.1.smithi078.stdout:1/654: write d7/f57 [204573,2340] 0 2024-04-26T04:52:58.354 INFO:tasks.workunit.client.1.smithi078.stdout:1/655: fsync d7/d1e/d9e/db/d49/fb7 0 2024-04-26T04:52:58.357 INFO:tasks.workunit.client.0.smithi077.stdout:4/279: truncate d4/d17/f3b 3383065 0 2024-04-26T04:52:58.358 INFO:tasks.workunit.client.1.smithi078.stdout:7/584: creat d9/da/d11/d13/d1a/d2d/fbf x:0 0 0 2024-04-26T04:52:58.359 INFO:tasks.workunit.client.1.smithi078.stdout:5/550: dwrite d7/d18/d1c/d5a/f9d [0,4194304] 0 2024-04-26T04:52:58.359 INFO:tasks.workunit.client.1.smithi078.stdout:3/639: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/de4 0 2024-04-26T04:52:58.360 INFO:tasks.workunit.client.0.smithi077.stdout:9/251: dwrite d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:52:58.361 INFO:tasks.workunit.client.0.smithi077.stdout:6/295: sync 2024-04-26T04:52:58.362 INFO:tasks.workunit.client.1.smithi078.stdout:3/640: chown d16/d26/d2b/d31/d71/d8a/d98/fa1 20995 1 2024-04-26T04:52:58.365 INFO:tasks.workunit.client.0.smithi077.stdout:4/280: unlink d4/d17/f24 0 2024-04-26T04:52:58.365 INFO:tasks.workunit.client.1.smithi078.stdout:7/585: dwrite d9/da/d11/fad [0,4194304] 0 2024-04-26T04:52:58.367 INFO:tasks.workunit.client.1.smithi078.stdout:4/640: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/d69/f7a [0,4194304] 0 2024-04-26T04:52:58.368 INFO:tasks.workunit.client.0.smithi077.stdout:9/252: dread d9/f1c [0,4194304] 0 2024-04-26T04:52:58.369 INFO:tasks.workunit.client.1.smithi078.stdout:2/584: creat d8/d11/d6e/fd2 x:0 0 0 2024-04-26T04:52:58.372 INFO:tasks.workunit.client.1.smithi078.stdout:2/585: write d8/d3e/d8e/d91/fb2 [671709,17498] 0 2024-04-26T04:52:58.372 INFO:tasks.workunit.client.0.smithi077.stdout:6/296: dwrite d8/f22 [0,4194304] 0 2024-04-26T04:52:58.376 INFO:tasks.workunit.client.1.smithi078.stdout:7/586: dread d9/da/d11/d13/d1a/d52/d5a/d41/f47 [0,4194304] 0 2024-04-26T04:52:58.376 INFO:tasks.workunit.client.0.smithi077.stdout:4/281: rmdir d4/de 39 2024-04-26T04:52:58.377 INFO:tasks.workunit.client.1.smithi078.stdout:7/587: fsync f6 0 2024-04-26T04:52:58.377 INFO:tasks.workunit.client.1.smithi078.stdout:4/641: dwrite d0/d1d/de1/d20/f23 [0,4194304] 0 2024-04-26T04:52:58.378 INFO:tasks.workunit.client.0.smithi077.stdout:9/253: unlink d9/d3a/f54 0 2024-04-26T04:52:58.379 INFO:tasks.workunit.client.0.smithi077.stdout:6/297: dread d8/d1c/f28 [0,4194304] 0 2024-04-26T04:52:58.381 INFO:tasks.workunit.client.0.smithi077.stdout:4/282: truncate d4/d4e/f5a 383896 0 2024-04-26T04:52:58.385 INFO:tasks.workunit.client.1.smithi078.stdout:1/656: dread d7/d2e/f33 [0,4194304] 0 2024-04-26T04:52:58.385 INFO:tasks.workunit.client.0.smithi077.stdout:9/254: dread d9/dd/d3d/f47 [0,4194304] 0 2024-04-26T04:52:58.386 INFO:tasks.workunit.client.1.smithi078.stdout:1/657: write d7/d1e/f54 [2365988,78829] 0 2024-04-26T04:52:58.387 INFO:tasks.workunit.client.0.smithi077.stdout:9/255: write d9/dd/d1b/f23 [1690086,96951] 0 2024-04-26T04:52:58.387 INFO:tasks.workunit.client.1.smithi078.stdout:1/658: write d7/d1e/d9e/dd/d20/f50 [653979,71460] 0 2024-04-26T04:52:58.392 INFO:tasks.workunit.client.0.smithi077.stdout:6/298: rename d8/c16 to d8/d52/d53/c67 0 2024-04-26T04:52:58.398 INFO:tasks.workunit.client.1.smithi078.stdout:7/588: mkdir d9/da/d33/d4b/dc0 0 2024-04-26T04:52:58.400 INFO:tasks.workunit.client.0.smithi077.stdout:6/299: dwrite d8/f43 [4194304,4194304] 0 2024-04-26T04:52:58.404 INFO:tasks.workunit.client.1.smithi078.stdout:8/569: sync 2024-04-26T04:52:58.404 INFO:tasks.workunit.client.1.smithi078.stdout:4/642: creat d0/de/d22/d27/de7/fee x:0 0 0 2024-04-26T04:52:58.407 INFO:tasks.workunit.client.1.smithi078.stdout:7/589: rmdir d9/da/d11/d13/d1a/d52/d5a 39 2024-04-26T04:52:58.408 INFO:tasks.workunit.client.1.smithi078.stdout:3/641: creat d16/d26/d2b/d31/fe5 x:0 0 0 2024-04-26T04:52:58.409 INFO:tasks.workunit.client.1.smithi078.stdout:2/586: rmdir d8/d3e/dce 0 2024-04-26T04:52:58.410 INFO:tasks.workunit.client.1.smithi078.stdout:3/642: chown d16/d26/d2b/d54/cda 137236020 1 2024-04-26T04:52:58.411 INFO:tasks.workunit.client.1.smithi078.stdout:4/643: dread d0/d1d/de1/d20/d31/d47/db2/d44/f68 [0,4194304] 0 2024-04-26T04:52:58.413 INFO:tasks.workunit.client.1.smithi078.stdout:5/551: getdents d7/d21/d2f 0 2024-04-26T04:52:58.414 INFO:tasks.workunit.client.1.smithi078.stdout:5/552: stat d7/d48/d57/d65/f5b 0 2024-04-26T04:52:58.414 INFO:tasks.workunit.client.1.smithi078.stdout:3/643: rename d16/d26/d2b/d31/f84 to d16/d26/d2b/fe6 0 2024-04-26T04:52:58.416 INFO:tasks.workunit.client.1.smithi078.stdout:2/587: mkdir d8/d11/d14/d69/d8f/d34/dd3 0 2024-04-26T04:52:58.419 INFO:tasks.workunit.client.1.smithi078.stdout:4/644: dwrite d0/d1d/de1/d20/d31/d47/db2/f4d [0,4194304] 0 2024-04-26T04:52:58.419 INFO:tasks.workunit.client.1.smithi078.stdout:3/644: dread - d16/d26/d2b/d31/d71/d8a/dbc/fdf zero size 2024-04-26T04:52:58.419 INFO:tasks.workunit.client.1.smithi078.stdout:2/588: readlink d8/d3e/d8e/la9 0 2024-04-26T04:52:58.421 INFO:tasks.workunit.client.1.smithi078.stdout:7/590: rename d9/da/d33/d4b/d4c/d5b/d83/f9d to d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc1 0 2024-04-26T04:52:58.422 INFO:tasks.workunit.client.1.smithi078.stdout:1/659: read d7/d1e/d9e/dd/d20/f50 [1882022,117403] 0 2024-04-26T04:52:58.422 INFO:tasks.workunit.client.1.smithi078.stdout:4/645: chown d0/de/d22/d27/de7/fee 16184878 1 2024-04-26T04:52:58.424 INFO:tasks.workunit.client.1.smithi078.stdout:5/553: dwrite d7/d48/d57/d65/f53 [0,4194304] 0 2024-04-26T04:52:58.425 INFO:tasks.workunit.client.1.smithi078.stdout:2/589: symlink d8/d53/ld4 0 2024-04-26T04:52:58.427 INFO:tasks.workunit.client.1.smithi078.stdout:5/554: stat d7/d18/d32/d37/f3d 0 2024-04-26T04:52:58.428 INFO:tasks.workunit.client.1.smithi078.stdout:7/591: read d9/da/d11/d13/f25 [1566994,67242] 0 2024-04-26T04:52:58.428 INFO:tasks.workunit.client.1.smithi078.stdout:4/646: unlink d0/d1d/f36 0 2024-04-26T04:52:58.429 INFO:tasks.workunit.client.1.smithi078.stdout:3/645: dwrite d16/d26/d2b/d66/fc2 [0,4194304] 0 2024-04-26T04:52:58.429 INFO:tasks.workunit.client.1.smithi078.stdout:5/555: chown d7/d48/d57/d65/fb0 23 1 2024-04-26T04:52:58.435 INFO:tasks.workunit.client.1.smithi078.stdout:1/660: getdents d7/d1e/d9e/dd/d20/dca 0 2024-04-26T04:52:58.437 INFO:tasks.workunit.client.1.smithi078.stdout:7/592: chown d9/da/d11/d13/c18 3443869 1 2024-04-26T04:52:58.438 INFO:tasks.workunit.client.1.smithi078.stdout:3/646: creat d16/d26/d2b/d31/d71/d8a/dbc/fe7 x:0 0 0 2024-04-26T04:52:58.438 INFO:tasks.workunit.client.1.smithi078.stdout:2/590: mknod d8/d11/d14/d69/d8f/cd5 0 2024-04-26T04:52:58.439 INFO:tasks.workunit.client.1.smithi078.stdout:5/556: creat d7/d11/fb5 x:0 0 0 2024-04-26T04:52:58.440 INFO:tasks.workunit.client.1.smithi078.stdout:2/591: chown d8/l10 536344854 1 2024-04-26T04:52:58.441 INFO:tasks.workunit.client.0.smithi077.stdout:4/283: write d4/f27 [1544683,61648] 0 2024-04-26T04:52:58.441 INFO:tasks.workunit.client.1.smithi078.stdout:1/661: readlink d7/d1e/d9e/dd/lad 0 2024-04-26T04:52:58.441 INFO:tasks.workunit.client.1.smithi078.stdout:3/647: readlink d16/d26/d2b/d66/ld9 0 2024-04-26T04:52:58.442 INFO:tasks.workunit.client.1.smithi078.stdout:4/647: dwrite d0/d1d/de1/d20/d31/f73 [0,4194304] 0 2024-04-26T04:52:58.443 INFO:tasks.workunit.client.0.smithi077.stdout:4/284: fsync d4/d17/d18/d43/d53/d56/f1b 0 2024-04-26T04:52:58.445 INFO:tasks.workunit.client.1.smithi078.stdout:5/557: mkdir d7/d21/d2f/db6 0 2024-04-26T04:52:58.445 INFO:tasks.workunit.client.0.smithi077.stdout:6/300: write d8/d2a/f32 [3331777,1123] 0 2024-04-26T04:52:58.447 INFO:tasks.workunit.client.1.smithi078.stdout:7/593: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc2 x:0 0 0 2024-04-26T04:52:58.450 INFO:tasks.workunit.client.1.smithi078.stdout:2/592: dwrite d8/d11/fbe [0,4194304] 0 2024-04-26T04:52:58.451 INFO:tasks.workunit.client.1.smithi078.stdout:1/662: creat d7/d1e/d9e/db/d37/d58/d5c/fdc x:0 0 0 2024-04-26T04:52:58.453 INFO:tasks.workunit.client.0.smithi077.stdout:6/301: chown d8/d52/d53/c56 2 1 2024-04-26T04:52:58.454 INFO:tasks.workunit.client.1.smithi078.stdout:5/558: dwrite d7/da/f10 [0,4194304] 0 2024-04-26T04:52:58.455 INFO:tasks.workunit.client.1.smithi078.stdout:3/648: mknod d16/d26/d2b/d31/d71/d8a/dbc/ce8 0 2024-04-26T04:52:58.455 INFO:tasks.workunit.client.1.smithi078.stdout:8/570: truncate d0/d19/d52/faa 1514468 0 2024-04-26T04:52:58.459 INFO:tasks.workunit.client.1.smithi078.stdout:4/648: mknod d0/d1d/de1/d20/d31/d47/db2/d44/cef 0 2024-04-26T04:52:58.460 INFO:tasks.workunit.client.1.smithi078.stdout:3/649: truncate d16/d26/d2e/d5f/d49/dad/fd8 944267 0 2024-04-26T04:52:58.462 INFO:tasks.workunit.client.0.smithi077.stdout:6/302: mknod d8/d52/d53/c68 0 2024-04-26T04:52:58.465 INFO:tasks.workunit.client.1.smithi078.stdout:2/593: fdatasync d8/d11/d18/d5d/d61/f4a 0 2024-04-26T04:52:58.467 INFO:tasks.workunit.client.1.smithi078.stdout:5/559: fsync d7/d48/d57/d65/d46/f4f 0 2024-04-26T04:52:58.468 INFO:tasks.workunit.client.0.smithi077.stdout:6/303: dwrite d8/d55/f62 [0,4194304] 0 2024-04-26T04:52:58.473 INFO:tasks.workunit.client.1.smithi078.stdout:1/663: getdents d7/d1e/d9e/db/d37/d58/d5c/d68 0 2024-04-26T04:52:58.475 INFO:tasks.workunit.client.0.smithi077.stdout:6/304: creat d8/d52/d53/d5e/f69 x:0 0 0 2024-04-26T04:52:58.476 INFO:tasks.workunit.client.0.smithi077.stdout:6/305: write d8/f5a [879098,105808] 0 2024-04-26T04:52:58.476 INFO:tasks.workunit.client.1.smithi078.stdout:4/649: truncate d0/d1d/de1/d20/f15 35790 0 2024-04-26T04:52:58.477 INFO:tasks.workunit.client.0.smithi077.stdout:6/306: chown d8/f3f 4 1 2024-04-26T04:52:58.477 INFO:tasks.workunit.client.1.smithi078.stdout:1/664: read d7/d1e/d9e/db/d49/d93/faf [29588,90333] 0 2024-04-26T04:52:58.477 INFO:tasks.workunit.client.1.smithi078.stdout:5/560: creat d7/da/d8a/fb7 x:0 0 0 2024-04-26T04:52:58.479 INFO:tasks.workunit.client.0.smithi077.stdout:6/307: unlink d8/d1c/c21 0 2024-04-26T04:52:58.481 INFO:tasks.workunit.client.1.smithi078.stdout:1/665: unlink d7/d1e/f72 0 2024-04-26T04:52:58.483 INFO:tasks.workunit.client.1.smithi078.stdout:1/666: write d7/d1e/d9e/db/d49/fd3 [390342,24620] 0 2024-04-26T04:52:58.483 INFO:tasks.workunit.client.1.smithi078.stdout:3/650: dread d16/d26/d2b/d54/d56/f7d [0,4194304] 0 2024-04-26T04:52:58.484 INFO:tasks.workunit.client.1.smithi078.stdout:4/650: rmdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab 39 2024-04-26T04:52:58.485 INFO:tasks.workunit.client.1.smithi078.stdout:7/594: dread f4 [4194304,4194304] 0 2024-04-26T04:52:58.486 INFO:tasks.workunit.client.1.smithi078.stdout:7/595: write d9/da/faf [1226720,74379] 0 2024-04-26T04:52:58.486 INFO:tasks.workunit.client.0.smithi077.stdout:6/308: dwrite d8/d52/f4d [0,4194304] 0 2024-04-26T04:52:58.489 INFO:tasks.workunit.client.0.smithi077.stdout:2/229: sync 2024-04-26T04:52:58.489 INFO:tasks.workunit.client.0.smithi077.stdout:0/212: sync 2024-04-26T04:52:58.489 INFO:tasks.workunit.client.0.smithi077.stdout:3/221: sync 2024-04-26T04:52:58.490 INFO:tasks.workunit.client.1.smithi078.stdout:6/680: sync 2024-04-26T04:52:58.490 INFO:tasks.workunit.client.1.smithi078.stdout:4/651: readlink d0/d1d/de1/d20/d31/d47/db2/d44/d55/d5e/lbb 0 2024-04-26T04:52:58.491 INFO:tasks.workunit.client.0.smithi077.stdout:0/213: read - d0/d47/d11/f3e zero size 2024-04-26T04:52:58.491 INFO:tasks.workunit.client.1.smithi078.stdout:8/571: dread d0/d19/d25/d36/f48 [0,4194304] 0 2024-04-26T04:52:58.491 INFO:tasks.workunit.client.1.smithi078.stdout:4/652: stat d0/de/d22/c6d 0 2024-04-26T04:52:58.494 INFO:tasks.workunit.client.0.smithi077.stdout:6/309: symlink d8/d52/d53/l6a 0 2024-04-26T04:52:58.494 INFO:tasks.workunit.client.0.smithi077.stdout:6/310: dread - d8/f48 zero size 2024-04-26T04:52:58.496 INFO:tasks.workunit.client.0.smithi077.stdout:2/230: chown d9/c2a 2519777 1 2024-04-26T04:52:58.498 INFO:tasks.workunit.client.0.smithi077.stdout:6/311: dread d8/d2a/f32 [0,4194304] 0 2024-04-26T04:52:58.498 INFO:tasks.workunit.client.1.smithi078.stdout:7/596: mkdir d9/da/d33/d4b/d4c/d5b/dc3 0 2024-04-26T04:52:58.499 INFO:tasks.workunit.client.0.smithi077.stdout:4/285: truncate d4/f8 11179997 0 2024-04-26T04:52:58.499 INFO:tasks.workunit.client.0.smithi077.stdout:6/312: write d8/d2a/f66 [317970,49691] 0 2024-04-26T04:52:58.501 INFO:tasks.workunit.client.1.smithi078.stdout:1/667: getdents d7/dd7 0 2024-04-26T04:52:58.503 INFO:tasks.workunit.client.1.smithi078.stdout:6/681: creat d2/de/d27/da9/da/d36/db9/d97/fd2 x:0 0 0 2024-04-26T04:52:58.504 INFO:tasks.workunit.client.1.smithi078.stdout:1/668: dread d7/d2e/f33 [0,4194304] 0 2024-04-26T04:52:58.505 INFO:tasks.workunit.client.1.smithi078.stdout:8/572: rename d0/c2d to d0/d19/d52/d35/d44/cb6 0 2024-04-26T04:52:58.507 INFO:tasks.workunit.client.1.smithi078.stdout:4/653: truncate d0/d1d/de1/d20/d31/d47/db2/fc5 704774 0 2024-04-26T04:52:58.507 INFO:tasks.workunit.client.1.smithi078.stdout:2/594: write d8/d11/f2a [1843890,39436] 0 2024-04-26T04:52:58.508 INFO:tasks.workunit.client.0.smithi077.stdout:3/222: mknod d0/c45 0 2024-04-26T04:52:58.508 INFO:tasks.workunit.client.1.smithi078.stdout:2/595: read - d8/d11/d14/d69/d8f/d34/fad zero size 2024-04-26T04:52:58.509 INFO:tasks.workunit.client.1.smithi078.stdout:1/669: dread d7/d1e/d9e/db/d37/f88 [0,4194304] 0 2024-04-26T04:52:58.510 INFO:tasks.workunit.client.0.smithi077.stdout:3/223: truncate d0/d9/d32/f35 1060130 0 2024-04-26T04:52:58.510 INFO:tasks.workunit.client.1.smithi078.stdout:7/597: mkdir d9/da/d11/d13/d1a/dc4 0 2024-04-26T04:52:58.510 INFO:tasks.workunit.client.1.smithi078.stdout:7/598: write d9/da/d11/f2f [1127286,93761] 0 2024-04-26T04:52:58.513 INFO:tasks.workunit.client.1.smithi078.stdout:6/682: mknod d2/de/d27/da9/da/d36/db9/cd3 0 2024-04-26T04:52:58.514 INFO:tasks.workunit.client.0.smithi077.stdout:7/302: sync 2024-04-26T04:52:58.514 INFO:tasks.workunit.client.0.smithi077.stdout:9/256: sync 2024-04-26T04:52:58.514 INFO:tasks.workunit.client.0.smithi077.stdout:1/365: sync 2024-04-26T04:52:58.514 INFO:tasks.workunit.client.0.smithi077.stdout:8/268: sync 2024-04-26T04:52:58.514 INFO:tasks.workunit.client.0.smithi077.stdout:5/293: sync 2024-04-26T04:52:58.515 INFO:tasks.workunit.client.0.smithi077.stdout:4/286: mknod d4/d17/d18/d43/d53/d56/c61 0 2024-04-26T04:52:58.518 INFO:tasks.workunit.client.0.smithi077.stdout:4/287: dread f2 [0,4194304] 0 2024-04-26T04:52:58.519 INFO:tasks.workunit.client.1.smithi078.stdout:4/654: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [1653014,92201] 0 2024-04-26T04:52:58.524 INFO:tasks.workunit.client.0.smithi077.stdout:3/224: unlink d0/d9/f23 0 2024-04-26T04:52:58.525 INFO:tasks.workunit.client.0.smithi077.stdout:3/225: chown d0/l39 1 1 2024-04-26T04:52:58.530 INFO:tasks.workunit.client.0.smithi077.stdout:9/257: mkdir d9/d58 0 2024-04-26T04:52:58.534 INFO:tasks.workunit.client.0.smithi077.stdout:3/226: dread d0/d9/d32/f2f [0,4194304] 0 2024-04-26T04:52:58.536 INFO:tasks.workunit.client.0.smithi077.stdout:5/294: mkdir d1/d8/d72 0 2024-04-26T04:52:58.537 INFO:tasks.workunit.client.0.smithi077.stdout:5/295: dread - d1/d8/f70 zero size 2024-04-26T04:52:58.542 INFO:tasks.workunit.client.0.smithi077.stdout:2/231: link d9/ff d9/d15/d1a/f54 0 2024-04-26T04:52:58.545 INFO:tasks.workunit.client.0.smithi077.stdout:0/214: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:52:58.548 INFO:tasks.workunit.client.0.smithi077.stdout:9/258: creat d9/dd/d3d/f59 x:0 0 0 2024-04-26T04:52:58.549 INFO:tasks.workunit.client.0.smithi077.stdout:3/227: rmdir d0/d7/d15 39 2024-04-26T04:52:58.556 INFO:tasks.workunit.client.0.smithi077.stdout:2/232: creat d9/dc/d1d/d1e/f55 x:0 0 0 2024-04-26T04:52:58.557 INFO:tasks.workunit.client.0.smithi077.stdout:6/313: link d8/d2a/l36 d8/d2a/d57/l6b 0 2024-04-26T04:52:58.557 INFO:tasks.workunit.client.0.smithi077.stdout:5/296: dwrite d1/d42/d62/f6f [0,4194304] 0 2024-04-26T04:52:58.558 INFO:tasks.workunit.client.0.smithi077.stdout:5/297: chown d1/d12/d45 1487 1 2024-04-26T04:52:58.560 INFO:tasks.workunit.client.0.smithi077.stdout:5/298: readlink d1/d12/d21/d25/d31/l32 0 2024-04-26T04:52:58.560 INFO:tasks.workunit.client.1.smithi078.stdout:6/683: creat d2/de/d27/da9/d2a/dbf/d5f/fd4 x:0 0 0 2024-04-26T04:52:58.560 INFO:tasks.workunit.client.1.smithi078.stdout:4/655: fdatasync d0/d1d/de1/d20/d31/d47/db2/d44/d55/fe5 0 2024-04-26T04:52:58.561 INFO:tasks.workunit.client.0.smithi077.stdout:5/299: dread - d1/d30/d35/f57 zero size 2024-04-26T04:52:58.561 INFO:tasks.workunit.client.0.smithi077.stdout:5/300: write d1/d30/d35/f6c [515625,73303] 0 2024-04-26T04:52:58.562 INFO:tasks.workunit.client.1.smithi078.stdout:8/573: truncate d0/d19/d52/d35/d44/f4d 3571391 0 2024-04-26T04:52:58.564 INFO:tasks.workunit.client.0.smithi077.stdout:9/259: chown d9/dd/d1b/d2b/l37 36 1 2024-04-26T04:52:58.564 INFO:tasks.workunit.client.0.smithi077.stdout:8/269: write d4/d17/f2c [4574533,116110] 0 2024-04-26T04:52:58.565 INFO:tasks.workunit.client.1.smithi078.stdout:5/561: dwrite d7/d18/d32/d37/f6b [0,4194304] 0 2024-04-26T04:52:58.565 INFO:tasks.workunit.client.0.smithi077.stdout:4/288: write f2 [4933634,118651] 0 2024-04-26T04:52:58.565 INFO:tasks.workunit.client.1.smithi078.stdout:8/574: read d0/d19/d52/d35/d44/f5d [2703519,7012] 0 2024-04-26T04:52:58.569 INFO:tasks.workunit.client.1.smithi078.stdout:5/562: truncate d7/da/d8a/fb7 261949 0 2024-04-26T04:52:58.573 INFO:tasks.workunit.client.0.smithi077.stdout:9/260: dwrite d9/f26 [0,4194304] 0 2024-04-26T04:52:58.574 INFO:tasks.workunit.client.0.smithi077.stdout:1/366: truncate d9/db/d2b/f3d 660676 0 2024-04-26T04:52:58.575 INFO:tasks.workunit.client.0.smithi077.stdout:9/261: chown d9/dd/d1b/c48 266815 1 2024-04-26T04:52:58.576 INFO:tasks.workunit.client.0.smithi077.stdout:2/233: unlink d9/dc/d1d/f1f 0 2024-04-26T04:52:58.577 INFO:tasks.workunit.client.0.smithi077.stdout:2/234: readlink d9/dc/d1d/d45/l48 0 2024-04-26T04:52:58.578 INFO:tasks.workunit.client.0.smithi077.stdout:2/235: write d9/d15/d1a/f30 [861765,59683] 0 2024-04-26T04:52:58.580 INFO:tasks.workunit.client.0.smithi077.stdout:2/236: write d9/d15/f2f [103019,14653] 0 2024-04-26T04:52:58.582 INFO:tasks.workunit.client.0.smithi077.stdout:7/303: link da/d15/c29 da/d13/d22/c74 0 2024-04-26T04:52:58.583 INFO:tasks.workunit.client.0.smithi077.stdout:7/304: write da/f36 [1053371,129385] 0 2024-04-26T04:52:58.584 INFO:tasks.workunit.client.1.smithi078.stdout:6/684: rmdir d2/de/d27/da9/d3a/d69/daa 39 2024-04-26T04:52:58.591 INFO:tasks.workunit.client.0.smithi077.stdout:7/305: dwrite da/d15/f2f [0,4194304] 0 2024-04-26T04:52:58.591 INFO:tasks.workunit.client.1.smithi078.stdout:2/596: write d8/f1f [553132,82558] 0 2024-04-26T04:52:58.593 INFO:tasks.workunit.client.1.smithi078.stdout:7/599: link d9/da/d11/d13/d1a/d52/d5a/c19 d9/da/d33/d4b/dc0/cc5 0 2024-04-26T04:52:58.593 INFO:tasks.workunit.client.1.smithi078.stdout:7/600: stat d9/da/d11/d13/d50/f86 0 2024-04-26T04:52:58.595 INFO:tasks.workunit.client.1.smithi078.stdout:7/601: chown d9/da/d11/d13/d1a/d52/d5a/f44 10 1 2024-04-26T04:52:58.596 INFO:tasks.workunit.client.1.smithi078.stdout:7/602: chown d9/da/d11/d13/d50/l74 14636 1 2024-04-26T04:52:58.597 INFO:tasks.workunit.client.1.smithi078.stdout:9/569: sync 2024-04-26T04:52:58.598 INFO:tasks.workunit.client.1.smithi078.stdout:0/646: sync 2024-04-26T04:52:58.599 INFO:tasks.workunit.client.1.smithi078.stdout:9/570: dread - dc/df/d4b/d22/d2a/d92/fa2 zero size 2024-04-26T04:52:58.600 INFO:tasks.workunit.client.1.smithi078.stdout:9/571: write dc/df/d27/d81/fa5 [643969,16712] 0 2024-04-26T04:52:58.602 INFO:tasks.workunit.client.1.smithi078.stdout:4/656: write d0/d1d/de1/d20/d31/d47/db2/fa3 [690040,21748] 0 2024-04-26T04:52:58.603 INFO:tasks.workunit.client.1.smithi078.stdout:4/657: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa 47 1 2024-04-26T04:52:58.610 INFO:tasks.workunit.client.0.smithi077.stdout:7/306: unlink da/d13/l73 0 2024-04-26T04:52:58.617 INFO:tasks.workunit.client.0.smithi077.stdout:0/215: link d0/c23 d0/d28/d39/c4a 0 2024-04-26T04:52:58.618 INFO:tasks.workunit.client.0.smithi077.stdout:8/270: dread d4/fd [0,4194304] 0 2024-04-26T04:52:58.624 INFO:tasks.workunit.client.1.smithi078.stdout:3/651: sync 2024-04-26T04:52:58.625 INFO:tasks.workunit.client.1.smithi078.stdout:3/652: fdatasync d16/d26/d2e/d74/f97 0 2024-04-26T04:52:58.625 INFO:tasks.workunit.client.0.smithi077.stdout:7/307: chown da/d15/d42/l4f 341344374 1 2024-04-26T04:52:58.629 INFO:tasks.workunit.client.1.smithi078.stdout:3/653: dread f9 [0,4194304] 0 2024-04-26T04:52:58.629 INFO:tasks.workunit.client.0.smithi077.stdout:3/228: rename d0/d7/d15/c17 to d0/d7/c46 0 2024-04-26T04:52:58.631 INFO:tasks.workunit.client.0.smithi077.stdout:3/229: write d0/f1d [929458,71835] 0 2024-04-26T04:52:58.632 INFO:tasks.workunit.client.0.smithi077.stdout:7/308: dwrite da/d6b/d63/f70 [0,4194304] 0 2024-04-26T04:52:58.632 INFO:tasks.workunit.client.0.smithi077.stdout:9/262: rmdir d9/d58 0 2024-04-26T04:52:58.633 INFO:tasks.workunit.client.0.smithi077.stdout:3/230: write d0/f42 [520936,82305] 0 2024-04-26T04:52:58.634 INFO:tasks.workunit.client.0.smithi077.stdout:3/231: read - d0/f3d zero size 2024-04-26T04:52:58.636 INFO:tasks.workunit.client.0.smithi077.stdout:6/314: getdents d8/d2a/d57 0 2024-04-26T04:52:58.638 INFO:tasks.workunit.client.0.smithi077.stdout:3/232: read d0/d7/f3e [680848,129206] 0 2024-04-26T04:52:58.639 INFO:tasks.workunit.client.0.smithi077.stdout:8/271: truncate d4/df/f15 52278 0 2024-04-26T04:52:58.642 INFO:tasks.workunit.client.0.smithi077.stdout:5/301: getdents d1/d42 0 2024-04-26T04:52:58.643 INFO:tasks.workunit.client.0.smithi077.stdout:7/309: dwrite da/d13/d22/f2e [0,4194304] 0 2024-04-26T04:52:58.645 INFO:tasks.workunit.client.0.smithi077.stdout:5/302: truncate d1/d12/d45/f5e 1261534 0 2024-04-26T04:52:58.645 INFO:tasks.workunit.client.0.smithi077.stdout:5/303: dread - d1/d12/f60 zero size 2024-04-26T04:52:58.646 INFO:tasks.workunit.client.0.smithi077.stdout:5/304: chown d1/d30/d33/l6e 327403 1 2024-04-26T04:52:58.649 INFO:tasks.workunit.client.0.smithi077.stdout:9/263: chown d9/dd/l29 125313 1 2024-04-26T04:52:58.656 INFO:tasks.workunit.client.0.smithi077.stdout:0/216: symlink d0/d28/d39/l4b 0 2024-04-26T04:52:58.657 INFO:tasks.workunit.client.1.smithi078.stdout:6/685: write d2/de/d27/da9/f8 [2530127,29728] 0 2024-04-26T04:52:58.659 INFO:tasks.workunit.client.0.smithi077.stdout:6/315: mkdir d8/d52/d53/d5e/d6c 0 2024-04-26T04:52:58.663 INFO:tasks.workunit.client.0.smithi077.stdout:2/237: getdents d9/dc/d1d/d34 0 2024-04-26T04:52:58.664 INFO:tasks.workunit.client.0.smithi077.stdout:0/217: dwrite d0/d2c/f43 [0,4194304] 0 2024-04-26T04:52:58.665 INFO:tasks.workunit.client.0.smithi077.stdout:0/218: chown d0/f3 6423269 1 2024-04-26T04:52:58.667 INFO:tasks.workunit.client.1.smithi078.stdout:5/563: mknod d7/d48/cb8 0 2024-04-26T04:52:58.669 INFO:tasks.workunit.client.0.smithi077.stdout:7/310: mknod da/d13/d22/d34/c75 0 2024-04-26T04:52:58.670 INFO:tasks.workunit.client.0.smithi077.stdout:0/219: dwrite d0/d47/f16 [0,4194304] 0 2024-04-26T04:52:58.670 INFO:tasks.workunit.client.0.smithi077.stdout:7/311: write da/f57 [467080,48590] 0 2024-04-26T04:52:58.671 INFO:tasks.workunit.client.1.smithi078.stdout:7/603: unlink d9/da/d11/f2f 0 2024-04-26T04:52:58.672 INFO:tasks.workunit.client.1.smithi078.stdout:0/647: fdatasync d2/f10 0 2024-04-26T04:52:58.674 INFO:tasks.workunit.client.1.smithi078.stdout:9/572: write dc/df/d4b/f86 [2060888,83736] 0 2024-04-26T04:52:58.675 INFO:tasks.workunit.client.1.smithi078.stdout:9/573: chown dc/f39 2 1 2024-04-26T04:52:58.681 INFO:tasks.workunit.client.0.smithi077.stdout:2/238: mkdir d9/d15/d56 0 2024-04-26T04:52:58.683 INFO:tasks.workunit.client.1.smithi078.stdout:1/670: sync 2024-04-26T04:52:58.684 INFO:tasks.workunit.client.1.smithi078.stdout:1/671: readlink d7/d1e/d9e/db/d49/dba/dc5/lc7 0 2024-04-26T04:52:58.684 INFO:tasks.workunit.client.1.smithi078.stdout:1/672: read - d7/d1e/d9e/dd/d20/fb0 zero size 2024-04-26T04:52:58.685 INFO:tasks.workunit.client.1.smithi078.stdout:2/597: mkdir d8/d11/d14/d69/d8f/d34/dd6 0 2024-04-26T04:52:58.686 INFO:tasks.workunit.client.1.smithi078.stdout:2/598: write d8/d11/d18/f47 [507017,105240] 0 2024-04-26T04:52:58.689 INFO:tasks.workunit.client.1.smithi078.stdout:5/564: mknod d7/d11/cb9 0 2024-04-26T04:52:58.689 INFO:tasks.workunit.client.1.smithi078.stdout:4/658: write d0/de/d22/d71/f79 [3315873,43688] 0 2024-04-26T04:52:58.692 INFO:tasks.workunit.client.1.smithi078.stdout:7/604: creat d9/da/d11/d13/d1a/d52/d5a/d28/d6d/fc6 x:0 0 0 2024-04-26T04:52:58.693 INFO:tasks.workunit.client.1.smithi078.stdout:0/648: rmdir d2/d9/d60/d6a/da5 39 2024-04-26T04:52:58.695 INFO:tasks.workunit.client.0.smithi077.stdout:9/264: link d9/dd/d1b/d2b/f38 d9/dd/d3d/f5a 0 2024-04-26T04:52:58.697 INFO:tasks.workunit.client.0.smithi077.stdout:6/316: unlink d8/l3c 0 2024-04-26T04:52:58.699 INFO:tasks.workunit.client.0.smithi077.stdout:5/305: truncate d1/d42/d62/f6f 1998923 0 2024-04-26T04:52:58.701 INFO:tasks.workunit.client.1.smithi078.stdout:8/575: dread d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:52:58.703 INFO:tasks.workunit.client.0.smithi077.stdout:3/233: link d0/d9/d32/cf d0/d9/c47 0 2024-04-26T04:52:58.703 INFO:tasks.workunit.client.0.smithi077.stdout:6/317: dwrite d8/f22 [0,4194304] 0 2024-04-26T04:52:58.704 INFO:tasks.workunit.client.1.smithi078.stdout:6/686: mkdir d2/d1a/d38/dd5 0 2024-04-26T04:52:58.704 INFO:tasks.workunit.client.0.smithi077.stdout:3/234: truncate d0/f3d 629745 0 2024-04-26T04:52:58.706 INFO:tasks.workunit.client.0.smithi077.stdout:3/235: write d0/d9/f2b [358096,8481] 0 2024-04-26T04:52:58.710 INFO:tasks.workunit.client.0.smithi077.stdout:7/312: truncate da/ff 1478831 0 2024-04-26T04:52:58.712 INFO:tasks.workunit.client.1.smithi078.stdout:2/599: mknod d8/d11/d14/d69/cd7 0 2024-04-26T04:52:58.716 INFO:tasks.workunit.client.1.smithi078.stdout:5/565: readlink d7/da/d1e/lb3 0 2024-04-26T04:52:58.719 INFO:tasks.workunit.client.0.smithi077.stdout:7/313: dwrite da/f57 [0,4194304] 0 2024-04-26T04:52:58.721 INFO:tasks.workunit.client.1.smithi078.stdout:4/659: unlink d0/de/d22/d27/d7b/f8f 0 2024-04-26T04:52:58.721 INFO:tasks.workunit.client.0.smithi077.stdout:9/265: dwrite d9/f2d [0,4194304] 0 2024-04-26T04:52:58.724 INFO:tasks.workunit.client.0.smithi077.stdout:5/306: symlink d1/d12/d21/l73 0 2024-04-26T04:52:58.725 INFO:tasks.workunit.client.0.smithi077.stdout:0/220: write d0/d19/d1e/f26 [76306,99228] 0 2024-04-26T04:52:58.725 INFO:tasks.workunit.client.1.smithi078.stdout:7/605: write d9/f14 [5398569,71205] 0 2024-04-26T04:52:58.726 INFO:tasks.workunit.client.1.smithi078.stdout:7/606: chown d9/da/d11/d13/d50/l74 309471 1 2024-04-26T04:52:58.727 INFO:tasks.workunit.client.0.smithi077.stdout:5/307: truncate d1/d12/f50 759935 0 2024-04-26T04:52:58.729 INFO:tasks.workunit.client.0.smithi077.stdout:9/266: dread d9/dd/d28/f33 [8388608,4194304] 0 2024-04-26T04:52:58.732 INFO:tasks.workunit.client.1.smithi078.stdout:8/576: creat d0/d19/d25/d36/d58/fb7 x:0 0 0 2024-04-26T04:52:58.733 INFO:tasks.workunit.client.0.smithi077.stdout:8/272: write d4/df/f15 [1080375,14238] 0 2024-04-26T04:52:58.733 INFO:tasks.workunit.client.1.smithi078.stdout:3/654: link d16/d9b/cd4 d16/d26/d2e/d5f/d49/dcc/ce9 0 2024-04-26T04:52:58.734 INFO:tasks.workunit.client.1.smithi078.stdout:7/607: dwrite d9/f90 [0,4194304] 0 2024-04-26T04:52:58.734 INFO:tasks.workunit.client.0.smithi077.stdout:8/273: chown d4/df/d22 51 1 2024-04-26T04:52:58.737 INFO:tasks.workunit.client.1.smithi078.stdout:6/687: creat d2/de/d27/d58/d5b/fd6 x:0 0 0 2024-04-26T04:52:58.741 INFO:tasks.workunit.client.1.smithi078.stdout:2/600: truncate d8/d11/d14/d69/d8f/f5c 576178 0 2024-04-26T04:52:58.742 INFO:tasks.workunit.client.0.smithi077.stdout:7/314: symlink da/d13/d22/l76 0 2024-04-26T04:52:58.743 INFO:tasks.workunit.client.1.smithi078.stdout:5/566: symlink d7/d18/d32/lba 0 2024-04-26T04:52:58.744 INFO:tasks.workunit.client.1.smithi078.stdout:0/649: dwrite d2/d9/d31/d4a/f5d [0,4194304] 0 2024-04-26T04:52:58.745 INFO:tasks.workunit.client.1.smithi078.stdout:5/567: truncate d7/d48/d57/d65/fb0 859400 0 2024-04-26T04:52:58.746 INFO:tasks.workunit.client.1.smithi078.stdout:4/660: creat d0/de/d22/d71/ff0 x:0 0 0 2024-04-26T04:52:58.748 INFO:tasks.workunit.client.0.smithi077.stdout:9/267: symlink d9/dd/d3d/l5b 0 2024-04-26T04:52:58.749 INFO:tasks.workunit.client.1.smithi078.stdout:3/655: creat d16/d7e/fea x:0 0 0 2024-04-26T04:52:58.754 INFO:tasks.workunit.client.1.smithi078.stdout:2/601: symlink d8/d3e/ld8 0 2024-04-26T04:52:58.755 INFO:tasks.workunit.client.0.smithi077.stdout:7/315: creat da/d15/d42/f77 x:0 0 0 2024-04-26T04:52:58.761 INFO:tasks.workunit.client.0.smithi077.stdout:5/308: mknod d1/d42/d62/d6b/c74 0 2024-04-26T04:52:58.762 INFO:tasks.workunit.client.1.smithi078.stdout:5/568: creat d7/d18/d1c/d5a/fbb x:0 0 0 2024-04-26T04:52:58.762 INFO:tasks.workunit.client.0.smithi077.stdout:7/316: dwrite da/d6b/d63/f70 [0,4194304] 0 2024-04-26T04:52:58.763 INFO:tasks.workunit.client.1.smithi078.stdout:8/577: symlink d0/d19/d52/d95/d9d/lb8 0 2024-04-26T04:52:58.764 INFO:tasks.workunit.client.0.smithi077.stdout:7/317: write da/d13/f4b [524322,4355] 0 2024-04-26T04:52:58.764 INFO:tasks.workunit.client.0.smithi077.stdout:6/318: creat d8/f6d x:0 0 0 2024-04-26T04:52:58.765 INFO:tasks.workunit.client.0.smithi077.stdout:7/318: readlink da/d15/d1a/d39/l62 0 2024-04-26T04:52:58.765 INFO:tasks.workunit.client.1.smithi078.stdout:8/578: read d0/d19/d52/f29 [1198019,17577] 0 2024-04-26T04:52:58.766 INFO:tasks.workunit.client.1.smithi078.stdout:3/656: mknod d16/d9b/ceb 0 2024-04-26T04:52:58.769 INFO:tasks.workunit.client.0.smithi077.stdout:3/236: creat d0/d7/f48 x:0 0 0 2024-04-26T04:52:58.772 INFO:tasks.workunit.client.1.smithi078.stdout:1/673: rename d7/d1e/d9e/db/d37/d58/d86 to d7/ddd 0 2024-04-26T04:52:58.773 INFO:tasks.workunit.client.1.smithi078.stdout:6/688: dread d2/d1a/d5c/f6d [0,4194304] 0 2024-04-26T04:52:58.773 INFO:tasks.workunit.client.0.smithi077.stdout:7/319: dwrite da/d15/f32 [0,4194304] 0 2024-04-26T04:52:58.774 INFO:tasks.workunit.client.1.smithi078.stdout:1/674: chown d7/d1e/d9e/db/d47/f90 9905207 1 2024-04-26T04:52:58.774 INFO:tasks.workunit.client.0.smithi077.stdout:7/320: chown da/f68 338977233 1 2024-04-26T04:52:58.778 INFO:tasks.workunit.client.1.smithi078.stdout:2/602: dwrite f3 [4194304,4194304] 0 2024-04-26T04:52:58.779 INFO:tasks.workunit.client.0.smithi077.stdout:6/319: creat d8/d52/f6e x:0 0 0 2024-04-26T04:52:58.779 INFO:tasks.workunit.client.0.smithi077.stdout:0/221: dread d0/fe [0,4194304] 0 2024-04-26T04:52:58.781 INFO:tasks.workunit.client.0.smithi077.stdout:6/320: truncate d8/d52/d53/f65 318509 0 2024-04-26T04:52:58.781 INFO:tasks.workunit.client.1.smithi078.stdout:4/661: mknod d0/de/d22/d27/d56/d5a/ddc/cf1 0 2024-04-26T04:52:58.782 INFO:tasks.workunit.client.0.smithi077.stdout:0/222: read d0/d19/d1e/d2f/f41 [757,88458] 0 2024-04-26T04:52:58.782 INFO:tasks.workunit.client.0.smithi077.stdout:9/268: symlink d9/dd/d28/d50/l5c 0 2024-04-26T04:52:58.783 INFO:tasks.workunit.client.0.smithi077.stdout:0/223: readlink d0/d2c/l3a 0 2024-04-26T04:52:58.785 INFO:tasks.workunit.client.0.smithi077.stdout:3/237: dread d0/f1 [0,4194304] 0 2024-04-26T04:52:58.791 INFO:tasks.workunit.client.1.smithi078.stdout:2/603: dwrite d8/d11/d18/d5d/fa8 [0,4194304] 0 2024-04-26T04:52:58.792 INFO:tasks.workunit.client.1.smithi078.stdout:4/662: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:52:58.792 INFO:tasks.workunit.client.0.smithi077.stdout:8/274: dread d4/f9 [0,4194304] 0 2024-04-26T04:52:58.795 INFO:tasks.workunit.client.1.smithi078.stdout:8/579: write d0/d19/d25/d32/d8a/f57 [1100163,10241] 0 2024-04-26T04:52:58.797 INFO:tasks.workunit.client.1.smithi078.stdout:3/657: dread d16/d3c/d81/fd7 [0,4194304] 0 2024-04-26T04:52:58.797 INFO:tasks.workunit.client.1.smithi078.stdout:0/650: unlink d2/d9/d60/d6a/da5/lbc 0 2024-04-26T04:52:58.798 INFO:tasks.workunit.client.1.smithi078.stdout:8/580: truncate d0/d19/d25/d36/d58/fb7 982931 0 2024-04-26T04:52:58.799 INFO:tasks.workunit.client.1.smithi078.stdout:7/608: sync 2024-04-26T04:52:58.799 INFO:tasks.workunit.client.1.smithi078.stdout:3/658: chown d16/d26/d2e/d5f/d49/dad/db0/db7/de4 1671 1 2024-04-26T04:52:58.800 INFO:tasks.workunit.client.1.smithi078.stdout:0/651: read d2/d9/d18/fd [2562412,73589] 0 2024-04-26T04:52:58.801 INFO:tasks.workunit.client.1.smithi078.stdout:0/652: write d2/d9/d18/fb8 [49009,81847] 0 2024-04-26T04:52:58.802 INFO:tasks.workunit.client.0.smithi077.stdout:0/224: creat d0/d47/d11/f4c x:0 0 0 2024-04-26T04:52:58.802 INFO:tasks.workunit.client.1.smithi078.stdout:0/653: chown d2/d9/d31/d4a/f5d 632336 1 2024-04-26T04:52:58.804 INFO:tasks.workunit.client.0.smithi077.stdout:3/238: mknod d0/d9/d32/d2a/c49 0 2024-04-26T04:52:58.804 INFO:tasks.workunit.client.0.smithi077.stdout:7/321: dwrite da/d13/f6d [4194304,4194304] 0 2024-04-26T04:52:58.805 INFO:tasks.workunit.client.0.smithi077.stdout:3/239: stat d0/l39 0 2024-04-26T04:52:58.807 INFO:tasks.workunit.client.0.smithi077.stdout:8/275: chown d4/d17/c3f 4 1 2024-04-26T04:52:58.808 INFO:tasks.workunit.client.1.smithi078.stdout:2/604: unlink d8/d11/d14/d69/d8f/d34/d70/fac 0 2024-04-26T04:52:58.811 INFO:tasks.workunit.client.0.smithi077.stdout:5/309: truncate d1/d8/fc 2128888 0 2024-04-26T04:52:58.813 INFO:tasks.workunit.client.0.smithi077.stdout:3/240: dwrite d0/fb [0,4194304] 0 2024-04-26T04:52:58.814 INFO:tasks.workunit.client.1.smithi078.stdout:8/581: mknod d0/d19/d52/d3c/cb9 0 2024-04-26T04:52:58.814 INFO:tasks.workunit.client.0.smithi077.stdout:3/241: stat d0/d7/f3e 0 2024-04-26T04:52:58.815 INFO:tasks.workunit.client.0.smithi077.stdout:3/242: chown d0/d7/f8 148091 1 2024-04-26T04:52:58.815 INFO:tasks.workunit.client.0.smithi077.stdout:6/321: symlink d8/l6f 0 2024-04-26T04:52:58.816 INFO:tasks.workunit.client.0.smithi077.stdout:3/243: stat d0/d9/d32/l1e 0 2024-04-26T04:52:58.816 INFO:tasks.workunit.client.0.smithi077.stdout:6/322: readlink l7 0 2024-04-26T04:52:58.817 INFO:tasks.workunit.client.0.smithi077.stdout:3/244: chown d0/d7/d15 2650152 1 2024-04-26T04:52:58.818 INFO:tasks.workunit.client.1.smithi078.stdout:3/659: mknod d16/d26/d2e/d74/cec 0 2024-04-26T04:52:58.819 INFO:tasks.workunit.client.1.smithi078.stdout:5/569: truncate d7/d18/d32/d37/f6b 1064193 0 2024-04-26T04:52:58.819 INFO:tasks.workunit.client.1.smithi078.stdout:5/570: fsync d7/d18/d1c/f95 0 2024-04-26T04:52:58.825 INFO:tasks.workunit.client.1.smithi078.stdout:6/689: symlink d2/de/d27/da9/d3a/d69/daa/ld7 0 2024-04-26T04:52:58.825 INFO:tasks.workunit.client.0.smithi077.stdout:3/245: dwrite d0/f3d [0,4194304] 0 2024-04-26T04:52:58.827 INFO:tasks.workunit.client.0.smithi077.stdout:0/225: dwrite d0/fe [0,4194304] 0 2024-04-26T04:52:58.829 INFO:tasks.workunit.client.0.smithi077.stdout:0/226: rename d0/d47/d11 to d0/d47/d11/d4d 22 2024-04-26T04:52:58.831 INFO:tasks.workunit.client.0.smithi077.stdout:5/310: unlink d1/d30/d33/c44 0 2024-04-26T04:52:58.834 INFO:tasks.workunit.client.1.smithi078.stdout:5/571: symlink d7/d48/da9/lbc 0 2024-04-26T04:52:58.834 INFO:tasks.workunit.client.0.smithi077.stdout:7/322: dread da/d15/f44 [0,4194304] 0 2024-04-26T04:52:58.834 INFO:tasks.workunit.client.0.smithi077.stdout:5/311: write d1/d30/f36 [302905,21049] 0 2024-04-26T04:52:58.834 INFO:tasks.workunit.client.0.smithi077.stdout:6/323: creat d8/d52/f70 x:0 0 0 2024-04-26T04:52:58.836 INFO:tasks.workunit.client.1.smithi078.stdout:9/574: rename dc/df/d8a/l8c to dc/df/d37/la6 0 2024-04-26T04:52:58.838 INFO:tasks.workunit.client.1.smithi078.stdout:2/605: mkdir d8/d11/d14/d69/d8f/d34/dd3/dd9 0 2024-04-26T04:52:58.839 INFO:tasks.workunit.client.1.smithi078.stdout:2/606: chown d8/c13 1678 1 2024-04-26T04:52:58.839 INFO:tasks.workunit.client.1.smithi078.stdout:9/575: write dc/df/d27/d81/f66 [3895486,87564] 0 2024-04-26T04:52:58.840 INFO:tasks.workunit.client.1.smithi078.stdout:2/607: chown d8/d11/d6e/cae 4 1 2024-04-26T04:52:58.840 INFO:tasks.workunit.client.1.smithi078.stdout:6/690: symlink d2/de/d27/da9/d2a/dbf/d5f/ld8 0 2024-04-26T04:52:58.841 INFO:tasks.workunit.client.1.smithi078.stdout:8/582: dread d0/d7a/d83/f94 [0,4194304] 0 2024-04-26T04:52:58.843 INFO:tasks.workunit.client.1.smithi078.stdout:8/583: write d0/d19/d52/d3c/f60 [2496608,6215] 0 2024-04-26T04:52:58.843 INFO:tasks.workunit.client.0.smithi077.stdout:5/312: dwrite d1/d12/f2b [0,4194304] 0 2024-04-26T04:52:58.844 INFO:tasks.workunit.client.1.smithi078.stdout:6/691: write d2/de/d27/da9/d3a/d42/f84 [2046609,78935] 0 2024-04-26T04:52:58.845 INFO:tasks.workunit.client.1.smithi078.stdout:8/584: dread - d0/d19/d52/d5c/fb3 zero size 2024-04-26T04:52:58.846 INFO:tasks.workunit.client.1.smithi078.stdout:6/692: write d2/de/d27/da9/d3a/d42/f7d [3645313,96869] 0 2024-04-26T04:52:58.846 INFO:tasks.workunit.client.0.smithi077.stdout:9/269: getdents d9/d45 0 2024-04-26T04:52:58.847 INFO:tasks.workunit.client.1.smithi078.stdout:6/693: chown d2/c46 14 1 2024-04-26T04:52:58.848 INFO:tasks.workunit.client.1.smithi078.stdout:6/694: chown d2/de/d27/da9/d2a/dbf/l91 57 1 2024-04-26T04:52:58.850 INFO:tasks.workunit.client.1.smithi078.stdout:6/695: stat d2/de/d27/da9/d3a/d69/c86 0 2024-04-26T04:52:58.850 INFO:tasks.workunit.client.1.smithi078.stdout:1/675: rename d7/d1e/d9e/db/d49/dba/f83 to d7/dd7/fde 0 2024-04-26T04:52:58.854 INFO:tasks.workunit.client.0.smithi077.stdout:0/227: rename d0/d47/d11 to d0/d2c/d4e 0 2024-04-26T04:52:58.855 INFO:tasks.workunit.client.1.smithi078.stdout:3/660: dread d16/d26/d2b/d31/f52 [0,4194304] 0 2024-04-26T04:52:58.857 INFO:tasks.workunit.client.1.smithi078.stdout:2/608: creat d8/d11/d18/d5d/d8b/db0/fda x:0 0 0 2024-04-26T04:52:58.857 INFO:tasks.workunit.client.1.smithi078.stdout:3/661: chown d16/d26/d2e/d74/f83 304 1 2024-04-26T04:52:58.859 INFO:tasks.workunit.client.0.smithi077.stdout:8/276: creat d4/f60 x:0 0 0 2024-04-26T04:52:58.861 INFO:tasks.workunit.client.0.smithi077.stdout:8/277: read d4/df/d18/f4c [126284,93045] 0 2024-04-26T04:52:58.861 INFO:tasks.workunit.client.0.smithi077.stdout:5/313: mknod d1/d12/d21/d25/c75 0 2024-04-26T04:52:58.862 INFO:tasks.workunit.client.0.smithi077.stdout:8/278: write d4/d17/f2c [4587222,62598] 0 2024-04-26T04:52:58.866 INFO:tasks.workunit.client.0.smithi077.stdout:5/314: dread d1/d8/f49 [0,4194304] 0 2024-04-26T04:52:58.867 INFO:tasks.workunit.client.1.smithi078.stdout:4/663: sync 2024-04-26T04:52:58.934 INFO:tasks.workunit.client.1.smithi078.stdout:8/585: creat d0/d19/d25/d32/fba x:0 0 0 2024-04-26T04:52:58.938 INFO:tasks.workunit.client.0.smithi077.stdout:0/228: mknod d0/d28/d39/c4f 0 2024-04-26T04:52:58.939 INFO:tasks.workunit.client.1.smithi078.stdout:8/586: dwrite d0/d19/d52/d35/d44/f93 [0,4194304] 0 2024-04-26T04:52:58.943 INFO:tasks.workunit.client.1.smithi078.stdout:6/696: mknod d2/de/d27/da9/d2a/dbf/cd9 0 2024-04-26T04:52:58.944 INFO:tasks.workunit.client.1.smithi078.stdout:9/576: rmdir dc/d55 39 2024-04-26T04:52:58.948 INFO:tasks.workunit.client.1.smithi078.stdout:6/697: dread d2/f71 [4194304,4194304] 0 2024-04-26T04:52:58.949 INFO:tasks.workunit.client.0.smithi077.stdout:8/279: mknod d4/df/d22/d2d/c61 0 2024-04-26T04:52:58.950 INFO:tasks.workunit.client.1.smithi078.stdout:4/664: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/ff2 x:0 0 0 2024-04-26T04:52:58.951 INFO:tasks.workunit.client.0.smithi077.stdout:3/246: truncate d0/f1d 1101444 0 2024-04-26T04:52:58.952 INFO:tasks.workunit.client.1.smithi078.stdout:5/572: creat d7/d48/fbd x:0 0 0 2024-04-26T04:52:58.952 INFO:tasks.workunit.client.0.smithi077.stdout:5/315: creat d1/d30/d33/f76 x:0 0 0 2024-04-26T04:52:58.956 INFO:tasks.workunit.client.0.smithi077.stdout:7/323: rename da/d13/d22/f2b to da/d15/f78 0 2024-04-26T04:52:58.956 INFO:tasks.workunit.client.1.smithi078.stdout:5/573: dwrite d7/da/f10 [0,4194304] 0 2024-04-26T04:52:58.959 INFO:tasks.workunit.client.1.smithi078.stdout:1/676: symlink d7/d1e/d9e/db/ldf 0 2024-04-26T04:52:58.959 INFO:tasks.workunit.client.0.smithi077.stdout:0/229: write d0/f9 [1738525,48409] 0 2024-04-26T04:52:58.960 INFO:tasks.workunit.client.0.smithi077.stdout:6/324: link d8/d52/c5f d8/d52/d53/c71 0 2024-04-26T04:52:58.960 INFO:tasks.workunit.client.0.smithi077.stdout:0/230: fsync d0/d19/f32 0 2024-04-26T04:52:58.964 INFO:tasks.workunit.client.1.smithi078.stdout:5/574: dwrite d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:52:58.970 INFO:tasks.workunit.client.0.smithi077.stdout:6/325: dwrite d8/f17 [0,4194304] 0 2024-04-26T04:52:58.973 INFO:tasks.workunit.client.0.smithi077.stdout:0/231: dwrite d0/d19/f35 [0,4194304] 0 2024-04-26T04:52:58.977 INFO:tasks.workunit.client.1.smithi078.stdout:7/609: rename d9/l89 to d9/da/lc7 0 2024-04-26T04:52:58.981 INFO:tasks.workunit.client.0.smithi077.stdout:3/247: creat d0/d7/d1f/f4a x:0 0 0 2024-04-26T04:52:58.982 INFO:tasks.workunit.client.1.smithi078.stdout:3/662: dwrite d16/f30 [0,4194304] 0 2024-04-26T04:52:58.983 INFO:tasks.workunit.client.0.smithi077.stdout:9/270: creat d9/dd/f5d x:0 0 0 2024-04-26T04:52:58.984 INFO:tasks.workunit.client.1.smithi078.stdout:9/577: dread dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:58.984 INFO:tasks.workunit.client.1.smithi078.stdout:2/609: link d8/d11/f48 d8/d3e/d52/fdb 0 2024-04-26T04:52:58.985 INFO:tasks.workunit.client.0.smithi077.stdout:5/316: creat d1/d42/d62/d6b/f77 x:0 0 0 2024-04-26T04:52:58.988 INFO:tasks.workunit.client.1.smithi078.stdout:4/665: truncate d0/d1d/de1/d20/fa9 1379879 0 2024-04-26T04:52:58.990 INFO:tasks.workunit.client.1.smithi078.stdout:0/654: rename d2/d9/d60/d6a/db4 to d2/d9/d60/dd5 0 2024-04-26T04:52:58.994 INFO:tasks.workunit.client.1.smithi078.stdout:2/610: dwrite d8/d3e/d8e/d91/fb2 [0,4194304] 0 2024-04-26T04:52:58.996 INFO:tasks.workunit.client.0.smithi077.stdout:6/326: readlink d8/d55/l59 0 2024-04-26T04:52:58.997 INFO:tasks.workunit.client.0.smithi077.stdout:6/327: chown d8/d52/f6e 2 1 2024-04-26T04:52:58.999 INFO:tasks.workunit.client.1.smithi078.stdout:3/663: unlink d16/d3c/d81/c96 0 2024-04-26T04:52:59.000 INFO:tasks.workunit.client.0.smithi077.stdout:0/232: write d0/d19/f49 [2948464,96069] 0 2024-04-26T04:52:59.001 INFO:tasks.workunit.client.1.smithi078.stdout:7/610: dwrite d9/fbc [0,4194304] 0 2024-04-26T04:52:59.001 INFO:tasks.workunit.client.1.smithi078.stdout:6/698: creat d2/de/d27/da9/d3a/d69/daa/fda x:0 0 0 2024-04-26T04:52:59.005 INFO:tasks.workunit.client.1.smithi078.stdout:7/611: dread d9/da/d11/d13/d1a/d2d/f3f [0,4194304] 0 2024-04-26T04:52:59.006 INFO:tasks.workunit.client.1.smithi078.stdout:9/578: mknod dc/df/d4b/d3e/d80/ca7 0 2024-04-26T04:52:59.007 INFO:tasks.workunit.client.0.smithi077.stdout:6/328: creat d8/d52/d53/f72 x:0 0 0 2024-04-26T04:52:59.008 INFO:tasks.workunit.client.1.smithi078.stdout:7/612: read d9/da/d11/d13/d1a/d52/d5a/f44 [3879409,14225] 0 2024-04-26T04:52:59.008 INFO:tasks.workunit.client.1.smithi078.stdout:1/677: dread f3 [0,4194304] 0 2024-04-26T04:52:59.009 INFO:tasks.workunit.client.1.smithi078.stdout:8/587: rename d0/l3a to d0/d19/d52/d35/d44/lbb 0 2024-04-26T04:52:59.010 INFO:tasks.workunit.client.1.smithi078.stdout:1/678: write d7/d1e/d9e/dd/fd5 [86072,62943] 0 2024-04-26T04:52:59.011 INFO:tasks.workunit.client.1.smithi078.stdout:0/655: mknod d2/d9/d31/d73/cd6 0 2024-04-26T04:52:59.011 INFO:tasks.workunit.client.0.smithi077.stdout:8/280: rename d4/df/d22/d2d/f5f to d4/df/d22/f62 0 2024-04-26T04:52:59.012 INFO:tasks.workunit.client.1.smithi078.stdout:5/575: link d7/da/f5f d7/d18/d32/fbe 0 2024-04-26T04:52:59.012 INFO:tasks.workunit.client.0.smithi077.stdout:7/324: creat da/d15/d1a/f79 x:0 0 0 2024-04-26T04:52:59.013 INFO:tasks.workunit.client.0.smithi077.stdout:7/325: fsync da/f36 0 2024-04-26T04:52:59.014 INFO:tasks.workunit.client.1.smithi078.stdout:5/576: readlink d7/da/d1e/l63 0 2024-04-26T04:52:59.016 INFO:tasks.workunit.client.1.smithi078.stdout:1/679: dread d7/d1e/d9e/db/d37/f88 [0,4194304] 0 2024-04-26T04:52:59.016 INFO:tasks.workunit.client.1.smithi078.stdout:2/611: read d8/d11/d14/d69/d8f/d34/f3d [250479,38670] 0 2024-04-26T04:52:59.017 INFO:tasks.workunit.client.0.smithi077.stdout:3/248: rename d0/d7/f48 to d0/d9/d32/d29/f4b 0 2024-04-26T04:52:59.017 INFO:tasks.workunit.client.1.smithi078.stdout:7/613: dwrite d9/da/f15 [0,4194304] 0 2024-04-26T04:52:59.018 INFO:tasks.workunit.client.1.smithi078.stdout:2/612: chown d8/d11/d18 152206 1 2024-04-26T04:52:59.018 INFO:tasks.workunit.client.1.smithi078.stdout:6/699: readlink d2/de/d27/da9/d3a/d42/lc5 0 2024-04-26T04:52:59.026 INFO:tasks.workunit.client.0.smithi077.stdout:8/281: truncate d4/df/f32 856139 0 2024-04-26T04:52:59.027 INFO:tasks.workunit.client.0.smithi077.stdout:8/282: chown d4/d17/d35/l49 3782678 1 2024-04-26T04:52:59.029 INFO:tasks.workunit.client.0.smithi077.stdout:0/233: creat d0/d47/f50 x:0 0 0 2024-04-26T04:52:59.029 INFO:tasks.workunit.client.0.smithi077.stdout:0/234: chown d0/f9 4388720 1 2024-04-26T04:52:59.030 INFO:tasks.workunit.client.1.smithi078.stdout:9/579: mkdir dc/df/d27/d81/da8 0 2024-04-26T04:52:59.032 INFO:tasks.workunit.client.1.smithi078.stdout:8/588: symlink d0/d19/d25/d36/lbc 0 2024-04-26T04:52:59.037 INFO:tasks.workunit.client.0.smithi077.stdout:9/271: rename f3 to d9/dd/d1b/d2b/d32/d36/f5e 0 2024-04-26T04:52:59.038 INFO:tasks.workunit.client.0.smithi077.stdout:9/272: write d9/dd/d3d/f3e [755164,36558] 0 2024-04-26T04:52:59.039 INFO:tasks.workunit.client.0.smithi077.stdout:0/235: symlink d0/d19/d1e/d2f/l51 0 2024-04-26T04:52:59.039 INFO:tasks.workunit.client.0.smithi077.stdout:0/236: fsync d0/f9 0 2024-04-26T04:52:59.043 INFO:tasks.workunit.client.0.smithi077.stdout:0/237: dread d0/d47/f16 [0,4194304] 0 2024-04-26T04:52:59.045 INFO:tasks.workunit.client.0.smithi077.stdout:8/283: rename d4/d17/f4f to d4/d17/d35/d5b/f63 0 2024-04-26T04:52:59.052 INFO:tasks.workunit.client.1.smithi078.stdout:0/656: mkdir d2/d9/d78/d84/dd7 0 2024-04-26T04:52:59.052 INFO:tasks.workunit.client.1.smithi078.stdout:5/577: symlink d7/da/d1e/d91/lbf 0 2024-04-26T04:52:59.052 INFO:tasks.workunit.client.1.smithi078.stdout:6/700: fdatasync d2/de/d27/f55 0 2024-04-26T04:52:59.053 INFO:tasks.workunit.client.1.smithi078.stdout:2/613: truncate d8/fa 4559539 0 2024-04-26T04:52:59.053 INFO:tasks.workunit.client.0.smithi077.stdout:0/238: readlink d0/l6 0 2024-04-26T04:52:59.054 INFO:tasks.workunit.client.1.smithi078.stdout:5/578: readlink d7/d21/d2f/l3c 0 2024-04-26T04:52:59.057 INFO:tasks.workunit.client.1.smithi078.stdout:8/589: dwrite d0/d19/d52/d5c/fb3 [0,4194304] 0 2024-04-26T04:52:59.059 INFO:tasks.workunit.client.1.smithi078.stdout:3/664: dread d16/d26/d2b/d31/f3d [0,4194304] 0 2024-04-26T04:52:59.064 INFO:tasks.workunit.client.1.smithi078.stdout:5/579: dread d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:52:59.076 INFO:tasks.workunit.client.1.smithi078.stdout:4/666: sync 2024-04-26T04:52:59.078 INFO:tasks.workunit.client.1.smithi078.stdout:0/657: dread d2/d9/d18/db7/fd1 [0,4194304] 0 2024-04-26T04:52:59.132 INFO:tasks.workunit.client.1.smithi078.stdout:1/680: sync 2024-04-26T04:52:59.135 INFO:tasks.workunit.client.0.smithi077.stdout:3/249: rmdir d0 39 2024-04-26T04:52:59.137 INFO:tasks.workunit.client.0.smithi077.stdout:8/284: dwrite d4/d17/d35/d5b/f63 [0,4194304] 0 2024-04-26T04:52:59.137 INFO:tasks.workunit.client.0.smithi077.stdout:9/273: mknod d9/dd/d1b/c5f 0 2024-04-26T04:52:59.140 INFO:tasks.workunit.client.1.smithi078.stdout:2/614: write d8/f9a [423917,120912] 0 2024-04-26T04:52:59.141 INFO:tasks.workunit.client.1.smithi078.stdout:2/615: truncate d8/d11/d14/f8d 236132 0 2024-04-26T04:52:59.147 INFO:tasks.workunit.client.0.smithi077.stdout:0/239: mkdir d0/d19/d1e/d2f/d52 0 2024-04-26T04:52:59.151 INFO:tasks.workunit.client.0.smithi077.stdout:3/250: fsync d0/f3d 0 2024-04-26T04:52:59.155 INFO:tasks.workunit.client.0.smithi077.stdout:9/274: symlink d9/dd/d1b/d2b/l60 0 2024-04-26T04:52:59.157 INFO:tasks.workunit.client.1.smithi078.stdout:6/701: sync 2024-04-26T04:52:59.163 INFO:tasks.workunit.client.1.smithi078.stdout:7/614: write d9/da/d11/d13/d1a/d52/d5a/f53 [4853931,29058] 0 2024-04-26T04:52:59.163 INFO:tasks.workunit.client.1.smithi078.stdout:5/580: fsync d7/f87 0 2024-04-26T04:52:59.164 INFO:tasks.workunit.client.0.smithi077.stdout:0/240: mknod d0/d2c/d4e/c53 0 2024-04-26T04:52:59.170 INFO:tasks.workunit.client.0.smithi077.stdout:9/275: creat d9/d43/f61 x:0 0 0 2024-04-26T04:52:59.174 INFO:tasks.workunit.client.0.smithi077.stdout:0/241: mknod d0/d19/d1e/d2f/c54 0 2024-04-26T04:52:59.186 INFO:tasks.workunit.client.1.smithi078.stdout:8/590: truncate d0/d19/d52/d35/f8f 327243 0 2024-04-26T04:52:59.194 INFO:tasks.workunit.client.1.smithi078.stdout:4/667: symlink d0/d1d/de1/dd8/lf3 0 2024-04-26T04:52:59.194 INFO:tasks.workunit.client.1.smithi078.stdout:4/668: chown d0/de/d22/d27/db5 27469 1 2024-04-26T04:52:59.195 INFO:tasks.workunit.client.1.smithi078.stdout:4/669: chown d0/d1d/de1/dd8/lf3 8211286 1 2024-04-26T04:52:59.196 INFO:tasks.workunit.client.1.smithi078.stdout:4/670: truncate d0/de/d22/d71/fed 405181 0 2024-04-26T04:52:59.198 INFO:tasks.workunit.client.1.smithi078.stdout:8/591: fdatasync d0/f2f 0 2024-04-26T04:52:59.200 INFO:tasks.workunit.client.1.smithi078.stdout:6/702: symlink d2/de/d27/ldb 0 2024-04-26T04:52:59.202 INFO:tasks.workunit.client.1.smithi078.stdout:2/616: creat d8/d11/d14/fdc x:0 0 0 2024-04-26T04:52:59.208 INFO:tasks.workunit.client.1.smithi078.stdout:6/703: dread d2/de/d27/da9/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:52:59.208 INFO:tasks.workunit.client.1.smithi078.stdout:7/615: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d95 0 2024-04-26T04:52:59.209 INFO:tasks.workunit.client.1.smithi078.stdout:6/704: fdatasync d2/de/d27/f37 0 2024-04-26T04:52:59.209 INFO:tasks.workunit.client.1.smithi078.stdout:7/616: chown d9/da/d11/d13/d1a/d52/l5d 54957575 1 2024-04-26T04:52:59.210 INFO:tasks.workunit.client.1.smithi078.stdout:9/580: rename dc/df/f8f to dc/df/d37/fa9 0 2024-04-26T04:52:59.212 INFO:tasks.workunit.client.1.smithi078.stdout:6/705: mkdir d2/de/d27/da9/da8/ddc 0 2024-04-26T04:52:59.212 INFO:tasks.workunit.client.1.smithi078.stdout:8/592: dread d0/d19/d52/d35/f4e [0,4194304] 0 2024-04-26T04:52:59.213 INFO:tasks.workunit.client.1.smithi078.stdout:9/581: read dc/df/d27/f13 [1448685,58122] 0 2024-04-26T04:52:59.214 INFO:tasks.workunit.client.1.smithi078.stdout:6/706: dread - d2/de/d27/da9/d2a/dbf/fce zero size 2024-04-26T04:52:59.215 INFO:tasks.workunit.client.1.smithi078.stdout:7/617: dread d9/da/d11/d13/d1a/d52/fb5 [0,4194304] 0 2024-04-26T04:52:59.219 INFO:tasks.workunit.client.1.smithi078.stdout:7/618: dread f6 [4194304,4194304] 0 2024-04-26T04:52:59.221 INFO:tasks.workunit.client.1.smithi078.stdout:6/707: fsync d2/de/d27/da9/d2a/d4d/f78 0 2024-04-26T04:52:59.221 INFO:tasks.workunit.client.1.smithi078.stdout:4/671: dread d0/d1d/de1/d20/f2a [0,4194304] 0 2024-04-26T04:52:59.224 INFO:tasks.workunit.client.1.smithi078.stdout:8/593: rename d0/d19/d52/d35/d44/d6b/c79 to d0/d19/d52/d95/cbd 0 2024-04-26T04:52:59.227 INFO:tasks.workunit.client.1.smithi078.stdout:7/619: dwrite d9/da/d11/d13/f91 [0,4194304] 0 2024-04-26T04:52:59.227 INFO:tasks.workunit.client.1.smithi078.stdout:2/617: dread d8/d11/d18/d5d/d61/f1b [0,4194304] 0 2024-04-26T04:52:59.228 INFO:tasks.workunit.client.1.smithi078.stdout:2/618: read d8/d11/d14/f8d [130886,72995] 0 2024-04-26T04:52:59.228 INFO:tasks.workunit.client.1.smithi078.stdout:8/594: mkdir d0/d7a/dbe 0 2024-04-26T04:52:59.230 INFO:tasks.workunit.client.1.smithi078.stdout:4/672: rename d0/de/d22/d71/f79 to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/ff4 0 2024-04-26T04:52:59.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/619: truncate d8/f2b 2519509 0 2024-04-26T04:52:59.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/620: fsync d8/f1f 0 2024-04-26T04:52:59.245 INFO:tasks.workunit.client.1.smithi078.stdout:7/620: dread d9/da/d11/d13/d50/d8a/f9c [0,4194304] 0 2024-04-26T04:52:59.246 INFO:tasks.workunit.client.1.smithi078.stdout:7/621: read - d9/da/d33/d4b/d4c/d5b/d83/fac zero size 2024-04-26T04:52:59.247 INFO:tasks.workunit.client.1.smithi078.stdout:2/621: read d8/d11/d14/d69/d8f/f5c [136687,115877] 0 2024-04-26T04:52:59.250 INFO:tasks.workunit.client.1.smithi078.stdout:2/622: creat d8/d11/d6e/fdd x:0 0 0 2024-04-26T04:52:59.250 INFO:tasks.workunit.client.1.smithi078.stdout:0/658: symlink d2/d9/d60/d6a/ld8 0 2024-04-26T04:52:59.261 INFO:tasks.workunit.client.1.smithi078.stdout:7/622: creat d9/da/d11/d13/d50/fc8 x:0 0 0 2024-04-26T04:52:59.273 INFO:tasks.workunit.client.1.smithi078.stdout:5/581: symlink d7/da/d1e/lc0 0 2024-04-26T04:52:59.273 INFO:tasks.workunit.client.1.smithi078.stdout:5/582: chown d7/d18/d32/d37/f3d 31 1 2024-04-26T04:52:59.276 INFO:tasks.workunit.client.1.smithi078.stdout:5/583: creat d7/d11/fc1 x:0 0 0 2024-04-26T04:52:59.278 INFO:tasks.workunit.client.1.smithi078.stdout:5/584: unlink d7/d48/da9/lbc 0 2024-04-26T04:52:59.279 INFO:tasks.workunit.client.1.smithi078.stdout:5/585: dread - d7/d18/d1c/f95 zero size 2024-04-26T04:52:59.283 INFO:tasks.workunit.client.1.smithi078.stdout:5/586: creat d7/d21/d2f/fc2 x:0 0 0 2024-04-26T04:52:59.298 INFO:tasks.workunit.client.1.smithi078.stdout:9/582: write dc/f39 [4110925,41315] 0 2024-04-26T04:52:59.299 INFO:tasks.workunit.client.1.smithi078.stdout:1/681: rmdir d7/d2e 39 2024-04-26T04:52:59.299 INFO:tasks.workunit.client.1.smithi078.stdout:9/583: stat dc/df/d4b/l84 0 2024-04-26T04:52:59.303 INFO:tasks.workunit.client.1.smithi078.stdout:1/682: creat d7/d2e/fe0 x:0 0 0 2024-04-26T04:52:59.307 INFO:tasks.workunit.client.1.smithi078.stdout:1/683: link d7/d2e/c30 d7/d1e/d9e/db/d47/ce1 0 2024-04-26T04:52:59.308 INFO:tasks.workunit.client.1.smithi078.stdout:6/708: dwrite d2/de/d27/da9/da/d35/f3e [0,4194304] 0 2024-04-26T04:52:59.310 INFO:tasks.workunit.client.1.smithi078.stdout:1/684: creat d7/d1e/fe2 x:0 0 0 2024-04-26T04:52:59.312 INFO:tasks.workunit.client.1.smithi078.stdout:6/709: symlink d2/de/d27/da9/d2a/ldd 0 2024-04-26T04:52:59.316 INFO:tasks.workunit.client.1.smithi078.stdout:3/665: mknod d16/d26/d2e/ced 0 2024-04-26T04:52:59.317 INFO:tasks.workunit.client.1.smithi078.stdout:8/595: dwrite d0/d19/f2a [0,4194304] 0 2024-04-26T04:52:59.318 INFO:tasks.workunit.client.1.smithi078.stdout:9/584: dread dc/df/f28 [0,4194304] 0 2024-04-26T04:52:59.319 INFO:tasks.workunit.client.1.smithi078.stdout:6/710: creat d2/d1a/d38/dd5/fde x:0 0 0 2024-04-26T04:52:59.319 INFO:tasks.workunit.client.1.smithi078.stdout:8/596: stat d0/d19/d25/d36/f37 0 2024-04-26T04:52:59.319 INFO:tasks.workunit.client.1.smithi078.stdout:3/666: write d16/d26/d2b/d31/f59 [33551,10232] 0 2024-04-26T04:52:59.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/673: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:2/239: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:7/326: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:5/317: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:4/289: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:1/367: sync 2024-04-26T04:52:59.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/329: sync 2024-04-26T04:52:59.322 INFO:tasks.workunit.client.1.smithi078.stdout:1/685: link d7/d1e/d9e/f62 d7/d1e/d9e/db/d49/dba/fe3 0 2024-04-26T04:52:59.322 INFO:tasks.workunit.client.1.smithi078.stdout:6/711: dread d2/f14 [0,4194304] 0 2024-04-26T04:52:59.322 INFO:tasks.workunit.client.1.smithi078.stdout:2/623: getdents d8/d11/d6e 0 2024-04-26T04:52:59.323 INFO:tasks.workunit.client.1.smithi078.stdout:3/667: fsync d16/d26/d2e/d5f/d49/d77/fc6 0 2024-04-26T04:52:59.323 INFO:tasks.workunit.client.1.smithi078.stdout:1/686: readlink d7/d1e/d9e/db/ldf 0 2024-04-26T04:52:59.323 INFO:tasks.workunit.client.1.smithi078.stdout:2/624: readlink d8/d3e/ld1 0 2024-04-26T04:52:59.323 INFO:tasks.workunit.client.0.smithi077.stdout:1/368: dread d9/f44 [0,4194304] 0 2024-04-26T04:52:59.325 INFO:tasks.workunit.client.0.smithi077.stdout:5/318: write d1/d12/d21/d25/d31/d3c/f4e [421080,53539] 0 2024-04-26T04:52:59.325 INFO:tasks.workunit.client.1.smithi078.stdout:3/668: rename d16 to d16/d26/d2b/d31/d71/d8a/dbc/dee 22 2024-04-26T04:52:59.325 INFO:tasks.workunit.client.1.smithi078.stdout:2/625: write d8/d11/d14/d5b/fd0 [947993,46624] 0 2024-04-26T04:52:59.326 INFO:tasks.workunit.client.1.smithi078.stdout:1/687: write d7/d1e/d9e/dd/d20/fb0 [855648,14874] 0 2024-04-26T04:52:59.326 INFO:tasks.workunit.client.0.smithi077.stdout:5/319: write d1/d8/f70 [844281,128343] 0 2024-04-26T04:52:59.328 INFO:tasks.workunit.client.0.smithi077.stdout:7/327: rmdir da/d13/d22/d34 39 2024-04-26T04:52:59.328 INFO:tasks.workunit.client.1.smithi078.stdout:9/585: dread dc/df/d27/f5c [0,4194304] 0 2024-04-26T04:52:59.329 INFO:tasks.workunit.client.1.smithi078.stdout:2/626: write d8/f84 [714316,104290] 0 2024-04-26T04:52:59.329 INFO:tasks.workunit.client.1.smithi078.stdout:1/688: chown d7/d1e/d9e/d4c/f26 477571690 1 2024-04-26T04:52:59.330 INFO:tasks.workunit.client.1.smithi078.stdout:6/712: chown d2/d1a/c26 953 1 2024-04-26T04:52:59.333 INFO:tasks.workunit.client.1.smithi078.stdout:0/659: dwrite d2/d39/d96/fa4 [0,4194304] 0 2024-04-26T04:52:59.335 INFO:tasks.workunit.client.0.smithi077.stdout:7/328: dwrite da/d15/d1a/f79 [0,4194304] 0 2024-04-26T04:52:59.337 INFO:tasks.workunit.client.0.smithi077.stdout:5/320: fsync d1/d8/f49 0 2024-04-26T04:52:59.338 INFO:tasks.workunit.client.0.smithi077.stdout:5/321: chown d1/d12/d21/d25/d31/d3c/l56 6370335 1 2024-04-26T04:52:59.339 INFO:tasks.workunit.client.1.smithi078.stdout:3/669: creat d16/d26/d2e/d5f/d49/dad/db0/fef x:0 0 0 2024-04-26T04:52:59.346 INFO:tasks.workunit.client.1.smithi078.stdout:2/627: symlink d8/d3e/d52/lde 0 2024-04-26T04:52:59.348 INFO:tasks.workunit.client.0.smithi077.stdout:7/329: fsync da/d13/d22/f27 0 2024-04-26T04:52:59.348 INFO:tasks.workunit.client.1.smithi078.stdout:8/597: unlink d0/d19/d25/d36/f48 0 2024-04-26T04:52:59.350 INFO:tasks.workunit.client.0.smithi077.stdout:5/322: symlink d1/d12/d21/l78 0 2024-04-26T04:52:59.350 INFO:tasks.workunit.client.0.smithi077.stdout:5/323: stat d1/d8/c65 0 2024-04-26T04:52:59.351 INFO:tasks.workunit.client.1.smithi078.stdout:6/713: mkdir d2/d1a/d38/dd5/ddf 0 2024-04-26T04:52:59.352 INFO:tasks.workunit.client.0.smithi077.stdout:5/324: dread d1/d8/f10 [4194304,4194304] 0 2024-04-26T04:52:59.354 INFO:tasks.workunit.client.1.smithi078.stdout:0/660: symlink d2/d9/d18/d7d/ld9 0 2024-04-26T04:52:59.355 INFO:tasks.workunit.client.0.smithi077.stdout:7/330: creat da/d15/d1a/d39/f7a x:0 0 0 2024-04-26T04:52:59.357 INFO:tasks.workunit.client.0.smithi077.stdout:2/240: getdents d9/d15/d1a 0 2024-04-26T04:52:59.361 INFO:tasks.workunit.client.0.smithi077.stdout:7/331: stat da/d15/d1a/l6a 0 2024-04-26T04:52:59.362 INFO:tasks.workunit.client.0.smithi077.stdout:8/285: rename d4/f60 to d4/df/d22/f64 0 2024-04-26T04:52:59.363 INFO:tasks.workunit.client.0.smithi077.stdout:8/286: chown d4/df/d22/d2d/l38 578781590 1 2024-04-26T04:52:59.367 INFO:tasks.workunit.client.0.smithi077.stdout:5/325: symlink d1/d30/d33/d66/l79 0 2024-04-26T04:52:59.368 INFO:tasks.workunit.client.1.smithi078.stdout:1/689: fsync d7/dd7/fde 0 2024-04-26T04:52:59.368 INFO:tasks.workunit.client.1.smithi078.stdout:6/714: dwrite d2/de/d27/da9/f6b [0,4194304] 0 2024-04-26T04:52:59.368 INFO:tasks.workunit.client.1.smithi078.stdout:2/628: rename d8/d11/d18/d5d/fa8 to d8/d53/fdf 0 2024-04-26T04:52:59.369 INFO:tasks.workunit.client.0.smithi077.stdout:2/241: dwrite d9/d15/f37 [0,4194304] 0 2024-04-26T04:52:59.370 INFO:tasks.workunit.client.1.smithi078.stdout:0/661: mkdir d2/d1e/d5a/dda 0 2024-04-26T04:52:59.370 INFO:tasks.workunit.client.0.smithi077.stdout:3/251: rename d0/d9/d32/f2f to d0/d9/d32/d2a/f4c 0 2024-04-26T04:52:59.371 INFO:tasks.workunit.client.1.smithi078.stdout:6/715: write d2/de/d27/da9/d2a/dbf/d5f/fd4 [581817,78842] 0 2024-04-26T04:52:59.371 INFO:tasks.workunit.client.0.smithi077.stdout:3/252: stat d0/d9/d32/f35 0 2024-04-26T04:52:59.371 INFO:tasks.workunit.client.0.smithi077.stdout:2/242: write d9/fe [1680741,50130] 0 2024-04-26T04:52:59.372 INFO:tasks.workunit.client.0.smithi077.stdout:3/253: readlink d0/d9/d32/l16 0 2024-04-26T04:52:59.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/242: creat d0/d47/f55 x:0 0 0 2024-04-26T04:52:59.376 INFO:tasks.workunit.client.0.smithi077.stdout:5/326: mknod d1/d12/d21/c7a 0 2024-04-26T04:52:59.377 INFO:tasks.workunit.client.0.smithi077.stdout:0/243: fsync d0/f2 0 2024-04-26T04:52:59.380 INFO:tasks.workunit.client.0.smithi077.stdout:9/276: rename d9/d35/f53 to d9/f62 0 2024-04-26T04:52:59.381 INFO:tasks.workunit.client.0.smithi077.stdout:2/243: dwrite d9/d15/f31 [0,4194304] 0 2024-04-26T04:52:59.382 INFO:tasks.workunit.client.1.smithi078.stdout:8/598: write d0/d19/d25/d36/d58/f70 [961199,7791] 0 2024-04-26T04:52:59.383 INFO:tasks.workunit.client.0.smithi077.stdout:3/254: mkdir d0/d7/d1f/d4d 0 2024-04-26T04:52:59.386 INFO:tasks.workunit.client.1.smithi078.stdout:4/674: rename d0/d1d/de1/d20/d31/d47/db2/d4c/c87 to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/cf5 0 2024-04-26T04:52:59.392 INFO:tasks.workunit.client.1.smithi078.stdout:0/662: write d2/f3 [95946,125723] 0 2024-04-26T04:52:59.393 INFO:tasks.workunit.client.1.smithi078.stdout:0/663: fsync d2/d9/d31/f4f 0 2024-04-26T04:52:59.394 INFO:tasks.workunit.client.1.smithi078.stdout:6/716: creat d2/d1a/d38/fe0 x:0 0 0 2024-04-26T04:52:59.395 INFO:tasks.workunit.client.0.smithi077.stdout:6/330: rename d8/c4b to d8/d1c/c73 0 2024-04-26T04:52:59.400 INFO:tasks.workunit.client.0.smithi077.stdout:3/255: creat d0/d9/d32/d2a/f4e x:0 0 0 2024-04-26T04:52:59.401 INFO:tasks.workunit.client.0.smithi077.stdout:5/327: mknod d1/d8/d72/c7b 0 2024-04-26T04:52:59.404 INFO:tasks.workunit.client.1.smithi078.stdout:4/675: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [0,4194304] 0 2024-04-26T04:52:59.406 INFO:tasks.workunit.client.0.smithi077.stdout:1/369: rename d9/db/d27 to d9/d53/d6e 0 2024-04-26T04:52:59.407 INFO:tasks.workunit.client.1.smithi078.stdout:4/676: fsync d0/d1d/de1/d20/d31/d47/db2/d44/d55/d5e/d75/fd1 0 2024-04-26T04:52:59.407 INFO:tasks.workunit.client.0.smithi077.stdout:3/256: dwrite d0/d9/d32/f21 [0,4194304] 0 2024-04-26T04:52:59.407 INFO:tasks.workunit.client.1.smithi078.stdout:9/586: dwrite dc/df/d17/f2e [0,4194304] 0 2024-04-26T04:52:59.410 INFO:tasks.workunit.client.1.smithi078.stdout:6/717: creat d2/fe1 x:0 0 0 2024-04-26T04:52:59.411 INFO:tasks.workunit.client.0.smithi077.stdout:2/244: symlink d9/dc/d1d/l57 0 2024-04-26T04:52:59.411 INFO:tasks.workunit.client.0.smithi077.stdout:2/245: fsync d9/d15/f31 0 2024-04-26T04:52:59.412 INFO:tasks.workunit.client.0.smithi077.stdout:2/246: write d9/d15/d1a/f30 [1522683,111036] 0 2024-04-26T04:52:59.413 INFO:tasks.workunit.client.0.smithi077.stdout:2/247: chown d9/d15/d1a/f3f 130 1 2024-04-26T04:52:59.415 INFO:tasks.workunit.client.1.smithi078.stdout:8/599: mknod d0/d19/d52/cbf 0 2024-04-26T04:52:59.420 INFO:tasks.workunit.client.0.smithi077.stdout:5/328: creat d1/d12/d28/d38/f7c x:0 0 0 2024-04-26T04:52:59.422 INFO:tasks.workunit.client.0.smithi077.stdout:0/244: creat d0/d47/f56 x:0 0 0 2024-04-26T04:52:59.426 INFO:tasks.workunit.client.1.smithi078.stdout:0/664: mknod d2/d9/d78/d84/dd7/cdb 0 2024-04-26T04:52:59.428 INFO:tasks.workunit.client.0.smithi077.stdout:1/370: unlink d9/l4f 0 2024-04-26T04:52:59.430 INFO:tasks.workunit.client.0.smithi077.stdout:3/257: symlink d0/d7/d15/l4f 0 2024-04-26T04:52:59.438 INFO:tasks.workunit.client.0.smithi077.stdout:9/277: link d9/dd/d1b/d2b/d32/d36/l3f d9/dd/d1b/d2b/d32/l63 0 2024-04-26T04:52:59.439 INFO:tasks.workunit.client.0.smithi077.stdout:9/278: dread - d9/d43/f61 zero size 2024-04-26T04:52:59.439 INFO:tasks.workunit.client.0.smithi077.stdout:9/279: read - d9/dd/d3d/f59 zero size 2024-04-26T04:52:59.440 INFO:tasks.workunit.client.1.smithi078.stdout:2/629: dwrite d8/d11/d6e/f7b [0,4194304] 0 2024-04-26T04:52:59.441 INFO:tasks.workunit.client.0.smithi077.stdout:2/248: creat d9/d15/d44/f58 x:0 0 0 2024-04-26T04:52:59.445 INFO:tasks.workunit.client.1.smithi078.stdout:1/690: truncate d7/f57 3174787 0 2024-04-26T04:52:59.450 INFO:tasks.workunit.client.1.smithi078.stdout:0/665: dread d2/d39/d96/fc8 [0,4194304] 0 2024-04-26T04:52:59.454 INFO:tasks.workunit.client.0.smithi077.stdout:3/258: unlink d0/d9/d32/d29/f41 0 2024-04-26T04:52:59.454 INFO:tasks.workunit.client.0.smithi077.stdout:3/259: write d0/f3d [3321598,15574] 0 2024-04-26T04:52:59.454 INFO:tasks.workunit.client.0.smithi077.stdout:3/260: read - d0/d9/d32/f28 zero size 2024-04-26T04:52:59.455 INFO:tasks.workunit.client.0.smithi077.stdout:6/331: rename d8/f39 to d8/f74 0 2024-04-26T04:52:59.455 INFO:tasks.workunit.client.1.smithi078.stdout:4/677: write d0/de/f7e [396417,102129] 0 2024-04-26T04:52:59.456 INFO:tasks.workunit.client.1.smithi078.stdout:1/691: creat d7/d1e/d9e/dd/d20/fe4 x:0 0 0 2024-04-26T04:52:59.458 INFO:tasks.workunit.client.0.smithi077.stdout:0/245: symlink d0/l57 0 2024-04-26T04:52:59.461 INFO:tasks.workunit.client.0.smithi077.stdout:4/290: rename d4/d17/d18/d43/d53/d56/f1b to d4/de/d15/f62 0 2024-04-26T04:52:59.462 INFO:tasks.workunit.client.1.smithi078.stdout:9/587: link dc/df/d27/d81/c6c dc/df/d4b/d22/d2d/caa 0 2024-04-26T04:52:59.463 INFO:tasks.workunit.client.0.smithi077.stdout:4/291: write d4/de/d15/d1f/f32 [2466574,125121] 0 2024-04-26T04:52:59.464 INFO:tasks.workunit.client.0.smithi077.stdout:4/292: truncate d4/de/d15/f16 4512157 0 2024-04-26T04:52:59.467 INFO:tasks.workunit.client.1.smithi078.stdout:5/587: sync 2024-04-26T04:52:59.467 INFO:tasks.workunit.client.1.smithi078.stdout:7/623: sync 2024-04-26T04:52:59.467 INFO:tasks.workunit.client.1.smithi078.stdout:5/588: chown d7/d21/d2f 674820949 1 2024-04-26T04:52:59.468 INFO:tasks.workunit.client.1.smithi078.stdout:5/589: truncate d7/d48/d57/faa 1340369 0 2024-04-26T04:52:59.469 INFO:tasks.workunit.client.1.smithi078.stdout:4/678: unlink d0/de/d22/d27/d56/d5a/ddc/cf1 0 2024-04-26T04:52:59.470 INFO:tasks.workunit.client.1.smithi078.stdout:4/679: write d0/de/f7e [880285,20460] 0 2024-04-26T04:52:59.470 INFO:tasks.workunit.client.1.smithi078.stdout:8/600: write d0/d19/d52/faa [1270746,61493] 0 2024-04-26T04:52:59.475 INFO:tasks.workunit.client.1.smithi078.stdout:2/630: creat d8/d11/d14/d69/d8f/d34/fe0 x:0 0 0 2024-04-26T04:52:59.475 INFO:tasks.workunit.client.1.smithi078.stdout:4/680: dread d0/d1d/de1/d20/d31/f73 [0,4194304] 0 2024-04-26T04:52:59.476 INFO:tasks.workunit.client.0.smithi077.stdout:6/332: creat d8/d2a/d57/f75 x:0 0 0 2024-04-26T04:52:59.476 INFO:tasks.workunit.client.0.smithi077.stdout:6/333: fsync d8/fb 0 2024-04-26T04:52:59.477 INFO:tasks.workunit.client.1.smithi078.stdout:0/666: mknod d2/d9/d78/cdc 0 2024-04-26T04:52:59.483 INFO:tasks.workunit.client.1.smithi078.stdout:7/624: creat d9/d57/fc9 x:0 0 0 2024-04-26T04:52:59.484 INFO:tasks.workunit.client.1.smithi078.stdout:1/692: dread d7/d1e/d9e/f3e [0,4194304] 0 2024-04-26T04:52:59.485 INFO:tasks.workunit.client.1.smithi078.stdout:5/590: fsync d7/da/d1e/f4c 0 2024-04-26T04:52:59.486 INFO:tasks.workunit.client.1.smithi078.stdout:5/591: write d7/d11/f66 [4844449,31473] 0 2024-04-26T04:52:59.486 INFO:tasks.workunit.client.0.smithi077.stdout:7/332: rename da/d13/c33 to da/d6b/c7b 0 2024-04-26T04:52:59.490 INFO:tasks.workunit.client.0.smithi077.stdout:4/293: mkdir d4/d17/d18/d43/d63 0 2024-04-26T04:52:59.492 INFO:tasks.workunit.client.1.smithi078.stdout:8/601: mknod d0/d19/d25/d32/d8a/d92/cc0 0 2024-04-26T04:52:59.492 INFO:tasks.workunit.client.1.smithi078.stdout:8/602: fdatasync d0/d19/d52/d35/d44/d6b/f9b 0 2024-04-26T04:52:59.493 INFO:tasks.workunit.client.0.smithi077.stdout:3/261: link d0/d9/d32/f36 d0/d7/d15/f50 0 2024-04-26T04:52:59.494 INFO:tasks.workunit.client.1.smithi078.stdout:2/631: mknod d8/d3e/d52/ce1 0 2024-04-26T04:52:59.495 INFO:tasks.workunit.client.1.smithi078.stdout:2/632: chown d8/d3e/d52/f9f 14 1 2024-04-26T04:52:59.496 INFO:tasks.workunit.client.1.smithi078.stdout:4/681: mkdir d0/d1d/de1/d20/d31/d47/db2/d4c/df6 0 2024-04-26T04:52:59.497 INFO:tasks.workunit.client.0.smithi077.stdout:6/334: creat d8/d55/f76 x:0 0 0 2024-04-26T04:52:59.497 INFO:tasks.workunit.client.0.smithi077.stdout:6/335: readlink d8/d1c/l31 0 2024-04-26T04:52:59.498 INFO:tasks.workunit.client.0.smithi077.stdout:5/329: link d1/l3d d1/d12/l7d 0 2024-04-26T04:52:59.499 INFO:tasks.workunit.client.0.smithi077.stdout:5/330: write d1/d8/f70 [1457852,87466] 0 2024-04-26T04:52:59.500 INFO:tasks.workunit.client.0.smithi077.stdout:0/246: symlink d0/d19/d1e/d2f/d52/l58 0 2024-04-26T04:52:59.506 INFO:tasks.workunit.client.0.smithi077.stdout:4/294: mkdir d4/d17/d18/d43/d53/d64 0 2024-04-26T04:52:59.507 INFO:tasks.workunit.client.0.smithi077.stdout:3/262: creat d0/d7/d1f/f51 x:0 0 0 2024-04-26T04:52:59.508 INFO:tasks.workunit.client.1.smithi078.stdout:3/670: sync 2024-04-26T04:52:59.508 INFO:tasks.workunit.client.1.smithi078.stdout:6/718: sync 2024-04-26T04:52:59.509 INFO:tasks.workunit.client.1.smithi078.stdout:5/592: creat d7/d18/d1c/d5a/fc3 x:0 0 0 2024-04-26T04:52:59.509 INFO:tasks.workunit.client.1.smithi078.stdout:1/693: dwrite d7/d1e/d9e/db/d47/f90 [0,4194304] 0 2024-04-26T04:52:59.510 INFO:tasks.workunit.client.0.smithi077.stdout:4/295: dread f2 [0,4194304] 0 2024-04-26T04:52:59.510 INFO:tasks.workunit.client.1.smithi078.stdout:1/694: chown d7/lc4 29258 1 2024-04-26T04:52:59.511 INFO:tasks.workunit.client.0.smithi077.stdout:5/331: mkdir d1/d8/d7e 0 2024-04-26T04:52:59.511 INFO:tasks.workunit.client.1.smithi078.stdout:6/719: write d2/de/d27/da9/f9d [476615,55976] 0 2024-04-26T04:52:59.513 INFO:tasks.workunit.client.1.smithi078.stdout:1/695: write d7/d1e/d9e/dd/fd5 [490412,8336] 0 2024-04-26T04:52:59.514 INFO:tasks.workunit.client.0.smithi077.stdout:0/247: chown d0/d19/d1e/c2a 109 1 2024-04-26T04:52:59.515 INFO:tasks.workunit.client.1.smithi078.stdout:1/696: chown d7/d1e/d9e/db/d49/dba 19193 1 2024-04-26T04:52:59.519 INFO:tasks.workunit.client.0.smithi077.stdout:9/280: rename d9/d43/c55 to d9/dd/d1b/d2b/c64 0 2024-04-26T04:52:59.522 INFO:tasks.workunit.client.1.smithi078.stdout:0/667: link d2/d9/d31/f50 d2/d1e/d5a/da3/fdd 0 2024-04-26T04:52:59.523 INFO:tasks.workunit.client.0.smithi077.stdout:9/281: dread d9/dd/d1b/d2b/d32/d36/f5e [0,4194304] 0 2024-04-26T04:52:59.523 INFO:tasks.workunit.client.0.smithi077.stdout:6/336: link d8/d52/d53/d5e/f69 d8/d2a/d57/f77 0 2024-04-26T04:52:59.524 INFO:tasks.workunit.client.1.smithi078.stdout:8/603: write d0/d19/d52/d35/f5f [1155144,38203] 0 2024-04-26T04:52:59.526 INFO:tasks.workunit.client.0.smithi077.stdout:3/263: write d0/d9/d32/d29/f4b [196312,96960] 0 2024-04-26T04:52:59.526 INFO:tasks.workunit.client.0.smithi077.stdout:7/333: write da/d13/d22/f25 [1736445,29364] 0 2024-04-26T04:52:59.527 INFO:tasks.workunit.client.0.smithi077.stdout:3/264: readlink d0/d7/l24 0 2024-04-26T04:52:59.528 INFO:tasks.workunit.client.0.smithi077.stdout:5/332: chown d1/d8/fc 5322203 1 2024-04-26T04:52:59.530 INFO:tasks.workunit.client.0.smithi077.stdout:9/282: dread d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:52:59.531 INFO:tasks.workunit.client.1.smithi078.stdout:3/671: mkdir d16/d26/d2e/d5f/d49/dad/df0 0 2024-04-26T04:52:59.532 INFO:tasks.workunit.client.0.smithi077.stdout:2/249: rename d9/dc/d1d/d45/c52 to d9/d15/c59 0 2024-04-26T04:52:59.532 INFO:tasks.workunit.client.0.smithi077.stdout:2/250: chown f7 803636 1 2024-04-26T04:52:59.532 INFO:tasks.workunit.client.1.smithi078.stdout:5/593: mkdir d7/d48/d57/d65/d46/dc4 0 2024-04-26T04:52:59.533 INFO:tasks.workunit.client.0.smithi077.stdout:2/251: write d9/dc/d1d/f4a [527416,66299] 0 2024-04-26T04:52:59.535 INFO:tasks.workunit.client.0.smithi077.stdout:4/296: creat d4/d17/d18/d43/d53/d64/f65 x:0 0 0 2024-04-26T04:52:59.536 INFO:tasks.workunit.client.0.smithi077.stdout:2/252: dread d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:52:59.536 INFO:tasks.workunit.client.0.smithi077.stdout:7/334: mknod da/d13/c7c 0 2024-04-26T04:52:59.537 INFO:tasks.workunit.client.0.smithi077.stdout:7/335: chown da/d15/d1a/d39/f71 65 1 2024-04-26T04:52:59.538 INFO:tasks.workunit.client.1.smithi078.stdout:5/594: dwrite d7/d18/d32/d37/d79/fad [0,4194304] 0 2024-04-26T04:52:59.540 INFO:tasks.workunit.client.0.smithi077.stdout:3/265: rmdir d0/d9/d32/d2a 39 2024-04-26T04:52:59.544 INFO:tasks.workunit.client.0.smithi077.stdout:0/248: mkdir d0/d19/d1e/d59 0 2024-04-26T04:52:59.547 INFO:tasks.workunit.client.1.smithi078.stdout:9/588: getdents dc/df/d4b/d22/d2d 0 2024-04-26T04:52:59.577 INFO:tasks.workunit.client.1.smithi078.stdout:8/604: rename d0/d19/d52/d3c/l6e to d0/d19/d52/d95/lc1 0 2024-04-26T04:52:59.579 INFO:tasks.workunit.client.1.smithi078.stdout:7/625: creat d9/da/d11/d13/d1a/d52/fca x:0 0 0 2024-04-26T04:52:59.579 INFO:tasks.workunit.client.1.smithi078.stdout:2/633: write d8/d11/f48 [997401,88782] 0 2024-04-26T04:52:59.581 INFO:tasks.workunit.client.1.smithi078.stdout:0/668: write d2/d9/d18/f1c [4162602,5399] 0 2024-04-26T04:52:59.582 INFO:tasks.workunit.client.0.smithi077.stdout:4/297: mknod d4/de/d15/d1f/d25/c66 0 2024-04-26T04:52:59.583 INFO:tasks.workunit.client.1.smithi078.stdout:6/720: symlink d2/de/d27/da9/le2 0 2024-04-26T04:52:59.583 INFO:tasks.workunit.client.1.smithi078.stdout:6/721: dread - d2/f16 zero size 2024-04-26T04:52:59.583 INFO:tasks.workunit.client.0.smithi077.stdout:2/253: unlink d9/d15/d1a/c26 0 2024-04-26T04:52:59.586 INFO:tasks.workunit.client.0.smithi077.stdout:3/266: rmdir d0/d9 39 2024-04-26T04:52:59.589 INFO:tasks.workunit.client.1.smithi078.stdout:5/595: mkdir d7/d18/d32/d37/d8f/dc5 0 2024-04-26T04:52:59.590 INFO:tasks.workunit.client.0.smithi077.stdout:0/249: truncate d0/d47/f16 4328375 0 2024-04-26T04:52:59.592 INFO:tasks.workunit.client.1.smithi078.stdout:4/682: creat d0/de/ff7 x:0 0 0 2024-04-26T04:52:59.593 INFO:tasks.workunit.client.1.smithi078.stdout:4/683: chown d0/de/d22/d27/d56/d5a 191 1 2024-04-26T04:52:59.593 INFO:tasks.workunit.client.0.smithi077.stdout:9/283: symlink d9/dd/l65 0 2024-04-26T04:52:59.594 INFO:tasks.workunit.client.1.smithi078.stdout:4/684: dread - d0/d1d/de1/d20/d31/d47/db2/d44/d55/fe5 zero size 2024-04-26T04:52:59.596 INFO:tasks.workunit.client.0.smithi077.stdout:6/337: rename d8/l1b to d8/d2a/d57/l78 0 2024-04-26T04:52:59.596 INFO:tasks.workunit.client.1.smithi078.stdout:9/589: chown dc/df/d4b/d3e/c56 69195097 1 2024-04-26T04:52:59.597 INFO:tasks.workunit.client.1.smithi078.stdout:9/590: chown dc/df/d27/l9a 2 1 2024-04-26T04:52:59.597 INFO:tasks.workunit.client.1.smithi078.stdout:9/591: chown dc/df/d4b/d3e/d80 342250 1 2024-04-26T04:52:59.598 INFO:tasks.workunit.client.1.smithi078.stdout:8/605: creat d0/d19/d52/d35/d73/fc2 x:0 0 0 2024-04-26T04:52:59.599 INFO:tasks.workunit.client.0.smithi077.stdout:2/254: symlink d9/dc/d1d/d34/l5a 0 2024-04-26T04:52:59.601 INFO:tasks.workunit.client.1.smithi078.stdout:7/626: symlink d9/da/d33/d4b/d4c/d5b/lcb 0 2024-04-26T04:52:59.602 INFO:tasks.workunit.client.1.smithi078.stdout:3/672: creat d16/d26/d2e/d5f/d49/dad/db0/db7/de4/ff1 x:0 0 0 2024-04-26T04:52:59.605 INFO:tasks.workunit.client.0.smithi077.stdout:6/338: creat d8/d1c/f79 x:0 0 0 2024-04-26T04:52:59.606 INFO:tasks.workunit.client.0.smithi077.stdout:9/284: dread d9/f11 [0,4194304] 0 2024-04-26T04:52:59.608 INFO:tasks.workunit.client.0.smithi077.stdout:3/267: symlink d0/l52 0 2024-04-26T04:52:59.609 INFO:tasks.workunit.client.0.smithi077.stdout:5/333: getdents d1 0 2024-04-26T04:52:59.610 INFO:tasks.workunit.client.0.smithi077.stdout:0/250: mkdir d0/d19/d1e/d59/d5a 0 2024-04-26T04:52:59.612 INFO:tasks.workunit.client.0.smithi077.stdout:9/285: dwrite d9/dd/f5d [0,4194304] 0 2024-04-26T04:52:59.614 INFO:tasks.workunit.client.0.smithi077.stdout:6/339: creat d8/d1c/f7a x:0 0 0 2024-04-26T04:52:59.617 INFO:tasks.workunit.client.0.smithi077.stdout:5/334: rmdir d1 39 2024-04-26T04:52:59.624 INFO:tasks.workunit.client.0.smithi077.stdout:7/336: getdents da/d13/d22/d34 0 2024-04-26T04:52:59.624 INFO:tasks.workunit.client.0.smithi077.stdout:7/337: chown da/f36 1085728 1 2024-04-26T04:52:59.628 INFO:tasks.workunit.client.0.smithi077.stdout:6/340: creat d8/d55/f7b x:0 0 0 2024-04-26T04:52:59.629 INFO:tasks.workunit.client.0.smithi077.stdout:6/341: read f4 [3540572,65503] 0 2024-04-26T04:52:59.631 INFO:tasks.workunit.client.0.smithi077.stdout:3/268: mknod d0/d9/d32/d29/c53 0 2024-04-26T04:52:59.634 INFO:tasks.workunit.client.0.smithi077.stdout:5/335: dread d1/d30/f36 [0,4194304] 0 2024-04-26T04:52:59.635 INFO:tasks.workunit.client.0.smithi077.stdout:5/336: write d1/d42/f69 [167073,17842] 0 2024-04-26T04:52:59.635 INFO:tasks.workunit.client.0.smithi077.stdout:0/251: mknod d0/c5b 0 2024-04-26T04:52:59.637 INFO:tasks.workunit.client.0.smithi077.stdout:5/337: readlink d1/d12/d21/d25/d31/d3c/l56 0 2024-04-26T04:52:59.639 INFO:tasks.workunit.client.0.smithi077.stdout:5/338: dread d1/d12/f2a [0,4194304] 0 2024-04-26T04:52:59.643 INFO:tasks.workunit.client.0.smithi077.stdout:0/252: dwrite d0/d47/ff [0,4194304] 0 2024-04-26T04:52:59.644 INFO:tasks.workunit.client.1.smithi078.stdout:1/697: unlink d7/d1e/d9e/d92/ld1 0 2024-04-26T04:52:59.646 INFO:tasks.workunit.client.1.smithi078.stdout:5/596: mknod d7/da/d1e/cc6 0 2024-04-26T04:52:59.650 INFO:tasks.workunit.client.0.smithi077.stdout:2/255: dwrite d9/d15/d1a/d2b/f46 [0,4194304] 0 2024-04-26T04:52:59.652 INFO:tasks.workunit.client.0.smithi077.stdout:5/339: dwrite d1/d12/d21/d25/d31/d3c/f68 [0,4194304] 0 2024-04-26T04:52:59.653 INFO:tasks.workunit.client.0.smithi077.stdout:5/340: dread - d1/d12/d21/d25/d31/f71 zero size 2024-04-26T04:52:59.654 INFO:tasks.workunit.client.1.smithi078.stdout:8/606: unlink d0/d19/d52/d35/c55 0 2024-04-26T04:52:59.655 INFO:tasks.workunit.client.1.smithi078.stdout:8/607: fsync d0/d19/d52/d5c/f74 0 2024-04-26T04:52:59.659 INFO:tasks.workunit.client.1.smithi078.stdout:2/634: creat d8/db7/fe2 x:0 0 0 2024-04-26T04:52:59.661 INFO:tasks.workunit.client.1.smithi078.stdout:7/627: symlink d9/da/d33/d4b/d4c/d6c/lcc 0 2024-04-26T04:52:59.662 INFO:tasks.workunit.client.1.smithi078.stdout:0/669: mknod d2/d9/d78/cde 0 2024-04-26T04:52:59.664 INFO:tasks.workunit.client.1.smithi078.stdout:3/673: fsync d16/d26/d2b/d31/d71/d8a/d98/fa1 0 2024-04-26T04:52:59.666 INFO:tasks.workunit.client.0.smithi077.stdout:9/286: creat d9/dd/d1b/f66 x:0 0 0 2024-04-26T04:52:59.667 INFO:tasks.workunit.client.0.smithi077.stdout:7/338: write da/d13/d22/f46 [28964,65248] 0 2024-04-26T04:52:59.669 INFO:tasks.workunit.client.1.smithi078.stdout:1/698: creat d7/d1e/d9e/d92/fe5 x:0 0 0 2024-04-26T04:52:59.670 INFO:tasks.workunit.client.0.smithi077.stdout:0/253: rmdir d0/d28/d40 39 2024-04-26T04:52:59.670 INFO:tasks.workunit.client.0.smithi077.stdout:0/254: stat d0/d28/l33 0 2024-04-26T04:52:59.672 INFO:tasks.workunit.client.1.smithi078.stdout:5/597: rmdir d7/d18/d32 39 2024-04-26T04:52:59.675 INFO:tasks.workunit.client.1.smithi078.stdout:1/699: dwrite d7/d1e/d9e/db/d37/f88 [0,4194304] 0 2024-04-26T04:52:59.677 INFO:tasks.workunit.client.1.smithi078.stdout:1/700: chown d7/d1e/d9e/db/d49/d93/faf 155 1 2024-04-26T04:52:59.684 INFO:tasks.workunit.client.0.smithi077.stdout:3/269: dwrite d0/d9/d32/f36 [0,4194304] 0 2024-04-26T04:52:59.685 INFO:tasks.workunit.client.0.smithi077.stdout:3/270: fdatasync d0/fb 0 2024-04-26T04:52:59.686 INFO:tasks.workunit.client.1.smithi078.stdout:2/635: dread d8/d11/f48 [0,4194304] 0 2024-04-26T04:52:59.686 INFO:tasks.workunit.client.0.smithi077.stdout:2/256: mkdir d9/d15/d1a/d5b 0 2024-04-26T04:52:59.687 INFO:tasks.workunit.client.0.smithi077.stdout:3/271: chown d0/d9/d32/d29/c53 41013839 1 2024-04-26T04:52:59.688 INFO:tasks.workunit.client.0.smithi077.stdout:2/257: write d9/dc/f41 [929366,101423] 0 2024-04-26T04:52:59.689 INFO:tasks.workunit.client.1.smithi078.stdout:7/628: rmdir d9/da/d11/d13/d1a/d52/d5a/d41/d95 39 2024-04-26T04:52:59.690 INFO:tasks.workunit.client.1.smithi078.stdout:0/670: creat d2/dc1/fdf x:0 0 0 2024-04-26T04:52:59.691 INFO:tasks.workunit.client.0.smithi077.stdout:9/287: mknod d9/dd/d1b/d2b/c67 0 2024-04-26T04:52:59.692 INFO:tasks.workunit.client.1.smithi078.stdout:3/674: rmdir d16/d26/d2b/d31 39 2024-04-26T04:52:59.695 INFO:tasks.workunit.client.1.smithi078.stdout:6/722: rename d2/de/d27/da9/d3a/d69/daa/dba/lbc to d2/de/d27/da9/d3a/d69/le3 0 2024-04-26T04:52:59.696 INFO:tasks.workunit.client.1.smithi078.stdout:9/592: truncate dc/df/d27/f9b 2443502 0 2024-04-26T04:52:59.698 INFO:tasks.workunit.client.0.smithi077.stdout:6/342: link d8/d2a/c5d d8/d1c/c7c 0 2024-04-26T04:52:59.702 INFO:tasks.workunit.client.0.smithi077.stdout:0/255: dread d0/d19/d1e/f20 [0,4194304] 0 2024-04-26T04:52:59.703 INFO:tasks.workunit.client.1.smithi078.stdout:1/701: dread d7/d1e/d9e/db/d49/f6c [0,4194304] 0 2024-04-26T04:52:59.705 INFO:tasks.workunit.client.0.smithi077.stdout:3/272: fsync d0/f38 0 2024-04-26T04:52:59.708 INFO:tasks.workunit.client.0.smithi077.stdout:2/258: unlink d9/dc/d1d/l33 0 2024-04-26T04:52:59.710 INFO:tasks.workunit.client.1.smithi078.stdout:2/636: rmdir d8/d11/d14/d69/d8f/d34/d70 39 2024-04-26T04:52:59.710 INFO:tasks.workunit.client.0.smithi077.stdout:2/259: write d9/dc/d1d/d34/f36 [3411355,41404] 0 2024-04-26T04:52:59.712 INFO:tasks.workunit.client.0.smithi077.stdout:3/273: dwrite d0/d9/d32/f3b [0,4194304] 0 2024-04-26T04:52:59.712 INFO:tasks.workunit.client.0.smithi077.stdout:2/260: dread - d9/d15/d1a/f3f zero size 2024-04-26T04:52:59.714 INFO:tasks.workunit.client.0.smithi077.stdout:2/261: dread - d9/d15/d1a/d2b/f53 zero size 2024-04-26T04:52:59.715 INFO:tasks.workunit.client.1.smithi078.stdout:0/671: fdatasync d2/d9/d18/d7d/da9/dab/dc6/f9c 0 2024-04-26T04:52:59.716 INFO:tasks.workunit.client.0.smithi077.stdout:6/343: creat d8/d2a/d57/f7d x:0 0 0 2024-04-26T04:52:59.717 INFO:tasks.workunit.client.0.smithi077.stdout:6/344: dread d8/d52/d53/f65 [0,4194304] 0 2024-04-26T04:52:59.718 INFO:tasks.workunit.client.1.smithi078.stdout:3/675: chown d16/l24 1 1 2024-04-26T04:52:59.719 INFO:tasks.workunit.client.1.smithi078.stdout:9/593: dread dc/df/f1d [0,4194304] 0 2024-04-26T04:52:59.719 INFO:tasks.workunit.client.1.smithi078.stdout:7/629: dwrite f6 [4194304,4194304] 0 2024-04-26T04:52:59.721 INFO:tasks.workunit.client.1.smithi078.stdout:4/685: rename d0/de/d22/d27/d56/f61 to d0/d1d/db4/ff8 0 2024-04-26T04:52:59.732 INFO:tasks.workunit.client.1.smithi078.stdout:6/723: unlink d2/de/d27/d58/l9e 0 2024-04-26T04:52:59.733 INFO:tasks.workunit.client.0.smithi077.stdout:5/341: link d1/d42/d62/d6b/c74 d1/d12/d21/d25/c7f 0 2024-04-26T04:52:59.733 INFO:tasks.workunit.client.1.smithi078.stdout:6/724: write d2/de/d27/da9/f6b [1975653,107566] 0 2024-04-26T04:52:59.733 INFO:tasks.workunit.client.0.smithi077.stdout:5/342: write d1/d8/f10 [2917957,104666] 0 2024-04-26T04:52:59.736 INFO:tasks.workunit.client.1.smithi078.stdout:5/598: mkdir d7/d18/d32/d37/dc7 0 2024-04-26T04:52:59.740 INFO:tasks.workunit.client.0.smithi077.stdout:7/339: truncate da/d15/d1a/f20 734186 0 2024-04-26T04:52:59.742 INFO:tasks.workunit.client.1.smithi078.stdout:1/702: chown d7/c9a 913 1 2024-04-26T04:52:59.747 INFO:tasks.workunit.client.0.smithi077.stdout:6/345: unlink d8/f13 0 2024-04-26T04:52:59.751 INFO:tasks.workunit.client.1.smithi078.stdout:4/686: creat d0/d1d/de1/d20/d31/d47/db2/ff9 x:0 0 0 2024-04-26T04:52:59.752 INFO:tasks.workunit.client.0.smithi077.stdout:6/346: dwrite d8/f48 [0,4194304] 0 2024-04-26T04:52:59.755 INFO:tasks.workunit.client.1.smithi078.stdout:3/676: dread d16/f8d [0,4194304] 0 2024-04-26T04:52:59.760 INFO:tasks.workunit.client.1.smithi078.stdout:5/599: mknod d7/d21/d2f/cc8 0 2024-04-26T04:52:59.761 INFO:tasks.workunit.client.1.smithi078.stdout:8/608: getdents d0/d19/d52/d95 0 2024-04-26T04:52:59.765 INFO:tasks.workunit.client.0.smithi077.stdout:7/340: mkdir da/d13/d22/d7d 0 2024-04-26T04:52:59.766 INFO:tasks.workunit.client.0.smithi077.stdout:9/288: getdents d9/d43 0 2024-04-26T04:52:59.768 INFO:tasks.workunit.client.1.smithi078.stdout:6/725: write d2/d1a/d5c/f6d [1622633,29552] 0 2024-04-26T04:52:59.769 INFO:tasks.workunit.client.1.smithi078.stdout:6/726: write d2/d52/fbd [1917025,101868] 0 2024-04-26T04:52:59.771 INFO:tasks.workunit.client.0.smithi077.stdout:3/274: link d0/f3d d0/d7/f54 0 2024-04-26T04:52:59.772 INFO:tasks.workunit.client.0.smithi077.stdout:7/341: dwrite da/d15/d1a/d39/f7a [0,4194304] 0 2024-04-26T04:52:59.773 INFO:tasks.workunit.client.1.smithi078.stdout:1/703: write d7/d1e/d9e/db/d49/fb1 [3252001,51863] 0 2024-04-26T04:52:59.776 INFO:tasks.workunit.client.1.smithi078.stdout:7/630: symlink d9/da/d11/d13/d1a/d52/d5a/d28/dab/lcd 0 2024-04-26T04:52:59.776 INFO:tasks.workunit.client.1.smithi078.stdout:6/727: dwrite d2/de/d27/da9/da/fb8 [0,4194304] 0 2024-04-26T04:52:59.777 INFO:tasks.workunit.client.0.smithi077.stdout:2/262: rename d9/d15/d1a/d2b/f42 to d9/dc/f5c 0 2024-04-26T04:52:59.779 INFO:tasks.workunit.client.1.smithi078.stdout:6/728: stat d2/de/d27/da9/da/la4 0 2024-04-26T04:52:59.779 INFO:tasks.workunit.client.1.smithi078.stdout:3/677: readlink d16/l64 0 2024-04-26T04:52:59.779 INFO:tasks.workunit.client.0.smithi077.stdout:5/343: link d1/d12/d21/d25/d31/d3c/c4f d1/d12/d28/c80 0 2024-04-26T04:52:59.780 INFO:tasks.workunit.client.1.smithi078.stdout:7/631: chown d9/da/d33/d4b/d4c/d6c/fb6 44521 1 2024-04-26T04:52:59.780 INFO:tasks.workunit.client.1.smithi078.stdout:6/729: chown d2/d1a 95 1 2024-04-26T04:52:59.781 INFO:tasks.workunit.client.0.smithi077.stdout:9/289: fsync d9/dd/d1b/d2b/f38 0 2024-04-26T04:52:59.781 INFO:tasks.workunit.client.1.smithi078.stdout:4/687: write d0/de/f91 [5096286,35567] 0 2024-04-26T04:52:59.782 INFO:tasks.workunit.client.0.smithi077.stdout:2/263: dread d9/dc/d1d/f4a [0,4194304] 0 2024-04-26T04:52:59.783 INFO:tasks.workunit.client.1.smithi078.stdout:3/678: write d16/d26/d2b/d66/fc2 [1347917,33642] 0 2024-04-26T04:52:59.783 INFO:tasks.workunit.client.0.smithi077.stdout:2/264: readlink d9/dc/d1d/d1e/l18 0 2024-04-26T04:52:59.784 INFO:tasks.workunit.client.1.smithi078.stdout:1/704: dwrite d7/fbf [0,4194304] 0 2024-04-26T04:52:59.785 INFO:tasks.workunit.client.0.smithi077.stdout:6/347: symlink d8/l7e 0 2024-04-26T04:52:59.785 INFO:tasks.workunit.client.0.smithi077.stdout:6/348: read - d8/f6d zero size 2024-04-26T04:52:59.787 INFO:tasks.workunit.client.1.smithi078.stdout:3/679: truncate d16/d26/d2e/d5f/d49/f4e 2056598 0 2024-04-26T04:52:59.788 INFO:tasks.workunit.client.1.smithi078.stdout:2/637: creat d8/d11/d14/d69/d8f/fe3 x:0 0 0 2024-04-26T04:52:59.790 INFO:tasks.workunit.client.1.smithi078.stdout:9/594: symlink dc/df/d37/lab 0 2024-04-26T04:52:59.791 INFO:tasks.workunit.client.0.smithi077.stdout:9/290: write d9/dd/d1b/f23 [4637546,15802] 0 2024-04-26T04:52:59.792 INFO:tasks.workunit.client.1.smithi078.stdout:9/595: write dc/df/d4b/d22/d2a/d57/fa4 [1751984,72895] 0 2024-04-26T04:52:59.796 INFO:tasks.workunit.client.0.smithi077.stdout:6/349: chown d8/ca 9 1 2024-04-26T04:52:59.798 INFO:tasks.workunit.client.0.smithi077.stdout:9/291: dwrite d9/f22 [0,4194304] 0 2024-04-26T04:52:59.800 INFO:tasks.workunit.client.0.smithi077.stdout:5/344: mknod d1/d12/c81 0 2024-04-26T04:52:59.803 INFO:tasks.workunit.client.1.smithi078.stdout:7/632: mknod d9/da/d33/d4b/cce 0 2024-04-26T04:52:59.804 INFO:tasks.workunit.client.0.smithi077.stdout:2/265: creat d9/d15/d44/d4e/f5d x:0 0 0 2024-04-26T04:52:59.806 INFO:tasks.workunit.client.0.smithi077.stdout:6/350: symlink d8/d52/d53/l7f 0 2024-04-26T04:52:59.807 INFO:tasks.workunit.client.1.smithi078.stdout:6/730: truncate d2/de/d27/f55 614633 0 2024-04-26T04:52:59.807 INFO:tasks.workunit.client.1.smithi078.stdout:8/609: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:52:59.811 INFO:tasks.workunit.client.0.smithi077.stdout:2/266: creat d9/dc/d1d/d1e/f5e x:0 0 0 2024-04-26T04:52:59.812 INFO:tasks.workunit.client.1.smithi078.stdout:0/672: getdents d2/d9/d18/d7d/da9/dab/dc6 0 2024-04-26T04:52:59.813 INFO:tasks.workunit.client.0.smithi077.stdout:2/267: write d9/d15/d1a/d2b/f53 [301629,66202] 0 2024-04-26T04:52:59.814 INFO:tasks.workunit.client.0.smithi077.stdout:2/268: chown d9/dc/d1d/d1e/f21 30959 1 2024-04-26T04:52:59.814 INFO:tasks.workunit.client.0.smithi077.stdout:6/351: creat d8/d52/d53/f80 x:0 0 0 2024-04-26T04:52:59.816 INFO:tasks.workunit.client.1.smithi078.stdout:3/680: creat d16/d26/d2b/d54/d56/da8/ff2 x:0 0 0 2024-04-26T04:52:59.819 INFO:tasks.workunit.client.0.smithi077.stdout:5/345: symlink d1/d12/d21/d25/d31/l82 0 2024-04-26T04:52:59.821 INFO:tasks.workunit.client.1.smithi078.stdout:9/596: fsync dc/f26 0 2024-04-26T04:52:59.822 INFO:tasks.workunit.client.1.smithi078.stdout:1/705: write d7/d1e/d9e/db/d49/f61 [4111941,129321] 0 2024-04-26T04:52:59.824 INFO:tasks.workunit.client.0.smithi077.stdout:9/292: creat d9/dd/d28/f68 x:0 0 0 2024-04-26T04:52:59.826 INFO:tasks.workunit.client.1.smithi078.stdout:7/633: creat d9/da/d33/d4b/d4c/d6c/fcf x:0 0 0 2024-04-26T04:52:59.826 INFO:tasks.workunit.client.0.smithi077.stdout:5/346: mknod d1/d30/c83 0 2024-04-26T04:52:59.829 INFO:tasks.workunit.client.0.smithi077.stdout:2/269: mkdir d9/dc/d5f 0 2024-04-26T04:52:59.829 INFO:tasks.workunit.client.1.smithi078.stdout:6/731: read d2/de/f2b [3386388,38037] 0 2024-04-26T04:52:59.829 INFO:tasks.workunit.client.1.smithi078.stdout:1/706: dwrite d7/d1e/d9e/db/f41 [4194304,4194304] 0 2024-04-26T04:52:59.830 INFO:tasks.workunit.client.0.smithi077.stdout:2/270: chown f7 0 1 2024-04-26T04:52:59.830 INFO:tasks.workunit.client.1.smithi078.stdout:8/610: mkdir d0/d19/d52/d35/d44/d6b/dc3 0 2024-04-26T04:52:59.831 INFO:tasks.workunit.client.0.smithi077.stdout:2/271: read d9/dc/d1d/d1e/f21 [2036338,55043] 0 2024-04-26T04:52:59.833 INFO:tasks.workunit.client.1.smithi078.stdout:2/638: dwrite d8/d11/d18/d5d/d61/f3c [0,4194304] 0 2024-04-26T04:52:59.833 INFO:tasks.workunit.client.0.smithi077.stdout:5/347: dwrite d1/d12/f1f [0,4194304] 0 2024-04-26T04:52:59.835 INFO:tasks.workunit.client.0.smithi077.stdout:9/293: rename d9/f2e to d9/d35/f69 0 2024-04-26T04:52:59.836 INFO:tasks.workunit.client.1.smithi078.stdout:0/673: rmdir d2/d9/d78 39 2024-04-26T04:52:59.839 INFO:tasks.workunit.client.1.smithi078.stdout:5/600: getdents d7/d48/d57/d65/d7a 0 2024-04-26T04:52:59.843 INFO:tasks.workunit.client.0.smithi077.stdout:6/352: getdents d8/d55 0 2024-04-26T04:52:59.844 INFO:tasks.workunit.client.0.smithi077.stdout:6/353: read d8/d1c/f25 [378883,35200] 0 2024-04-26T04:52:59.845 INFO:tasks.workunit.client.0.smithi077.stdout:6/354: chown d8/d1c/f5c 30452147 1 2024-04-26T04:52:59.847 INFO:tasks.workunit.client.1.smithi078.stdout:5/601: dwrite d7/d48/fbd [0,4194304] 0 2024-04-26T04:52:59.847 INFO:tasks.workunit.client.0.smithi077.stdout:2/272: rename f7 to d9/dc/d5f/f60 0 2024-04-26T04:52:59.854 INFO:tasks.workunit.client.1.smithi078.stdout:4/688: link d0/de/d22/d27/d56/fe3 d0/de/d22/d27/ffa 0 2024-04-26T04:52:59.856 INFO:tasks.workunit.client.1.smithi078.stdout:1/707: creat d7/d1e/d9e/db/d37/fe6 x:0 0 0 2024-04-26T04:52:59.859 INFO:tasks.workunit.client.1.smithi078.stdout:8/611: fdatasync d0/d19/d52/d35/d44/f26 0 2024-04-26T04:52:59.860 INFO:tasks.workunit.client.1.smithi078.stdout:8/612: fsync d0/d19/fa1 0 2024-04-26T04:52:59.860 INFO:tasks.workunit.client.0.smithi077.stdout:6/355: dread d8/f27 [0,4194304] 0 2024-04-26T04:52:59.862 INFO:tasks.workunit.client.1.smithi078.stdout:1/708: dwrite d7/f8e [0,4194304] 0 2024-04-26T04:52:59.863 INFO:tasks.workunit.client.1.smithi078.stdout:9/597: link dc/df/f1d dc/df/d27/d81/da8/fac 0 2024-04-26T04:52:59.865 INFO:tasks.workunit.client.0.smithi077.stdout:2/273: creat d9/dc/d1d/d34/f61 x:0 0 0 2024-04-26T04:52:59.866 INFO:tasks.workunit.client.1.smithi078.stdout:1/709: dread d7/d1e/d9e/db/d49/f6c [0,4194304] 0 2024-04-26T04:52:59.866 INFO:tasks.workunit.client.1.smithi078.stdout:7/634: link d9/f14 d9/da/d11/d13/d50/d8a/d8f/fd0 0 2024-04-26T04:52:59.867 INFO:tasks.workunit.client.1.smithi078.stdout:1/710: fdatasync d7/d1e/d9e/db/d49/fb4 0 2024-04-26T04:52:59.868 INFO:tasks.workunit.client.1.smithi078.stdout:7/635: read d9/da/f15 [9862546,78762] 0 2024-04-26T04:52:59.868 INFO:tasks.workunit.client.1.smithi078.stdout:6/732: dread d2/de/d27/da9/da/d36/db9/f47 [0,4194304] 0 2024-04-26T04:52:59.869 INFO:tasks.workunit.client.1.smithi078.stdout:7/636: dread - d9/f58 zero size 2024-04-26T04:52:59.872 INFO:tasks.workunit.client.0.smithi077.stdout:6/356: truncate d8/d2a/f2e 5051513 0 2024-04-26T04:52:59.875 INFO:tasks.workunit.client.1.smithi078.stdout:0/674: dread d2/d9/d18/d7d/da9/dab/dc6/f79 [0,4194304] 0 2024-04-26T04:52:59.876 INFO:tasks.workunit.client.0.smithi077.stdout:2/274: read d9/dc/d1d/f4a [370409,123569] 0 2024-04-26T04:52:59.878 INFO:tasks.workunit.client.0.smithi077.stdout:5/348: getdents d1/d30/d33/d66 0 2024-04-26T04:52:59.879 INFO:tasks.workunit.client.0.smithi077.stdout:2/275: dread d9/dc/d1d/f4a [0,4194304] 0 2024-04-26T04:52:59.881 INFO:tasks.workunit.client.0.smithi077.stdout:9/294: dwrite d9/f11 [4194304,4194304] 0 2024-04-26T04:52:59.883 INFO:tasks.workunit.client.0.smithi077.stdout:6/357: rmdir d8 39 2024-04-26T04:52:59.887 INFO:tasks.workunit.client.1.smithi078.stdout:8/613: fdatasync d0/d19/d52/d35/d44/f26 0 2024-04-26T04:52:59.888 INFO:tasks.workunit.client.1.smithi078.stdout:9/598: fsync dc/df/f59 0 2024-04-26T04:52:59.890 INFO:tasks.workunit.client.0.smithi077.stdout:5/349: mkdir d1/d12/d21/d84 0 2024-04-26T04:52:59.892 INFO:tasks.workunit.client.0.smithi077.stdout:5/350: dread d1/d8/f49 [0,4194304] 0 2024-04-26T04:52:59.893 INFO:tasks.workunit.client.0.smithi077.stdout:2/276: rename d9/dc/d1d/f25 to d9/dc/d5f/f62 0 2024-04-26T04:52:59.895 INFO:tasks.workunit.client.1.smithi078.stdout:5/602: unlink d7/cc 0 2024-04-26T04:52:59.899 INFO:tasks.workunit.client.1.smithi078.stdout:6/733: symlink d2/de/le4 0 2024-04-26T04:52:59.901 INFO:tasks.workunit.client.1.smithi078.stdout:4/689: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/d55/f5d [0,4194304] 0 2024-04-26T04:52:59.901 INFO:tasks.workunit.client.0.smithi077.stdout:2/277: rename d9/dc/d1d/d45 to d9/dc/d1d/d34/d39/d63 0 2024-04-26T04:52:59.903 INFO:tasks.workunit.client.0.smithi077.stdout:9/295: creat d9/dd/d1b/d4e/f6a x:0 0 0 2024-04-26T04:52:59.905 INFO:tasks.workunit.client.1.smithi078.stdout:2/639: dwrite d8/d11/d14/d69/d8f/f6b [0,4194304] 0 2024-04-26T04:52:59.910 INFO:tasks.workunit.client.1.smithi078.stdout:8/614: creat d0/d19/d25/d32/d8a/fc4 x:0 0 0 2024-04-26T04:52:59.911 INFO:tasks.workunit.client.0.smithi077.stdout:2/278: dwrite d9/dc/d1d/f47 [0,4194304] 0 2024-04-26T04:52:59.912 INFO:tasks.workunit.client.0.smithi077.stdout:9/296: dread d9/dd/f5d [0,4194304] 0 2024-04-26T04:52:59.914 INFO:tasks.workunit.client.0.smithi077.stdout:9/297: readlink d9/dd/le 0 2024-04-26T04:52:59.916 INFO:tasks.workunit.client.0.smithi077.stdout:9/298: write d9/d45/f4a [1044731,53163] 0 2024-04-26T04:52:59.916 INFO:tasks.workunit.client.0.smithi077.stdout:5/351: dread d1/d8/f22 [0,4194304] 0 2024-04-26T04:52:59.917 INFO:tasks.workunit.client.1.smithi078.stdout:7/637: dwrite d9/da/d11/f3a [0,4194304] 0 2024-04-26T04:52:59.918 INFO:tasks.workunit.client.0.smithi077.stdout:5/352: write d1/d30/d33/f76 [95619,7363] 0 2024-04-26T04:52:59.919 INFO:tasks.workunit.client.0.smithi077.stdout:5/353: stat d1/d12/d28/c80 0 2024-04-26T04:52:59.924 INFO:tasks.workunit.client.0.smithi077.stdout:5/354: rmdir d1/d30/d33 39 2024-04-26T04:52:59.926 INFO:tasks.workunit.client.1.smithi078.stdout:1/711: chown d7/d1e/d9e/db/d49/dba/d87/fbb 201 1 2024-04-26T04:52:59.928 INFO:tasks.workunit.client.0.smithi077.stdout:2/279: creat d9/f64 x:0 0 0 2024-04-26T04:52:59.928 INFO:tasks.workunit.client.1.smithi078.stdout:4/690: rmdir d0 39 2024-04-26T04:52:59.930 INFO:tasks.workunit.client.0.smithi077.stdout:2/280: symlink d9/dc/d5f/l65 0 2024-04-26T04:52:59.930 INFO:tasks.workunit.client.1.smithi078.stdout:3/681: sync 2024-04-26T04:52:59.931 INFO:tasks.workunit.client.1.smithi078.stdout:0/675: link d2/f57 d2/d9/d31/fe0 0 2024-04-26T04:52:59.936 INFO:tasks.workunit.client.0.smithi077.stdout:2/281: creat d9/f66 x:0 0 0 2024-04-26T04:52:59.937 INFO:tasks.workunit.client.1.smithi078.stdout:7/638: mknod d9/da/d33/d4b/d4c/cd1 0 2024-04-26T04:52:59.938 INFO:tasks.workunit.client.1.smithi078.stdout:6/734: mkdir d2/de/d27/da9/de5 0 2024-04-26T04:52:59.938 INFO:tasks.workunit.client.0.smithi077.stdout:2/282: creat d9/dc/d1d/d34/d39/f67 x:0 0 0 2024-04-26T04:52:59.940 INFO:tasks.workunit.client.1.smithi078.stdout:3/682: unlink d16/d26/d2b/d54/d56/f80 0 2024-04-26T04:52:59.942 INFO:tasks.workunit.client.1.smithi078.stdout:8/615: unlink d0/d19/d52/l51 0 2024-04-26T04:52:59.943 INFO:tasks.workunit.client.0.smithi077.stdout:2/283: fsync d9/d15/d1a/f54 0 2024-04-26T04:52:59.949 INFO:tasks.workunit.client.1.smithi078.stdout:3/683: creat d16/d26/d2e/d5f/d49/d77/ff3 x:0 0 0 2024-04-26T04:52:59.950 INFO:tasks.workunit.client.1.smithi078.stdout:8/616: creat d0/d19/d52/d95/d9d/fc5 x:0 0 0 2024-04-26T04:52:59.951 INFO:tasks.workunit.client.0.smithi077.stdout:2/284: dread d9/f10 [0,4194304] 0 2024-04-26T04:52:59.956 INFO:tasks.workunit.client.0.smithi077.stdout:2/285: dwrite d9/dc/d1d/d34/f36 [0,4194304] 0 2024-04-26T04:52:59.957 INFO:tasks.workunit.client.1.smithi078.stdout:4/691: rename d0/fc to d0/ffb 0 2024-04-26T04:52:59.957 INFO:tasks.workunit.client.0.smithi077.stdout:2/286: chown d9/dc/d1d/d34/d39/f67 439062 1 2024-04-26T04:52:59.960 INFO:tasks.workunit.client.1.smithi078.stdout:8/617: link d0/d19/d52/faa d0/d19/d25/d32/d8a/d92/da4/fc6 0 2024-04-26T04:52:59.961 INFO:tasks.workunit.client.1.smithi078.stdout:8/618: write d0/d19/f2a [2766637,43900] 0 2024-04-26T04:52:59.965 INFO:tasks.workunit.client.1.smithi078.stdout:3/684: link d16/d26/d2e/ld2 d16/d26/d2b/d31/d71/d8a/lf4 0 2024-04-26T04:52:59.966 INFO:tasks.workunit.client.1.smithi078.stdout:8/619: creat d0/d19/d52/d95/fc7 x:0 0 0 2024-04-26T04:52:59.966 INFO:tasks.workunit.client.1.smithi078.stdout:4/692: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/df6/lfc 0 2024-04-26T04:52:59.966 INFO:tasks.workunit.client.1.smithi078.stdout:8/620: fsync d0/f23 0 2024-04-26T04:52:59.968 INFO:tasks.workunit.client.1.smithi078.stdout:8/621: chown d0/d19/d25/d32/d8a/f5e 38398 1 2024-04-26T04:52:59.969 INFO:tasks.workunit.client.0.smithi077.stdout:2/287: rename d9/d15/f2f to d9/dc/d1d/d34/d39/f68 0 2024-04-26T04:52:59.969 INFO:tasks.workunit.client.0.smithi077.stdout:2/288: dread - d9/dc/d1d/d34/f50 zero size 2024-04-26T04:52:59.971 INFO:tasks.workunit.client.0.smithi077.stdout:2/289: unlink d9/dc/d1d/d34/d39/f67 0 2024-04-26T04:52:59.972 INFO:tasks.workunit.client.1.smithi078.stdout:4/693: mknod d0/d1d/de1/cfd 0 2024-04-26T04:52:59.973 INFO:tasks.workunit.client.0.smithi077.stdout:2/290: write d9/dc/d1d/d34/f61 [856296,125406] 0 2024-04-26T04:52:59.973 INFO:tasks.workunit.client.1.smithi078.stdout:4/694: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa 2346693 1 2024-04-26T04:52:59.975 INFO:tasks.workunit.client.0.smithi077.stdout:2/291: creat d9/dc/d1d/d34/f69 x:0 0 0 2024-04-26T04:52:59.976 INFO:tasks.workunit.client.1.smithi078.stdout:4/695: unlink d0/de/d22/d27/lbf 0 2024-04-26T04:52:59.976 INFO:tasks.workunit.client.1.smithi078.stdout:8/622: link d0/d19/d52/d35/f8f d0/d19/d52/d35/d44/fc8 0 2024-04-26T04:52:59.977 INFO:tasks.workunit.client.0.smithi077.stdout:2/292: dread d9/f10 [0,4194304] 0 2024-04-26T04:52:59.978 INFO:tasks.workunit.client.1.smithi078.stdout:8/623: mknod d0/d19/d25/d32/d8a/d92/da4/cc9 0 2024-04-26T04:52:59.981 INFO:tasks.workunit.client.0.smithi077.stdout:2/293: read d9/ff [2525123,8514] 0 2024-04-26T04:52:59.983 INFO:tasks.workunit.client.0.smithi077.stdout:2/294: mkdir d9/dc/d1d/d34/d39/d63/d6a 0 2024-04-26T04:52:59.984 INFO:tasks.workunit.client.1.smithi078.stdout:8/624: dwrite d0/d19/d25/d32/d8a/db1/f88 [0,4194304] 0 2024-04-26T04:52:59.986 INFO:tasks.workunit.client.1.smithi078.stdout:3/685: dread d16/f3f [4194304,4194304] 0 2024-04-26T04:52:59.987 INFO:tasks.workunit.client.1.smithi078.stdout:8/625: unlink d0/d19/d52/f5a 0 2024-04-26T04:52:59.988 INFO:tasks.workunit.client.0.smithi077.stdout:2/295: dwrite d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:52:59.989 INFO:tasks.workunit.client.0.smithi077.stdout:2/296: dread - d9/f3d zero size 2024-04-26T04:52:59.991 INFO:tasks.workunit.client.0.smithi077.stdout:2/297: readlink d9/dc/d1d/d34/d39/d63/l48 0 2024-04-26T04:52:59.991 INFO:tasks.workunit.client.1.smithi078.stdout:3/686: rename d16/d26/d2b/d54/d56/da8/ff2 to d16/d26/d2e/d5f/ff5 0 2024-04-26T04:52:59.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:52:59 smithi077 ceph-mon[26311]: pgmap v19: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 277 MiB/s wr, 467 op/s 2024-04-26T04:52:59.993 INFO:tasks.workunit.client.1.smithi078.stdout:8/626: dwrite d0/fae [0,4194304] 0 2024-04-26T04:52:59.994 INFO:tasks.workunit.client.0.smithi077.stdout:2/298: mkdir d9/dc/d1d/d34/d39/d6b 0 2024-04-26T04:52:59.997 INFO:tasks.workunit.client.1.smithi078.stdout:8/627: symlink d0/lca 0 2024-04-26T04:52:59.998 INFO:tasks.workunit.client.1.smithi078.stdout:3/687: truncate d16/d26/d2e/d5f/d49/fb4 3386446 0 2024-04-26T04:53:00.001 INFO:tasks.workunit.client.0.smithi077.stdout:2/299: getdents d9/d15/d1a/d2b 0 2024-04-26T04:53:00.009 INFO:tasks.workunit.client.1.smithi078.stdout:8/628: link d0/d19/d52/d35/d44/d6b/d6c/c97 d0/d19/d25/d32/d8a/ccb 0 2024-04-26T04:53:00.016 INFO:tasks.workunit.client.1.smithi078.stdout:5/603: creat d7/d11/fc9 x:0 0 0 2024-04-26T04:53:00.016 INFO:tasks.workunit.client.1.smithi078.stdout:8/629: dwrite d0/f10 [4194304,4194304] 0 2024-04-26T04:53:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:52:59 smithi078 ceph-mon[33746]: pgmap v19: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 63 MiB/s rd, 277 MiB/s wr, 467 op/s 2024-04-26T04:53:00.017 INFO:tasks.workunit.client.0.smithi077.stdout:5/355: dwrite d1/d8/ff [0,4194304] 0 2024-04-26T04:53:00.017 INFO:tasks.workunit.client.1.smithi078.stdout:5/604: chown d7/da/d1e/d91/c9a 36020 1 2024-04-26T04:53:00.018 INFO:tasks.workunit.client.0.smithi077.stdout:5/356: fsync d1/f5 0 2024-04-26T04:53:00.018 INFO:tasks.workunit.client.0.smithi077.stdout:5/357: chown d1/d12/c4d 7529053 1 2024-04-26T04:53:00.019 INFO:tasks.workunit.client.0.smithi077.stdout:5/358: readlink d1/d12/d21/d25/d31/l82 0 2024-04-26T04:53:00.021 INFO:tasks.workunit.client.1.smithi078.stdout:9/599: truncate dc/df/d4b/d22/d2a/d57/fa4 698678 0 2024-04-26T04:53:00.026 INFO:tasks.workunit.client.1.smithi078.stdout:8/630: unlink d0/d19/c64 0 2024-04-26T04:53:00.030 INFO:tasks.workunit.client.1.smithi078.stdout:2/640: truncate d8/d3e/f6a 2763967 0 2024-04-26T04:53:00.033 INFO:tasks.workunit.client.0.smithi077.stdout:5/359: link d1/d12/d45/f63 d1/d42/d62/d6b/f85 0 2024-04-26T04:53:00.034 INFO:tasks.workunit.client.1.smithi078.stdout:6/735: write d2/de/d27/da9/d2a/d4d/f78 [309699,127483] 0 2024-04-26T04:53:00.035 INFO:tasks.workunit.client.1.smithi078.stdout:5/605: dwrite d7/d18/d1c/f9e [0,4194304] 0 2024-04-26T04:53:00.036 INFO:tasks.workunit.client.1.smithi078.stdout:9/600: link dc/df/d4b/d22/d2a/d92/fa3 dc/df/d4b/d22/d2a/d92/fad 0 2024-04-26T04:53:00.037 INFO:tasks.workunit.client.1.smithi078.stdout:1/712: truncate d7/d1e/d9e/dd/d20/f40 317990 0 2024-04-26T04:53:00.038 INFO:tasks.workunit.client.1.smithi078.stdout:5/606: truncate d7/d18/d1c/f92 171061 0 2024-04-26T04:53:00.040 INFO:tasks.workunit.client.0.smithi077.stdout:5/360: mknod d1/d12/d21/d84/c86 0 2024-04-26T04:53:00.041 INFO:tasks.workunit.client.0.smithi077.stdout:5/361: chown d1/d8/l2f 5564 1 2024-04-26T04:53:00.047 INFO:tasks.workunit.client.1.smithi078.stdout:5/607: fsync d7/d48/fa3 0 2024-04-26T04:53:00.048 INFO:tasks.workunit.client.0.smithi077.stdout:5/362: dread d1/d8/f70 [0,4194304] 0 2024-04-26T04:53:00.048 INFO:tasks.workunit.client.1.smithi078.stdout:6/736: symlink d2/de/d27/da9/da8/ddc/le6 0 2024-04-26T04:53:00.050 INFO:tasks.workunit.client.1.smithi078.stdout:1/713: symlink d7/d1e/d9e/d4c/d3d/le7 0 2024-04-26T04:53:00.052 INFO:tasks.workunit.client.1.smithi078.stdout:9/601: dwrite dc/df/d4b/d3e/d80/f95 [0,4194304] 0 2024-04-26T04:53:00.052 INFO:tasks.workunit.client.1.smithi078.stdout:5/608: mkdir d7/d48/d57/d65/d70/dca 0 2024-04-26T04:53:00.068 INFO:tasks.workunit.client.1.smithi078.stdout:0/676: sync 2024-04-26T04:53:00.068 INFO:tasks.workunit.client.1.smithi078.stdout:7/639: sync 2024-04-26T04:53:00.076 INFO:tasks.workunit.client.0.smithi077.stdout:2/300: dwrite d9/f2c [0,4194304] 0 2024-04-26T04:53:00.080 INFO:tasks.workunit.client.1.smithi078.stdout:5/609: rmdir d7/d18/d32/d37/dc7 0 2024-04-26T04:53:00.083 INFO:tasks.workunit.client.1.smithi078.stdout:2/641: rename d8/d3e to d8/d11/d14/d69/d8f/d34/dd3/de4 0 2024-04-26T04:53:00.084 INFO:tasks.workunit.client.1.smithi078.stdout:2/642: stat d8/l10 0 2024-04-26T04:53:00.086 INFO:tasks.workunit.client.1.smithi078.stdout:0/677: dread d2/f48 [0,4194304] 0 2024-04-26T04:53:00.087 INFO:tasks.workunit.client.1.smithi078.stdout:5/610: symlink d7/d48/d57/d65/d7a/d9c/lcb 0 2024-04-26T04:53:00.088 INFO:tasks.workunit.client.0.smithi077.stdout:8/287: sync 2024-04-26T04:53:00.089 INFO:tasks.workunit.client.1.smithi078.stdout:3/688: dwrite d16/d26/d2b/d54/d56/fa2 [0,4194304] 0 2024-04-26T04:53:00.089 INFO:tasks.workunit.client.1.smithi078.stdout:8/631: truncate d0/d19/d25/d32/d8a/f57 2096133 0 2024-04-26T04:53:00.091 INFO:tasks.workunit.client.1.smithi078.stdout:0/678: write d2/d9/d18/f4e [515370,90786] 0 2024-04-26T04:53:00.093 INFO:tasks.workunit.client.0.smithi077.stdout:8/288: mknod d4/df/d18/d56/c65 0 2024-04-26T04:53:00.093 INFO:tasks.workunit.client.0.smithi077.stdout:5/363: write d1/d8/f49 [449918,47980] 0 2024-04-26T04:53:00.094 INFO:tasks.workunit.client.0.smithi077.stdout:8/289: chown d4/d17/d35/d5b 294 1 2024-04-26T04:53:00.096 INFO:tasks.workunit.client.1.smithi078.stdout:3/689: unlink d16/d26/d2b/d31/f55 0 2024-04-26T04:53:00.098 INFO:tasks.workunit.client.1.smithi078.stdout:3/690: write d16/d26/d2e/d5f/fcf [861335,112983] 0 2024-04-26T04:53:00.098 INFO:tasks.workunit.client.1.smithi078.stdout:0/679: symlink d2/d9/d31/d4a/le1 0 2024-04-26T04:53:00.101 INFO:tasks.workunit.client.1.smithi078.stdout:5/611: creat d7/d18/d32/d37/d8f/dc5/fcc x:0 0 0 2024-04-26T04:53:00.101 INFO:tasks.workunit.client.1.smithi078.stdout:9/602: truncate dc/df/f2f 2968368 0 2024-04-26T04:53:00.101 INFO:tasks.workunit.client.1.smithi078.stdout:5/612: chown d7/da/d8a 250517 1 2024-04-26T04:53:00.102 INFO:tasks.workunit.client.0.smithi077.stdout:5/364: mkdir d1/d12/d87 0 2024-04-26T04:53:00.103 INFO:tasks.workunit.client.1.smithi078.stdout:1/714: dwrite d7/d1e/d9e/db/f5a [0,4194304] 0 2024-04-26T04:53:00.104 INFO:tasks.workunit.client.0.smithi077.stdout:8/290: symlink d4/df/d22/l66 0 2024-04-26T04:53:00.106 INFO:tasks.workunit.client.0.smithi077.stdout:2/301: write d9/ff [2528543,51522] 0 2024-04-26T04:53:00.108 INFO:tasks.workunit.client.0.smithi077.stdout:2/302: stat d9/d15/d1a/f22 0 2024-04-26T04:53:00.108 INFO:tasks.workunit.client.1.smithi078.stdout:9/603: dwrite dc/df/d4b/d22/d2a/d92/f71 [0,4194304] 0 2024-04-26T04:53:00.109 INFO:tasks.workunit.client.1.smithi078.stdout:0/680: mknod d2/d9/d18/ce2 0 2024-04-26T04:53:00.109 INFO:tasks.workunit.client.0.smithi077.stdout:5/365: dwrite d1/d12/d28/f2e [0,4194304] 0 2024-04-26T04:53:00.110 INFO:tasks.workunit.client.1.smithi078.stdout:9/604: chown dc/df/d4b/d58 481074 1 2024-04-26T04:53:00.110 INFO:tasks.workunit.client.1.smithi078.stdout:7/640: dread d9/f10 [0,4194304] 0 2024-04-26T04:53:00.111 INFO:tasks.workunit.client.1.smithi078.stdout:9/605: write dc/df/d4b/d22/d2a/d92/fa2 [597419,84025] 0 2024-04-26T04:53:00.112 INFO:tasks.workunit.client.1.smithi078.stdout:9/606: chown dc/df/d4b/d22/d2a/d92/fa3 87965596 1 2024-04-26T04:53:00.114 INFO:tasks.workunit.client.1.smithi078.stdout:9/607: write dc/df/d27/d81/f87 [515607,28477] 0 2024-04-26T04:53:00.122 INFO:tasks.workunit.client.0.smithi077.stdout:2/303: rmdir d9/dc/d1d/d34/d39/d63 39 2024-04-26T04:53:00.122 INFO:tasks.workunit.client.0.smithi077.stdout:5/366: dwrite d1/d8/f49 [0,4194304] 0 2024-04-26T04:53:00.123 INFO:tasks.workunit.client.1.smithi078.stdout:8/632: rename d0/d19/d25/d32/d8a/d92/cc0 to d0/d19/d25/ccc 0 2024-04-26T04:53:00.123 INFO:tasks.workunit.client.1.smithi078.stdout:2/643: write d8/d11/f45 [272597,87358] 0 2024-04-26T04:53:00.125 INFO:tasks.workunit.client.1.smithi078.stdout:8/633: write d0/d19/d52/d3c/f60 [3486526,102239] 0 2024-04-26T04:53:00.128 INFO:tasks.workunit.client.1.smithi078.stdout:1/715: fdatasync d7/d2e/f33 0 2024-04-26T04:53:00.130 INFO:tasks.workunit.client.1.smithi078.stdout:3/691: fdatasync d16/d26/f32 0 2024-04-26T04:53:00.132 INFO:tasks.workunit.client.0.smithi077.stdout:2/304: creat d9/dc/d1d/d34/d39/d63/f6c x:0 0 0 2024-04-26T04:53:00.134 INFO:tasks.workunit.client.1.smithi078.stdout:4/696: sync 2024-04-26T04:53:00.135 INFO:tasks.workunit.client.1.smithi078.stdout:7/641: fdatasync d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc1 0 2024-04-26T04:53:00.139 INFO:tasks.workunit.client.1.smithi078.stdout:3/692: dwrite d16/d26/d2b/d54/f58 [0,4194304] 0 2024-04-26T04:53:00.139 INFO:tasks.workunit.client.0.smithi077.stdout:2/305: dwrite d9/fd [0,4194304] 0 2024-04-26T04:53:00.141 INFO:tasks.workunit.client.0.smithi077.stdout:2/306: readlink d9/dc/d1d/l57 0 2024-04-26T04:53:00.143 INFO:tasks.workunit.client.0.smithi077.stdout:8/291: write d4/f40 [4169173,122483] 0 2024-04-26T04:53:00.146 INFO:tasks.workunit.client.0.smithi077.stdout:8/292: truncate d4/df/d22/d2d/d2b/f54 121283 0 2024-04-26T04:53:00.147 INFO:tasks.workunit.client.1.smithi078.stdout:5/613: link d7/d48/d57/d65/d7a/fac d7/d18/d1c/d8e/fcd 0 2024-04-26T04:53:00.149 INFO:tasks.workunit.client.1.smithi078.stdout:2/644: unlink l4 0 2024-04-26T04:53:00.150 INFO:tasks.workunit.client.1.smithi078.stdout:2/645: write f3 [894964,85672] 0 2024-04-26T04:53:00.153 INFO:tasks.workunit.client.1.smithi078.stdout:2/646: dread d8/f9a [0,4194304] 0 2024-04-26T04:53:00.158 INFO:tasks.workunit.client.1.smithi078.stdout:1/716: rename d7/dd7 to d7/d1e/d9e/db/d37/d58/d5c/d68/de8 0 2024-04-26T04:53:00.159 INFO:tasks.workunit.client.1.smithi078.stdout:0/681: creat d2/d1e/d5a/da3/fe3 x:0 0 0 2024-04-26T04:53:00.161 INFO:tasks.workunit.client.1.smithi078.stdout:7/642: creat d9/da/d33/fd2 x:0 0 0 2024-04-26T04:53:00.161 INFO:tasks.workunit.client.0.smithi077.stdout:2/307: rmdir d9/dc/d1d/d34/d39 39 2024-04-26T04:53:00.163 INFO:tasks.workunit.client.1.smithi078.stdout:5/614: mkdir d7/d48/d57/d65/d46/dce 0 2024-04-26T04:53:00.164 INFO:tasks.workunit.client.0.smithi077.stdout:2/308: write d9/dc/d5f/f60 [830077,83558] 0 2024-04-26T04:53:00.164 INFO:tasks.workunit.client.0.smithi077.stdout:2/309: stat d9/c4c 0 2024-04-26T04:53:00.165 INFO:tasks.workunit.client.0.smithi077.stdout:2/310: read - d9/f66 zero size 2024-04-26T04:53:00.166 INFO:tasks.workunit.client.1.smithi078.stdout:2/647: unlink d8/d11/d18/f47 0 2024-04-26T04:53:00.169 INFO:tasks.workunit.client.0.smithi077.stdout:2/311: rmdir d9/dc/d1d 39 2024-04-26T04:53:00.171 INFO:tasks.workunit.client.1.smithi078.stdout:9/608: write dc/f39 [1155944,28921] 0 2024-04-26T04:53:00.171 INFO:tasks.workunit.client.0.smithi077.stdout:2/312: symlink d9/dc/d5f/l6d 0 2024-04-26T04:53:00.173 INFO:tasks.workunit.client.1.smithi078.stdout:7/643: dread d9/da/d33/f56 [0,4194304] 0 2024-04-26T04:53:00.175 INFO:tasks.workunit.client.0.smithi077.stdout:2/313: unlink d9/dc/d5f/f60 0 2024-04-26T04:53:00.176 INFO:tasks.workunit.client.1.smithi078.stdout:5/615: dread f3 [0,4194304] 0 2024-04-26T04:53:00.179 INFO:tasks.workunit.client.0.smithi077.stdout:2/314: write d9/f10 [520332,42845] 0 2024-04-26T04:53:00.182 INFO:tasks.workunit.client.0.smithi077.stdout:8/293: dwrite d4/df/d18/f4c [0,4194304] 0 2024-04-26T04:53:00.187 INFO:tasks.workunit.client.1.smithi078.stdout:0/682: mkdir d2/d9/d60/dae/de4 0 2024-04-26T04:53:00.187 INFO:tasks.workunit.client.0.smithi077.stdout:2/315: write d9/dc/d1d/f24 [3753701,80275] 0 2024-04-26T04:53:00.187 INFO:tasks.workunit.client.0.smithi077.stdout:8/294: dread - d4/df/d22/d2d/f3d zero size 2024-04-26T04:53:00.194 INFO:tasks.workunit.client.0.smithi077.stdout:2/316: dwrite d9/f10 [0,4194304] 0 2024-04-26T04:53:00.196 INFO:tasks.workunit.client.0.smithi077.stdout:8/295: creat d4/df/d22/d2d/d2b/d33/f67 x:0 0 0 2024-04-26T04:53:00.199 INFO:tasks.workunit.client.1.smithi078.stdout:5/616: rmdir d7/d48/d57/d65/d70/d72/d85 39 2024-04-26T04:53:00.202 INFO:tasks.workunit.client.0.smithi077.stdout:2/317: mkdir d9/dc/d6e 0 2024-04-26T04:53:00.202 INFO:tasks.workunit.client.1.smithi078.stdout:8/634: getdents d0/d7a 0 2024-04-26T04:53:00.202 INFO:tasks.workunit.client.0.smithi077.stdout:2/318: write d9/d15/d1a/f30 [896214,125807] 0 2024-04-26T04:53:00.203 INFO:tasks.workunit.client.1.smithi078.stdout:1/717: creat d7/d1e/d9e/db/fe9 x:0 0 0 2024-04-26T04:53:00.205 INFO:tasks.workunit.client.1.smithi078.stdout:6/737: sync 2024-04-26T04:53:00.206 INFO:tasks.workunit.client.1.smithi078.stdout:9/609: mkdir dc/df/d4b/dae 0 2024-04-26T04:53:00.206 INFO:tasks.workunit.client.1.smithi078.stdout:2/648: dwrite d8/d11/d14/d69/d8f/f5c [0,4194304] 0 2024-04-26T04:53:00.208 INFO:tasks.workunit.client.0.smithi077.stdout:8/296: rename d4/df/f25 to d4/df/d22/f68 0 2024-04-26T04:53:00.210 INFO:tasks.workunit.client.0.smithi077.stdout:2/319: symlink d9/d15/d1a/l6f 0 2024-04-26T04:53:00.210 INFO:tasks.workunit.client.1.smithi078.stdout:5/617: dwrite d7/d18/f34 [4194304,4194304] 0 2024-04-26T04:53:00.214 INFO:tasks.workunit.client.1.smithi078.stdout:7/644: dread d9/da/d11/d13/d1a/d52/d5a/f44 [0,4194304] 0 2024-04-26T04:53:00.215 INFO:tasks.workunit.client.0.smithi077.stdout:2/320: dread d9/dc/d1d/f47 [0,4194304] 0 2024-04-26T04:53:00.215 INFO:tasks.workunit.client.1.smithi078.stdout:7/645: dread - d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 zero size 2024-04-26T04:53:00.216 INFO:tasks.workunit.client.0.smithi077.stdout:8/297: truncate d4/f42 844929 0 2024-04-26T04:53:00.221 INFO:tasks.workunit.client.1.smithi078.stdout:6/738: creat d2/de/d27/da9/da/d36/fe7 x:0 0 0 2024-04-26T04:53:00.224 INFO:tasks.workunit.client.1.smithi078.stdout:6/739: truncate d2/de/d27/da9/da/d36/db9/d97/fd2 235191 0 2024-04-26T04:53:00.225 INFO:tasks.workunit.client.1.smithi078.stdout:9/610: symlink dc/df/d4b/d22/d2a/d57/d69/laf 0 2024-04-26T04:53:00.225 INFO:tasks.workunit.client.1.smithi078.stdout:6/740: fdatasync d2/de/fac 0 2024-04-26T04:53:00.226 INFO:tasks.workunit.client.1.smithi078.stdout:9/611: truncate dc/df/d27/d81/f9e 774781 0 2024-04-26T04:53:00.286 INFO:tasks.workunit.client.1.smithi078.stdout:0/683: mknod d2/d9/d18/d7d/ce5 0 2024-04-26T04:53:00.287 INFO:tasks.workunit.client.0.smithi077.stdout:8/298: mkdir d4/d69 0 2024-04-26T04:53:00.288 INFO:tasks.workunit.client.1.smithi078.stdout:2/649: unlink d8/d11/d14/d69/d8f/l38 0 2024-04-26T04:53:00.290 INFO:tasks.workunit.client.0.smithi077.stdout:2/321: mknod d9/dc/d1d/d34/d39/d63/d6a/c70 0 2024-04-26T04:53:00.291 INFO:tasks.workunit.client.0.smithi077.stdout:8/299: dread - d4/df/d22/d2d/f30 zero size 2024-04-26T04:53:00.292 INFO:tasks.workunit.client.0.smithi077.stdout:2/322: mknod d9/d15/c71 0 2024-04-26T04:53:00.294 INFO:tasks.workunit.client.1.smithi078.stdout:5/618: rename d7/f38 to d7/d48/d57/fcf 0 2024-04-26T04:53:00.300 INFO:tasks.workunit.client.1.smithi078.stdout:5/619: dwrite d7/da/f62 [0,4194304] 0 2024-04-26T04:53:00.303 INFO:tasks.workunit.client.0.smithi077.stdout:8/300: link d4/df/d22/d2d/f3d d4/d17/f6a 0 2024-04-26T04:53:00.306 INFO:tasks.workunit.client.1.smithi078.stdout:1/718: truncate d7/d1e/d9e/db/f15 145056 0 2024-04-26T04:53:00.309 INFO:tasks.workunit.client.0.smithi077.stdout:2/323: creat d9/f72 x:0 0 0 2024-04-26T04:53:00.310 INFO:tasks.workunit.client.0.smithi077.stdout:2/324: chown d9/d15/d1a/d5b 2010593343 1 2024-04-26T04:53:00.311 INFO:tasks.workunit.client.1.smithi078.stdout:3/693: sync 2024-04-26T04:53:00.311 INFO:tasks.workunit.client.1.smithi078.stdout:8/635: sync 2024-04-26T04:53:00.311 INFO:tasks.workunit.client.1.smithi078.stdout:4/697: sync 2024-04-26T04:53:00.313 INFO:tasks.workunit.client.1.smithi078.stdout:4/698: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/ff2 zero size 2024-04-26T04:53:00.313 INFO:tasks.workunit.client.0.smithi077.stdout:8/301: fsync d4/df/f32 0 2024-04-26T04:53:00.314 INFO:tasks.workunit.client.0.smithi077.stdout:8/302: read - d4/df/f46 zero size 2024-04-26T04:53:00.315 INFO:tasks.workunit.client.1.smithi078.stdout:5/620: creat d7/da/d8a/fd0 x:0 0 0 2024-04-26T04:53:00.316 INFO:tasks.workunit.client.1.smithi078.stdout:1/719: chown d7/f11 17848184 1 2024-04-26T04:53:00.316 INFO:tasks.workunit.client.1.smithi078.stdout:5/621: write d7/d18/d1c/f95 [962336,18910] 0 2024-04-26T04:53:00.316 INFO:tasks.workunit.client.1.smithi078.stdout:0/684: link d2/d9/f54 d2/d1e/fe6 0 2024-04-26T04:53:00.317 INFO:tasks.workunit.client.1.smithi078.stdout:5/622: chown d7/d48/d57/d65/d70 48 1 2024-04-26T04:53:00.317 INFO:tasks.workunit.client.0.smithi077.stdout:8/303: truncate d4/fe 4597937 0 2024-04-26T04:53:00.319 INFO:tasks.workunit.client.1.smithi078.stdout:9/612: getdents dc/df/d89 0 2024-04-26T04:53:00.321 INFO:tasks.workunit.client.0.smithi077.stdout:2/325: dread d9/dc/d1d/d34/d39/f68 [0,4194304] 0 2024-04-26T04:53:00.321 INFO:tasks.workunit.client.0.smithi077.stdout:2/326: dread - d9/f72 zero size 2024-04-26T04:53:00.322 INFO:tasks.workunit.client.1.smithi078.stdout:9/613: chown dc/df/d4b/d22/d2a/d57/d69/laf 4124055 1 2024-04-26T04:53:00.323 INFO:tasks.workunit.client.1.smithi078.stdout:0/685: dwrite d2/d1e/d5a/da3/fe3 [0,4194304] 0 2024-04-26T04:53:00.324 INFO:tasks.workunit.client.1.smithi078.stdout:6/741: rename d2/de/d27/da9/da/d36/db5 to d2/de/d27/de8 0 2024-04-26T04:53:00.326 INFO:tasks.workunit.client.1.smithi078.stdout:3/694: creat d16/d26/d2e/d5f/d49/dad/db0/ff6 x:0 0 0 2024-04-26T04:53:00.327 INFO:tasks.workunit.client.0.smithi077.stdout:8/304: dwrite d4/df/d22/f68 [0,4194304] 0 2024-04-26T04:53:00.329 INFO:tasks.workunit.client.1.smithi078.stdout:4/699: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/ffe x:0 0 0 2024-04-26T04:53:00.330 INFO:tasks.workunit.client.1.smithi078.stdout:4/700: dread - d0/d1d/d46/f77 zero size 2024-04-26T04:53:00.330 INFO:tasks.workunit.client.0.smithi077.stdout:2/327: dwrite d9/dc/d1d/d1e/f55 [0,4194304] 0 2024-04-26T04:53:00.331 INFO:tasks.workunit.client.1.smithi078.stdout:4/701: stat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/de0 0 2024-04-26T04:53:00.331 INFO:tasks.workunit.client.1.smithi078.stdout:1/720: unlink d7/d1e/d9e/d4c/lb6 0 2024-04-26T04:53:00.336 INFO:tasks.workunit.client.1.smithi078.stdout:2/650: getdents d8/d11/d14/d69/d8f/d34/dd3/de4 0 2024-04-26T04:53:00.336 INFO:tasks.workunit.client.0.smithi077.stdout:2/328: mknod d9/dc/d1d/d1e/c73 0 2024-04-26T04:53:00.342 INFO:tasks.workunit.client.1.smithi078.stdout:0/686: dread - d2/d9/d60/f68 zero size 2024-04-26T04:53:00.343 INFO:tasks.workunit.client.1.smithi078.stdout:0/687: chown d2/d1e/d5a 61424810 1 2024-04-26T04:53:00.343 INFO:tasks.workunit.client.0.smithi077.stdout:2/329: fsync d9/dc/d1d/d1e/f13 0 2024-04-26T04:53:00.345 INFO:tasks.workunit.client.1.smithi078.stdout:6/742: truncate d2/de/f2b 4597003 0 2024-04-26T04:53:00.346 INFO:tasks.workunit.client.0.smithi077.stdout:8/305: truncate d4/df/d22/d2d/f1c 3308656 0 2024-04-26T04:53:00.348 INFO:tasks.workunit.client.0.smithi077.stdout:2/330: dread - d9/dc/d5f/f62 zero size 2024-04-26T04:53:00.348 INFO:tasks.workunit.client.0.smithi077.stdout:8/306: read d4/df/d22/d2d/f1e [458501,91343] 0 2024-04-26T04:53:00.348 INFO:tasks.workunit.client.0.smithi077.stdout:2/331: chown d9/d15/d44 1 1 2024-04-26T04:53:00.349 INFO:tasks.workunit.client.0.smithi077.stdout:2/332: chown d9/d15/d44/d4e 3 1 2024-04-26T04:53:00.349 INFO:tasks.workunit.client.1.smithi078.stdout:1/721: symlink d7/d1e/d9e/db/d49/d93/lea 0 2024-04-26T04:53:00.351 INFO:tasks.workunit.client.0.smithi077.stdout:8/307: mknod d4/d17/d35/c6b 0 2024-04-26T04:53:00.354 INFO:tasks.workunit.client.1.smithi078.stdout:9/614: truncate dc/df/d4b/d22/d2a/f4a 164363 0 2024-04-26T04:53:00.354 INFO:tasks.workunit.client.1.smithi078.stdout:9/615: chown dc/df/d8a 859473 1 2024-04-26T04:53:00.355 INFO:tasks.workunit.client.1.smithi078.stdout:9/616: chown dc/df/d4b/d3e/d80/ca7 16 1 2024-04-26T04:53:00.356 INFO:tasks.workunit.client.0.smithi077.stdout:2/333: dwrite d9/dc/d1d/d34/f61 [0,4194304] 0 2024-04-26T04:53:00.356 INFO:tasks.workunit.client.1.smithi078.stdout:8/636: creat d0/d19/d52/d5c/fcd x:0 0 0 2024-04-26T04:53:00.358 INFO:tasks.workunit.client.1.smithi078.stdout:6/743: unlink d2/de/d27/da9/d3a/f62 0 2024-04-26T04:53:00.360 INFO:tasks.workunit.client.1.smithi078.stdout:9/617: rmdir dc/df/d4b/d3e/d80 39 2024-04-26T04:53:00.361 INFO:tasks.workunit.client.1.smithi078.stdout:8/637: mkdir d0/d19/d52/d35/d44/d6b/dce 0 2024-04-26T04:53:00.362 INFO:tasks.workunit.client.0.smithi077.stdout:8/308: getdents d4/df 0 2024-04-26T04:53:00.364 INFO:tasks.workunit.client.1.smithi078.stdout:3/695: rename d16/d9b/cd4 to d16/d3c/cf7 0 2024-04-26T04:53:00.365 INFO:tasks.workunit.client.1.smithi078.stdout:6/744: rmdir d2/de/d27/da9/d11 39 2024-04-26T04:53:00.365 INFO:tasks.workunit.client.0.smithi077.stdout:2/334: dwrite d9/dc/d1d/d34/f61 [0,4194304] 0 2024-04-26T04:53:00.369 INFO:tasks.workunit.client.1.smithi078.stdout:5/623: getdents d7/da/d1e 0 2024-04-26T04:53:00.370 INFO:tasks.workunit.client.1.smithi078.stdout:5/624: chown d7/d18/d1c/d5a/f9d 679228304 1 2024-04-26T04:53:00.370 INFO:tasks.workunit.client.0.smithi077.stdout:8/309: unlink d4/df/d22/c27 0 2024-04-26T04:53:00.373 INFO:tasks.workunit.client.0.smithi077.stdout:2/335: mknod d9/d15/d1a/d2b/c74 0 2024-04-26T04:53:00.375 INFO:tasks.workunit.client.1.smithi078.stdout:2/651: write d8/d11/d14/f8d [137233,116138] 0 2024-04-26T04:53:00.375 INFO:tasks.workunit.client.0.smithi077.stdout:8/310: symlink d4/df/l6c 0 2024-04-26T04:53:00.378 INFO:tasks.workunit.client.0.smithi077.stdout:8/311: dread d4/df/d18/f26 [0,4194304] 0 2024-04-26T04:53:00.379 INFO:tasks.workunit.client.0.smithi077.stdout:2/336: dread d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:53:00.379 INFO:tasks.workunit.client.0.smithi077.stdout:8/312: fsync d4/df/d22/d2d/d2b/d33/f67 0 2024-04-26T04:53:00.381 INFO:tasks.workunit.client.1.smithi078.stdout:4/702: rename d0/de/f1a to d0/de/d22/d27/db5/fff 0 2024-04-26T04:53:00.386 INFO:tasks.workunit.client.1.smithi078.stdout:5/625: dread - d7/f87 zero size 2024-04-26T04:53:00.386 INFO:tasks.workunit.client.1.smithi078.stdout:0/688: dwrite d2/d9/d60/d6a/daa/faf [0,4194304] 0 2024-04-26T04:53:00.393 INFO:tasks.workunit.client.0.smithi077.stdout:2/337: getdents d9/d15/d56 0 2024-04-26T04:53:00.396 INFO:tasks.workunit.client.1.smithi078.stdout:6/745: rename d2/de/d27/da9/d2a/dbf/d5f/ld8 to d2/le9 0 2024-04-26T04:53:00.398 INFO:tasks.workunit.client.0.smithi077.stdout:2/338: dread d9/dc/d1d/d1e/f55 [0,4194304] 0 2024-04-26T04:53:00.399 INFO:tasks.workunit.client.0.smithi077.stdout:8/313: symlink d4/df/d22/d2d/d2b/l6d 0 2024-04-26T04:53:00.399 INFO:tasks.workunit.client.1.smithi078.stdout:7/646: sync 2024-04-26T04:53:00.399 INFO:tasks.workunit.client.1.smithi078.stdout:9/618: truncate dc/df/d4b/f86 2589314 0 2024-04-26T04:53:00.401 INFO:tasks.workunit.client.0.smithi077.stdout:2/339: dread d9/fd [0,4194304] 0 2024-04-26T04:53:00.402 INFO:tasks.workunit.client.0.smithi077.stdout:2/340: dread - d9/f3d zero size 2024-04-26T04:53:00.403 INFO:tasks.workunit.client.0.smithi077.stdout:2/341: readlink d9/dc/d1d/d1e/l18 0 2024-04-26T04:53:00.404 INFO:tasks.workunit.client.0.smithi077.stdout:2/342: chown d9/f3d 7 1 2024-04-26T04:53:00.405 INFO:tasks.workunit.client.1.smithi078.stdout:0/689: creat d2/d1e/d5a/fe7 x:0 0 0 2024-04-26T04:53:00.405 INFO:tasks.workunit.client.1.smithi078.stdout:0/690: readlink d2/d39/l43 0 2024-04-26T04:53:00.406 INFO:tasks.workunit.client.1.smithi078.stdout:8/638: dread d0/d19/f1c [0,4194304] 0 2024-04-26T04:53:00.418 INFO:tasks.workunit.client.1.smithi078.stdout:3/696: creat d16/d26/d2e/d5f/ff8 x:0 0 0 2024-04-26T04:53:00.421 INFO:tasks.workunit.client.1.smithi078.stdout:0/691: mknod d2/d9/d60/d6a/da5/ce8 0 2024-04-26T04:53:00.423 INFO:tasks.workunit.client.1.smithi078.stdout:4/703: dwrite d0/ffb [0,4194304] 0 2024-04-26T04:53:00.423 INFO:tasks.workunit.client.1.smithi078.stdout:0/692: write d2/d9/d60/d6a/fc4 [472475,5752] 0 2024-04-26T04:53:00.423 INFO:tasks.workunit.client.1.smithi078.stdout:7/647: dread d9/f6a [0,4194304] 0 2024-04-26T04:53:00.425 INFO:tasks.workunit.client.1.smithi078.stdout:2/652: rename d8/d53/c96 to d8/d11/d14/d69/ce5 0 2024-04-26T04:53:00.426 INFO:tasks.workunit.client.0.smithi077.stdout:8/314: dwrite d4/df/d22/d2d/f1c [0,4194304] 0 2024-04-26T04:53:00.428 INFO:tasks.workunit.client.1.smithi078.stdout:5/626: write d7/d18/d1c/d5a/f7c [1920239,85659] 0 2024-04-26T04:53:00.428 INFO:tasks.workunit.client.0.smithi077.stdout:8/315: truncate d4/df/d22/f50 1211409 0 2024-04-26T04:53:00.431 INFO:tasks.workunit.client.1.smithi078.stdout:9/619: fsync dc/df/f9f 0 2024-04-26T04:53:00.431 INFO:tasks.workunit.client.0.smithi077.stdout:8/316: rmdir d4/df/d22/d2d 39 2024-04-26T04:53:00.433 INFO:tasks.workunit.client.1.smithi078.stdout:7/648: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fb8 [0,4194304] 0 2024-04-26T04:53:00.433 INFO:tasks.workunit.client.1.smithi078.stdout:0/693: chown d2/d9/d78/d84/f88 110 1 2024-04-26T04:53:00.434 INFO:tasks.workunit.client.1.smithi078.stdout:4/704: unlink d0/d1d/de1/d20/d31/d47/db2/f4d 0 2024-04-26T04:53:00.434 INFO:tasks.workunit.client.1.smithi078.stdout:7/649: dread - d9/da/d11/d13/d50/fc8 zero size 2024-04-26T04:53:00.435 INFO:tasks.workunit.client.1.smithi078.stdout:8/639: rename d0/d7a/d83/f94 to d0/d19/d25/d32/d8a/d92/fcf 0 2024-04-26T04:53:00.437 INFO:tasks.workunit.client.1.smithi078.stdout:1/722: sync 2024-04-26T04:53:00.444 INFO:tasks.workunit.client.1.smithi078.stdout:6/746: dwrite d2/d1a/fd [4194304,4194304] 0 2024-04-26T04:53:00.445 INFO:tasks.workunit.client.1.smithi078.stdout:3/697: dwrite d16/d26/d2e/d5f/d49/fa4 [4194304,4194304] 0 2024-04-26T04:53:00.445 INFO:tasks.workunit.client.1.smithi078.stdout:6/747: chown d2/de/dcd 619 1 2024-04-26T04:53:00.446 INFO:tasks.workunit.client.0.smithi077.stdout:8/317: getdents d4/df/d22/d2d/d2b 0 2024-04-26T04:53:00.447 INFO:tasks.workunit.client.1.smithi078.stdout:6/748: write d2/fe1 [713737,66740] 0 2024-04-26T04:53:00.448 INFO:tasks.workunit.client.1.smithi078.stdout:0/694: rename d2/d9/d18/d7d/da9/dab/dc6/f79 to d2/d39/d96/fe9 0 2024-04-26T04:53:00.449 INFO:tasks.workunit.client.0.smithi077.stdout:8/318: rmdir d4/df/d22/d2d/d2b/d33 39 2024-04-26T04:53:00.451 INFO:tasks.workunit.client.1.smithi078.stdout:3/698: dread d16/f3f [4194304,4194304] 0 2024-04-26T04:53:00.452 INFO:tasks.workunit.client.1.smithi078.stdout:2/653: mknod d8/d11/d14/d69/d8f/d34/dd6/ce6 0 2024-04-26T04:53:00.453 INFO:tasks.workunit.client.0.smithi077.stdout:8/319: mknod d4/d17/d35/d5b/c6e 0 2024-04-26T04:53:00.454 INFO:tasks.workunit.client.1.smithi078.stdout:1/723: rmdir d7/d1e/d9e/db/d37/d58/d5c/d68/de8 39 2024-04-26T04:53:00.455 INFO:tasks.workunit.client.0.smithi077.stdout:8/320: chown d4/l8 527351924 1 2024-04-26T04:53:00.457 INFO:tasks.workunit.client.1.smithi078.stdout:9/620: symlink dc/d55/lb0 0 2024-04-26T04:53:00.458 INFO:tasks.workunit.client.0.smithi077.stdout:8/321: mkdir d4/df/d6f 0 2024-04-26T04:53:00.459 INFO:tasks.workunit.client.1.smithi078.stdout:8/640: dwrite d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:53:00.460 INFO:tasks.workunit.client.1.smithi078.stdout:0/695: symlink d2/dc1/lea 0 2024-04-26T04:53:00.462 INFO:tasks.workunit.client.1.smithi078.stdout:6/749: truncate d2/de/d27/da9/da/d36/db9/f57 3648160 0 2024-04-26T04:53:00.462 INFO:tasks.workunit.client.1.smithi078.stdout:2/654: dwrite d8/d11/d14/d69/d8f/f88 [0,4194304] 0 2024-04-26T04:53:00.464 INFO:tasks.workunit.client.1.smithi078.stdout:3/699: creat d16/d26/d2b/d31/d71/d8a/d98/ff9 x:0 0 0 2024-04-26T04:53:00.466 INFO:tasks.workunit.client.1.smithi078.stdout:3/700: chown d16/d26/d2e/d5f/d49/dad/db0/lce 169 1 2024-04-26T04:53:00.468 INFO:tasks.workunit.client.0.smithi077.stdout:8/322: truncate d4/f5 4612358 0 2024-04-26T04:53:00.468 INFO:tasks.workunit.client.1.smithi078.stdout:5/627: write d7/d18/f27 [3144865,61250] 0 2024-04-26T04:53:00.469 INFO:tasks.workunit.client.1.smithi078.stdout:4/705: truncate d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/f8d 581553 0 2024-04-26T04:53:00.469 INFO:tasks.workunit.client.1.smithi078.stdout:4/706: stat d0/de/d22/l5f 0 2024-04-26T04:53:00.474 INFO:tasks.workunit.client.1.smithi078.stdout:5/628: dwrite d7/d18/d32/f4a [4194304,4194304] 0 2024-04-26T04:53:00.475 INFO:tasks.workunit.client.0.smithi077.stdout:8/323: dwrite d4/d17/f2c [0,4194304] 0 2024-04-26T04:53:00.477 INFO:tasks.workunit.client.1.smithi078.stdout:8/641: rmdir d0/d19/d25 39 2024-04-26T04:53:00.479 INFO:tasks.workunit.client.0.smithi077.stdout:8/324: creat d4/d17/d35/f70 x:0 0 0 2024-04-26T04:53:00.480 INFO:tasks.workunit.client.0.smithi077.stdout:8/325: chown d4/df/d22/d2d/f1b 126 1 2024-04-26T04:53:00.487 INFO:tasks.workunit.client.1.smithi078.stdout:7/650: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/d68/f7b [4194304,4194304] 0 2024-04-26T04:53:00.489 INFO:tasks.workunit.client.0.smithi077.stdout:8/326: creat d4/df/d6f/f71 x:0 0 0 2024-04-26T04:53:00.489 INFO:tasks.workunit.client.0.smithi077.stdout:8/327: readlink d4/l8 0 2024-04-26T04:53:00.493 INFO:tasks.workunit.client.0.smithi077.stdout:8/328: creat d4/df/d18/d56/f72 x:0 0 0 2024-04-26T04:53:00.493 INFO:tasks.workunit.client.1.smithi078.stdout:4/707: dwrite d0/de/d22/d27/d56/f5c [4194304,4194304] 0 2024-04-26T04:53:00.496 INFO:tasks.workunit.client.1.smithi078.stdout:0/696: mkdir d2/d9/d78/deb 0 2024-04-26T04:53:00.498 INFO:tasks.workunit.client.1.smithi078.stdout:7/651: rmdir d9/da 39 2024-04-26T04:53:00.498 INFO:tasks.workunit.client.1.smithi078.stdout:8/642: mknod d0/d19/d52/d5c/da2/cd0 0 2024-04-26T04:53:00.500 INFO:tasks.workunit.client.1.smithi078.stdout:0/697: chown d2/d1e/d5a/f71 2139 1 2024-04-26T04:53:00.500 INFO:tasks.workunit.client.1.smithi078.stdout:2/655: rmdir d8/d11/d14/d69/dc5 0 2024-04-26T04:53:00.501 INFO:tasks.workunit.client.1.smithi078.stdout:9/621: dwrite dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:53:00.502 INFO:tasks.workunit.client.1.smithi078.stdout:6/750: link d2/d1a/d38/f68 d2/de/d27/da9/d3a/d69/daa/fea 0 2024-04-26T04:53:00.502 INFO:tasks.workunit.client.1.smithi078.stdout:2/656: stat d8/d11/d18/d5d/d8b/db0/fda 0 2024-04-26T04:53:00.503 INFO:tasks.workunit.client.1.smithi078.stdout:3/701: rename d16/d26/d2b/d66/ld9 to d16/d26/d2b/d31/lfa 0 2024-04-26T04:53:00.506 INFO:tasks.workunit.client.1.smithi078.stdout:3/702: write d16/d26/d2b/d31/d71/d8a/d98/ff9 [277711,130143] 0 2024-04-26T04:53:00.507 INFO:tasks.workunit.client.0.smithi077.stdout:8/329: dread d4/df/d22/d2d/f1e [0,4194304] 0 2024-04-26T04:53:00.508 INFO:tasks.workunit.client.0.smithi077.stdout:8/330: fsync d4/d17/d35/f52 0 2024-04-26T04:53:00.510 INFO:tasks.workunit.client.1.smithi078.stdout:9/622: creat dc/df/d27/fb1 x:0 0 0 2024-04-26T04:53:00.514 INFO:tasks.workunit.client.1.smithi078.stdout:1/724: rename d7/d1e/d9e/db/d49/ca7 to d7/d1e/d9e/db/d49/dba/dc5/dcf/ceb 0 2024-04-26T04:53:00.514 INFO:tasks.workunit.client.0.smithi077.stdout:8/331: mkdir d4/d69/d73 0 2024-04-26T04:53:00.515 INFO:tasks.workunit.client.0.smithi077.stdout:8/332: chown d4/f42 415246042 1 2024-04-26T04:53:00.516 INFO:tasks.workunit.client.1.smithi078.stdout:6/751: mknod d2/de/d27/da9/ceb 0 2024-04-26T04:53:00.518 INFO:tasks.workunit.client.0.smithi077.stdout:8/333: dread d4/df/d18/f26 [0,4194304] 0 2024-04-26T04:53:00.518 INFO:tasks.workunit.client.0.smithi077.stdout:8/334: chown d4/d17/f3c 31902 1 2024-04-26T04:53:00.519 INFO:tasks.workunit.client.1.smithi078.stdout:2/657: mknod d8/d11/d14/d69/d8f/d34/ce7 0 2024-04-26T04:53:00.523 INFO:tasks.workunit.client.1.smithi078.stdout:3/703: rename d16/d26/d2b/d54 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb 0 2024-04-26T04:53:00.524 INFO:tasks.workunit.client.0.smithi077.stdout:8/335: creat d4/df/d18/f74 x:0 0 0 2024-04-26T04:53:00.524 INFO:tasks.workunit.client.1.smithi078.stdout:5/629: dread d7/d18/f2e [0,4194304] 0 2024-04-26T04:53:00.525 INFO:tasks.workunit.client.0.smithi077.stdout:8/336: chown d4/f40 2948 1 2024-04-26T04:53:00.525 INFO:tasks.workunit.client.1.smithi078.stdout:5/630: write d7/d48/d57/d65/f5b [1397971,57080] 0 2024-04-26T04:53:00.526 INFO:tasks.workunit.client.1.smithi078.stdout:6/752: rmdir d2/de/d27/da9/da/d36 39 2024-04-26T04:53:00.531 INFO:tasks.workunit.client.0.smithi077.stdout:8/337: stat d4/df/d22/d2d/d2b/f5d 0 2024-04-26T04:53:00.532 INFO:tasks.workunit.client.0.smithi077.stdout:8/338: write d4/df/d22/f68 [2267424,122969] 0 2024-04-26T04:53:00.533 INFO:tasks.workunit.client.1.smithi078.stdout:3/704: unlink d16/d26/d2e/ca3 0 2024-04-26T04:53:00.534 INFO:tasks.workunit.client.1.smithi078.stdout:8/643: write d0/d19/d52/faa [700099,4739] 0 2024-04-26T04:53:00.538 INFO:tasks.workunit.client.1.smithi078.stdout:7/652: dread d9/da/d11/d13/d1a/f40 [0,4194304] 0 2024-04-26T04:53:00.538 INFO:tasks.workunit.client.1.smithi078.stdout:6/753: fsync d2/de/f67 0 2024-04-26T04:53:00.540 INFO:tasks.workunit.client.1.smithi078.stdout:2/658: mknod d8/d11/d14/d69/d8f/d34/ce8 0 2024-04-26T04:53:00.540 INFO:tasks.workunit.client.1.smithi078.stdout:0/698: dwrite d2/f12 [0,4194304] 0 2024-04-26T04:53:00.541 INFO:tasks.workunit.client.1.smithi078.stdout:2/659: fsync d8/d11/d14/f8d 0 2024-04-26T04:53:00.541 INFO:tasks.workunit.client.0.smithi077.stdout:8/339: mkdir d4/df/d22/d75 0 2024-04-26T04:53:00.542 INFO:tasks.workunit.client.1.smithi078.stdout:0/699: read d2/f15 [7383,15599] 0 2024-04-26T04:53:00.543 INFO:tasks.workunit.client.1.smithi078.stdout:9/623: getdents dc/df/d4b/d22/d2a 0 2024-04-26T04:53:00.546 INFO:tasks.workunit.client.1.smithi078.stdout:8/644: fdatasync d0/d19/d25/d32/d8a/fc4 0 2024-04-26T04:53:00.549 INFO:tasks.workunit.client.1.smithi078.stdout:7/653: chown d9/da/d33/l5e 712546 1 2024-04-26T04:53:00.551 INFO:tasks.workunit.client.1.smithi078.stdout:6/754: unlink d2/d1a/c26 0 2024-04-26T04:53:00.551 INFO:tasks.workunit.client.1.smithi078.stdout:6/755: chown d2/de/d27/d58/f76 116996 1 2024-04-26T04:53:00.552 INFO:tasks.workunit.client.1.smithi078.stdout:6/756: write d2/d1a/d5c/f6d [1810664,58396] 0 2024-04-26T04:53:00.556 INFO:tasks.workunit.client.1.smithi078.stdout:2/660: rename d8/d11/d18/d5d/c85 to d8/d11/d14/d69/d8f/d34/dd3/dd9/ce9 0 2024-04-26T04:53:00.557 INFO:tasks.workunit.client.1.smithi078.stdout:9/624: symlink dc/df/d89/lb2 0 2024-04-26T04:53:00.558 INFO:tasks.workunit.client.0.smithi077.stdout:8/340: write d4/f5 [273424,93858] 0 2024-04-26T04:53:00.558 INFO:tasks.workunit.client.1.smithi078.stdout:5/631: creat d7/d21/d2f/db6/fd1 x:0 0 0 2024-04-26T04:53:00.559 INFO:tasks.workunit.client.0.smithi077.stdout:8/341: write d4/df/d18/d56/f72 [940931,95591] 0 2024-04-26T04:53:00.560 INFO:tasks.workunit.client.0.smithi077.stdout:8/342: write d4/d17/d35/f52 [5192229,17691] 0 2024-04-26T04:53:00.567 INFO:tasks.workunit.client.1.smithi078.stdout:3/705: link d16/d26/d2e/d5f/d49/d77/f7b d16/d9b/ffc 0 2024-04-26T04:53:00.567 INFO:tasks.workunit.client.1.smithi078.stdout:8/645: write d0/d19/d25/d36/f37 [5192334,84688] 0 2024-04-26T04:53:00.569 INFO:tasks.workunit.client.1.smithi078.stdout:2/661: rename d8/db7/fc1 to d8/d11/d14/d69/d8f/d34/dd3/de4/fea 0 2024-04-26T04:53:00.570 INFO:tasks.workunit.client.1.smithi078.stdout:0/700: dwrite d2/d39/f9d [0,4194304] 0 2024-04-26T04:53:00.570 INFO:tasks.workunit.client.1.smithi078.stdout:7/654: write d9/da/d11/d13/d1a/f42 [1145258,81362] 0 2024-04-26T04:53:00.570 INFO:tasks.workunit.client.1.smithi078.stdout:9/625: mkdir dc/df/d4b/d22/d2a/d57/d69/db3 0 2024-04-26T04:53:00.573 INFO:tasks.workunit.client.1.smithi078.stdout:2/662: read d8/f1f [133049,7261] 0 2024-04-26T04:53:00.574 INFO:tasks.workunit.client.1.smithi078.stdout:3/706: rmdir d16/d26/d2b/d66 39 2024-04-26T04:53:00.574 INFO:tasks.workunit.client.1.smithi078.stdout:8/646: mknod d0/d19/d52/d95/cd1 0 2024-04-26T04:53:00.575 INFO:tasks.workunit.client.1.smithi078.stdout:7/655: fdatasync d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc1 0 2024-04-26T04:53:00.576 INFO:tasks.workunit.client.1.smithi078.stdout:5/632: link d7/d18/d1c/f71 d7/d21/d2f/fd2 0 2024-04-26T04:53:00.580 INFO:tasks.workunit.client.0.smithi077.stdout:8/343: dread d4/f40 [0,4194304] 0 2024-04-26T04:53:00.581 INFO:tasks.workunit.client.1.smithi078.stdout:8/647: chown d0/d19/d25/d32/d8a/ccb 28559 1 2024-04-26T04:53:00.581 INFO:tasks.workunit.client.0.smithi077.stdout:8/344: read - d4/df/d22/f62 zero size 2024-04-26T04:53:00.582 INFO:tasks.workunit.client.0.smithi077.stdout:8/345: chown d4/df/d22/f50 2 1 2024-04-26T04:53:00.583 INFO:tasks.workunit.client.1.smithi078.stdout:9/626: creat dc/df/d4b/d22/d2d/d98/fb4 x:0 0 0 2024-04-26T04:53:00.583 INFO:tasks.workunit.client.1.smithi078.stdout:2/663: dwrite d8/d11/d14/d69/d8f/d34/dd3/de4/d52/fca [0,4194304] 0 2024-04-26T04:53:00.585 INFO:tasks.workunit.client.1.smithi078.stdout:6/757: dread d2/de/d27/da9/d11/fc2 [0,4194304] 0 2024-04-26T04:53:00.585 INFO:tasks.workunit.client.1.smithi078.stdout:5/633: creat d7/d18/d32/d84/fd3 x:0 0 0 2024-04-26T04:53:00.586 INFO:tasks.workunit.client.1.smithi078.stdout:7/656: mknod d9/cd3 0 2024-04-26T04:53:00.587 INFO:tasks.workunit.client.1.smithi078.stdout:5/634: chown d7/d48/d57/d65/d70/dca 14226444 1 2024-04-26T04:53:00.587 INFO:tasks.workunit.client.1.smithi078.stdout:4/708: sync 2024-04-26T04:53:00.587 INFO:tasks.workunit.client.1.smithi078.stdout:1/725: sync 2024-04-26T04:53:00.587 INFO:tasks.workunit.client.0.smithi077.stdout:8/346: unlink d4/d17/c5e 0 2024-04-26T04:53:00.589 INFO:tasks.workunit.client.1.smithi078.stdout:4/709: chown d0/d1d/de1/d20/d31/f6f 12094 1 2024-04-26T04:53:00.594 INFO:tasks.workunit.client.0.smithi077.stdout:8/347: creat d4/f76 x:0 0 0 2024-04-26T04:53:00.594 INFO:tasks.workunit.client.1.smithi078.stdout:9/627: mkdir dc/df/d4b/d22/d2a/d57/d69/db5 0 2024-04-26T04:53:00.595 INFO:tasks.workunit.client.1.smithi078.stdout:7/657: mkdir d9/da/d11/d13/d1a/dd4 0 2024-04-26T04:53:00.596 INFO:tasks.workunit.client.1.smithi078.stdout:5/635: unlink d7/d18/d32/d37/f3d 0 2024-04-26T04:53:00.597 INFO:tasks.workunit.client.1.smithi078.stdout:3/707: link f6 d16/d26/d2e/d5f/dbb/ffd 0 2024-04-26T04:53:00.598 INFO:tasks.workunit.client.0.smithi077.stdout:8/348: symlink d4/df/d22/d2d/d2b/l77 0 2024-04-26T04:53:00.599 INFO:tasks.workunit.client.0.smithi077.stdout:8/349: chown d4/df/f46 3786047 1 2024-04-26T04:53:00.599 INFO:tasks.workunit.client.0.smithi077.stdout:8/350: chown d4/d17/f2c 889 1 2024-04-26T04:53:00.601 INFO:tasks.workunit.client.1.smithi078.stdout:6/758: dread d2/d1a/d5c/f6d [0,4194304] 0 2024-04-26T04:53:00.601 INFO:tasks.workunit.client.1.smithi078.stdout:0/701: dread d2/d9/d31/fce [0,4194304] 0 2024-04-26T04:53:00.603 INFO:tasks.workunit.client.0.smithi077.stdout:8/351: dread d4/df/f15 [0,4194304] 0 2024-04-26T04:53:00.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/664: write d8/d11/d14/d69/d8f/d34/dd3/de4/f6a [2471074,44679] 0 2024-04-26T04:53:00.605 INFO:tasks.workunit.client.1.smithi078.stdout:2/665: dread - d8/db7/fe2 zero size 2024-04-26T04:53:00.615 INFO:tasks.workunit.client.0.smithi077.stdout:8/352: write d4/df/d22/f64 [380341,2706] 0 2024-04-26T04:53:00.617 INFO:tasks.workunit.client.0.smithi077.stdout:8/353: unlink d4/df/d22/f64 0 2024-04-26T04:53:00.624 INFO:tasks.workunit.client.0.smithi077.stdout:8/354: dwrite d4/df/d22/d2d/d2b/d33/f67 [0,4194304] 0 2024-04-26T04:53:00.628 INFO:tasks.workunit.client.1.smithi078.stdout:8/648: rename d0/d19/d25/d32/d8a/d92/l84 to d0/d19/d25/ld2 0 2024-04-26T04:53:00.631 INFO:tasks.workunit.client.1.smithi078.stdout:3/708: creat d16/d7e/ffe x:0 0 0 2024-04-26T04:53:00.632 INFO:tasks.workunit.client.0.smithi077.stdout:8/355: dwrite d4/df/d6f/f71 [0,4194304] 0 2024-04-26T04:53:00.639 INFO:tasks.workunit.client.1.smithi078.stdout:6/759: creat d2/de/d27/da9/d3a/d69/daa/dba/fec x:0 0 0 2024-04-26T04:53:00.640 INFO:tasks.workunit.client.0.smithi077.stdout:8/356: fsync d4/f10 0 2024-04-26T04:53:00.642 INFO:tasks.workunit.client.1.smithi078.stdout:2/666: creat d8/d11/d14/d69/d8f/d34/dd3/de4/d52/feb x:0 0 0 2024-04-26T04:53:00.644 INFO:tasks.workunit.client.1.smithi078.stdout:8/649: mkdir d0/d19/d52/d5c/da2/dd3 0 2024-04-26T04:53:00.645 INFO:tasks.workunit.client.1.smithi078.stdout:9/628: creat dc/df/d4b/d22/d2a/d57/d69/db5/fb6 x:0 0 0 2024-04-26T04:53:00.646 INFO:tasks.workunit.client.1.smithi078.stdout:5/636: getdents d7/d48/d57/d65/d70/dca 0 2024-04-26T04:53:00.647 INFO:tasks.workunit.client.1.smithi078.stdout:0/702: mknod d2/d9/d18/db7/d92/cec 0 2024-04-26T04:53:00.650 INFO:tasks.workunit.client.1.smithi078.stdout:2/667: mkdir d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/dec 0 2024-04-26T04:53:00.654 INFO:tasks.workunit.client.0.smithi077.stdout:8/357: dread d4/df/d22/f43 [0,4194304] 0 2024-04-26T04:53:00.655 INFO:tasks.workunit.client.0.smithi077.stdout:7/342: sync 2024-04-26T04:53:00.655 INFO:tasks.workunit.client.0.smithi077.stdout:0/256: sync 2024-04-26T04:53:00.655 INFO:tasks.workunit.client.0.smithi077.stdout:1/371: sync 2024-04-26T04:53:00.655 INFO:tasks.workunit.client.0.smithi077.stdout:4/298: sync 2024-04-26T04:53:00.656 INFO:tasks.workunit.client.0.smithi077.stdout:9/299: sync 2024-04-26T04:53:00.656 INFO:tasks.workunit.client.0.smithi077.stdout:3/275: sync 2024-04-26T04:53:00.656 INFO:tasks.workunit.client.0.smithi077.stdout:6/358: sync 2024-04-26T04:53:00.656 INFO:tasks.workunit.client.0.smithi077.stdout:5/367: sync 2024-04-26T04:53:00.656 INFO:tasks.workunit.client.1.smithi078.stdout:5/637: creat d7/da/d1e/d91/fd4 x:0 0 0 2024-04-26T04:53:00.657 INFO:tasks.workunit.client.0.smithi077.stdout:9/300: stat d9/dd/d1b/c48 0 2024-04-26T04:53:00.657 INFO:tasks.workunit.client.0.smithi077.stdout:3/276: write d0/d7/d15/f44 [757384,43031] 0 2024-04-26T04:53:00.657 INFO:tasks.workunit.client.1.smithi078.stdout:2/668: dwrite d8/d11/d14/d69/d8f/d34/dd3/de4/f6a [0,4194304] 0 2024-04-26T04:53:00.658 INFO:tasks.workunit.client.0.smithi077.stdout:1/372: chown d9/db/dd/l17 431519741 1 2024-04-26T04:53:00.659 INFO:tasks.workunit.client.1.smithi078.stdout:8/650: dread d0/d19/d25/d32/f76 [0,4194304] 0 2024-04-26T04:53:00.660 INFO:tasks.workunit.client.1.smithi078.stdout:7/658: rename f7 to d9/fd5 0 2024-04-26T04:53:00.662 INFO:tasks.workunit.client.0.smithi077.stdout:6/359: write d8/f11 [1050232,95820] 0 2024-04-26T04:53:00.665 INFO:tasks.workunit.client.1.smithi078.stdout:2/669: dwrite d8/d11/d14/fbf [0,4194304] 0 2024-04-26T04:53:00.665 INFO:tasks.workunit.client.0.smithi077.stdout:6/360: truncate d8/d52/d53/d5e/f69 444239 0 2024-04-26T04:53:00.673 INFO:tasks.workunit.client.1.smithi078.stdout:6/760: rmdir d2/de/d27/da9/d3a/d69/daa/dba 39 2024-04-26T04:53:00.674 INFO:tasks.workunit.client.1.smithi078.stdout:5/638: fdatasync d7/d48/d57/d65/f3a 0 2024-04-26T04:53:00.675 INFO:tasks.workunit.client.1.smithi078.stdout:4/710: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [0,4194304] 0 2024-04-26T04:53:00.675 INFO:tasks.workunit.client.0.smithi077.stdout:7/343: dread da/f11 [0,4194304] 0 2024-04-26T04:53:00.676 INFO:tasks.workunit.client.0.smithi077.stdout:3/277: symlink d0/d7/d1f/l55 0 2024-04-26T04:53:00.678 INFO:tasks.workunit.client.1.smithi078.stdout:8/651: rename d0/d19/d52/d35/d73/lab to d0/d19/d52/d5c/da2/ld4 0 2024-04-26T04:53:00.680 INFO:tasks.workunit.client.1.smithi078.stdout:7/659: symlink d9/da/d11/d13/d50/ld6 0 2024-04-26T04:53:00.681 INFO:tasks.workunit.client.0.smithi077.stdout:7/344: dwrite da/d15/f32 [0,4194304] 0 2024-04-26T04:53:00.684 INFO:tasks.workunit.client.0.smithi077.stdout:2/343: sync 2024-04-26T04:53:00.686 INFO:tasks.workunit.client.0.smithi077.stdout:5/368: symlink d1/d30/d33/l88 0 2024-04-26T04:53:00.687 INFO:tasks.workunit.client.1.smithi078.stdout:2/670: symlink d8/d11/d14/d69/d8f/d34/dd3/de4/d52/led 0 2024-04-26T04:53:00.690 INFO:tasks.workunit.client.1.smithi078.stdout:3/709: getdents d16/d26/d2b/d31/d71 0 2024-04-26T04:53:00.690 INFO:tasks.workunit.client.1.smithi078.stdout:0/703: write d2/d9/d18/f8a [71302,106592] 0 2024-04-26T04:53:00.691 INFO:tasks.workunit.client.0.smithi077.stdout:6/361: mknod d8/d52/d53/c81 0 2024-04-26T04:53:00.693 INFO:tasks.workunit.client.1.smithi078.stdout:9/629: mknod dc/df/d37/cb7 0 2024-04-26T04:53:00.695 INFO:tasks.workunit.client.1.smithi078.stdout:1/726: sync 2024-04-26T04:53:00.697 INFO:tasks.workunit.client.0.smithi077.stdout:8/358: dwrite d4/d17/f6a [0,4194304] 0 2024-04-26T04:53:00.698 INFO:tasks.workunit.client.1.smithi078.stdout:6/761: dread d2/d1a/fa6 [0,4194304] 0 2024-04-26T04:53:00.705 INFO:tasks.workunit.client.1.smithi078.stdout:4/711: rename d0/d1d/de1/d20/d31/d47/db2/d44/d55/d5e/d75 to d0/d1d/d46/d100 0 2024-04-26T04:53:00.705 INFO:tasks.workunit.client.0.smithi077.stdout:4/299: truncate d4/f27 1032186 0 2024-04-26T04:53:00.707 INFO:tasks.workunit.client.0.smithi077.stdout:1/373: link d9/d53/d6e/f40 d9/d53/f6f 0 2024-04-26T04:53:00.709 INFO:tasks.workunit.client.0.smithi077.stdout:7/345: unlink da/d13/c72 0 2024-04-26T04:53:00.710 INFO:tasks.workunit.client.1.smithi078.stdout:8/652: read d0/d19/d52/d35/f8f [59883,43813] 0 2024-04-26T04:53:00.710 INFO:tasks.workunit.client.0.smithi077.stdout:7/346: fdatasync da/d13/f4b 0 2024-04-26T04:53:00.710 INFO:tasks.workunit.client.1.smithi078.stdout:8/653: chown d0/d19/d52/d35/d44/l6a 13541 1 2024-04-26T04:53:00.711 INFO:tasks.workunit.client.0.smithi077.stdout:2/344: unlink d9/c40 0 2024-04-26T04:53:00.711 INFO:tasks.workunit.client.1.smithi078.stdout:3/710: readlink d16/d26/d2b/d31/lfa 0 2024-04-26T04:53:00.712 INFO:tasks.workunit.client.0.smithi077.stdout:2/345: write d9/d15/f37 [3781811,105195] 0 2024-04-26T04:53:00.713 INFO:tasks.workunit.client.1.smithi078.stdout:0/704: unlink d2/d1e/d5a/fe7 0 2024-04-26T04:53:00.714 INFO:tasks.workunit.client.0.smithi077.stdout:6/362: mknod d8/d52/d53/c82 0 2024-04-26T04:53:00.715 INFO:tasks.workunit.client.0.smithi077.stdout:6/363: stat d8/d2a/c5d 0 2024-04-26T04:53:00.715 INFO:tasks.workunit.client.0.smithi077.stdout:6/364: fdatasync d8/d52/d53/f72 0 2024-04-26T04:53:00.717 INFO:tasks.workunit.client.0.smithi077.stdout:6/365: write d8/d52/f6e [14933,18746] 0 2024-04-26T04:53:00.717 INFO:tasks.workunit.client.1.smithi078.stdout:3/711: dwrite d16/d26/d2e/d5f/d49/d77/fc6 [0,4194304] 0 2024-04-26T04:53:00.717 INFO:tasks.workunit.client.0.smithi077.stdout:8/359: stat d4/df/d22/d2d/c59 0 2024-04-26T04:53:00.720 INFO:tasks.workunit.client.0.smithi077.stdout:9/301: creat d9/dd/d28/f6b x:0 0 0 2024-04-26T04:53:00.732 INFO:tasks.workunit.client.1.smithi078.stdout:5/639: creat d7/d48/d57/d65/d46/dc4/fd5 x:0 0 0 2024-04-26T04:53:00.737 INFO:tasks.workunit.client.1.smithi078.stdout:4/712: creat d0/d1d/de1/d20/d31/d47/db2/d4c/df6/f101 x:0 0 0 2024-04-26T04:53:00.738 INFO:tasks.workunit.client.0.smithi077.stdout:0/257: getdents d0/d2c/d4e 0 2024-04-26T04:53:00.739 INFO:tasks.workunit.client.1.smithi078.stdout:8/654: creat d0/d19/d52/d35/d44/d6b/fd5 x:0 0 0 2024-04-26T04:53:00.740 INFO:tasks.workunit.client.0.smithi077.stdout:8/360: dread d4/fe [0,4194304] 0 2024-04-26T04:53:00.740 INFO:tasks.workunit.client.1.smithi078.stdout:7/660: dread d9/da/d11/d13/d1a/f29 [4194304,4194304] 0 2024-04-26T04:53:00.741 INFO:tasks.workunit.client.0.smithi077.stdout:2/346: mknod d9/dc/d1d/d34/d39/d63/c75 0 2024-04-26T04:53:00.745 INFO:tasks.workunit.client.0.smithi077.stdout:6/366: truncate d8/f3f 203930 0 2024-04-26T04:53:00.747 INFO:tasks.workunit.client.0.smithi077.stdout:9/302: mknod d9/dd/d1b/d4e/c6c 0 2024-04-26T04:53:00.749 INFO:tasks.workunit.client.0.smithi077.stdout:2/347: dwrite d9/f12 [4194304,4194304] 0 2024-04-26T04:53:00.751 INFO:tasks.workunit.client.1.smithi078.stdout:1/727: mknod d7/d1e/d9e/dd/d20/d53/cec 0 2024-04-26T04:53:00.752 INFO:tasks.workunit.client.1.smithi078.stdout:1/728: readlink d7/ddd/dab/lc2 0 2024-04-26T04:53:00.753 INFO:tasks.workunit.client.1.smithi078.stdout:0/705: write d2/d1e/f67 [3693511,67054] 0 2024-04-26T04:53:00.754 INFO:tasks.workunit.client.1.smithi078.stdout:0/706: chown d2/d9/f75 0 1 2024-04-26T04:53:00.754 INFO:tasks.workunit.client.1.smithi078.stdout:0/707: chown d2/dc1 1015175470 1 2024-04-26T04:53:00.754 INFO:tasks.workunit.client.0.smithi077.stdout:7/347: truncate da/d13/f19 2987015 0 2024-04-26T04:53:00.758 INFO:tasks.workunit.client.1.smithi078.stdout:9/630: truncate dc/df/d17/f2e 4107508 0 2024-04-26T04:53:00.759 INFO:tasks.workunit.client.1.smithi078.stdout:9/631: readlink dc/df/d17/l50 0 2024-04-26T04:53:00.760 INFO:tasks.workunit.client.1.smithi078.stdout:5/640: mkdir d7/da/d1e/d91/dd6 0 2024-04-26T04:53:00.760 INFO:tasks.workunit.client.1.smithi078.stdout:9/632: chown dc/df/d4b/d3e/c56 7855 1 2024-04-26T04:53:00.764 INFO:tasks.workunit.client.1.smithi078.stdout:2/671: rename d8/d11/d18/dbb/lbc to d8/d11/d14/d69/d8f/d34/dd3/de4/lee 0 2024-04-26T04:53:00.769 INFO:tasks.workunit.client.0.smithi077.stdout:6/367: truncate d8/f49 2550088 0 2024-04-26T04:53:00.769 INFO:tasks.workunit.client.0.smithi077.stdout:6/368: dread - d8/d1c/f4f zero size 2024-04-26T04:53:00.770 INFO:tasks.workunit.client.0.smithi077.stdout:6/369: chown d8/d55/f62 558470 1 2024-04-26T04:53:00.771 INFO:tasks.workunit.client.1.smithi078.stdout:3/712: mkdir d16/dff 0 2024-04-26T04:53:00.773 INFO:tasks.workunit.client.1.smithi078.stdout:1/729: symlink d7/d1e/d9e/db/d49/dba/d87/led 0 2024-04-26T04:53:00.774 INFO:tasks.workunit.client.1.smithi078.stdout:0/708: fsync d2/f48 0 2024-04-26T04:53:00.777 INFO:tasks.workunit.client.1.smithi078.stdout:6/762: link d2/d1a/d38/cb1 d2/de/d27/da9/d3a/d42/ced 0 2024-04-26T04:53:00.784 INFO:tasks.workunit.client.0.smithi077.stdout:8/361: mknod d4/d69/d73/c78 0 2024-04-26T04:53:00.785 INFO:tasks.workunit.client.1.smithi078.stdout:2/672: dread d8/d11/d14/d69/d8f/f49 [0,4194304] 0 2024-04-26T04:53:00.786 INFO:tasks.workunit.client.0.smithi077.stdout:4/300: getdents d4/d17/d18/d43/d53/d56 0 2024-04-26T04:53:00.786 INFO:tasks.workunit.client.0.smithi077.stdout:0/258: dwrite d0/d47/f31 [0,4194304] 0 2024-04-26T04:53:00.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/301: fdatasync d4/de/d15/d1f/f32 0 2024-04-26T04:53:00.789 INFO:tasks.workunit.client.1.smithi078.stdout:7/661: dwrite d9/da/d11/d13/d1a/d2d/f3f [0,4194304] 0 2024-04-26T04:53:00.793 INFO:tasks.workunit.client.1.smithi078.stdout:2/673: dwrite d8/d11/d14/d69/d8f/d34/dd3/de4/f58 [0,4194304] 0 2024-04-26T04:53:00.794 INFO:tasks.workunit.client.0.smithi077.stdout:9/303: link d9/dd/d1b/f66 d9/dd/d1b/d2b/d32/d36/f6d 0 2024-04-26T04:53:00.796 INFO:tasks.workunit.client.1.smithi078.stdout:1/730: mknod d7/d1e/d9e/d92/cee 0 2024-04-26T04:53:00.798 INFO:tasks.workunit.client.1.smithi078.stdout:1/731: fsync d7/d1e/d9e/db/d49/f61 0 2024-04-26T04:53:00.798 INFO:tasks.workunit.client.0.smithi077.stdout:6/370: dread d8/f74 [0,4194304] 0 2024-04-26T04:53:00.799 INFO:tasks.workunit.client.0.smithi077.stdout:6/371: read - d8/d1c/f4c zero size 2024-04-26T04:53:00.800 INFO:tasks.workunit.client.1.smithi078.stdout:5/641: dwrite d7/d18/f1a [0,4194304] 0 2024-04-26T04:53:00.801 INFO:tasks.workunit.client.0.smithi077.stdout:7/348: creat da/d13/d22/d7d/f7e x:0 0 0 2024-04-26T04:53:00.802 INFO:tasks.workunit.client.0.smithi077.stdout:7/349: chown da/d6b/d63/d64/c69 6840697 1 2024-04-26T04:53:00.803 INFO:tasks.workunit.client.1.smithi078.stdout:8/655: truncate d0/d19/d25/d32/d8a/d92/fcf 2633630 0 2024-04-26T04:53:00.805 INFO:tasks.workunit.client.0.smithi077.stdout:8/362: symlink d4/d69/l79 0 2024-04-26T04:53:00.854 INFO:tasks.workunit.client.1.smithi078.stdout:9/633: sync 2024-04-26T04:53:00.887 INFO:tasks.workunit.client.1.smithi078.stdout:7/662: creat d9/da/d33/fd7 x:0 0 0 2024-04-26T04:53:00.888 INFO:tasks.workunit.client.0.smithi077.stdout:1/374: link d9/db/dd/l3a d9/db/dd/l70 0 2024-04-26T04:53:00.905 INFO:tasks.workunit.client.0.smithi077.stdout:9/304: creat d9/d3a/f6e x:0 0 0 2024-04-26T04:53:00.907 INFO:tasks.workunit.client.1.smithi078.stdout:4/713: truncate d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/ff4 2336970 0 2024-04-26T04:53:00.909 INFO:tasks.workunit.client.0.smithi077.stdout:6/372: rmdir d8 39 2024-04-26T04:53:00.910 INFO:tasks.workunit.client.1.smithi078.stdout:0/709: mknod d2/d9/d78/deb/ced 0 2024-04-26T04:53:00.913 INFO:tasks.workunit.client.0.smithi077.stdout:7/350: unlink da/d13/f5b 0 2024-04-26T04:53:00.914 INFO:tasks.workunit.client.0.smithi077.stdout:7/351: truncate da/d15/d1a/f65 131492 0 2024-04-26T04:53:00.915 INFO:tasks.workunit.client.0.smithi077.stdout:7/352: chown da/f11 55 1 2024-04-26T04:53:00.916 INFO:tasks.workunit.client.0.smithi077.stdout:8/363: symlink d4/df/d6f/l7a 0 2024-04-26T04:53:00.916 INFO:tasks.workunit.client.1.smithi078.stdout:0/710: dwrite d2/d9/d18/d7d/fcf [0,4194304] 0 2024-04-26T04:53:00.917 INFO:tasks.workunit.client.0.smithi077.stdout:8/364: read - d4/df/d22/d2d/f30 zero size 2024-04-26T04:53:00.917 INFO:tasks.workunit.client.0.smithi077.stdout:8/365: chown d4/d17/d35 2 1 2024-04-26T04:53:00.917 INFO:tasks.workunit.client.1.smithi078.stdout:6/763: mkdir d2/de/d27/da9/d3a/d69/dee 0 2024-04-26T04:53:00.922 INFO:tasks.workunit.client.1.smithi078.stdout:9/634: creat dc/df/d4b/d22/d2a/d57/d69/db5/fb8 x:0 0 0 2024-04-26T04:53:00.922 INFO:tasks.workunit.client.1.smithi078.stdout:1/732: write d7/d1e/d9e/f62 [3155264,31815] 0 2024-04-26T04:53:00.922 INFO:tasks.workunit.client.0.smithi077.stdout:4/302: symlink d4/d17/d18/d43/d63/l67 0 2024-04-26T04:53:00.923 INFO:tasks.workunit.client.1.smithi078.stdout:7/663: rmdir d9 39 2024-04-26T04:53:00.927 INFO:tasks.workunit.client.0.smithi077.stdout:9/305: mknod d9/dd/d1b/d2b/d32/c6f 0 2024-04-26T04:53:00.927 INFO:tasks.workunit.client.1.smithi078.stdout:1/733: dwrite d7/d1e/d9e/db/d37/fe6 [0,4194304] 0 2024-04-26T04:53:00.928 INFO:tasks.workunit.client.0.smithi077.stdout:9/306: fdatasync d9/dd/d1b/d2b/f57 0 2024-04-26T04:53:00.928 INFO:tasks.workunit.client.0.smithi077.stdout:0/259: truncate d0/d2c/f43 2696917 0 2024-04-26T04:53:00.929 INFO:tasks.workunit.client.0.smithi077.stdout:2/348: getdents d9/dc/d1d/d34/d39/d63/d6a 0 2024-04-26T04:53:00.930 INFO:tasks.workunit.client.0.smithi077.stdout:9/307: dread d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:00.931 INFO:tasks.workunit.client.1.smithi078.stdout:2/674: mkdir d8/d11/d14/d69/d8f/d34/d70/def 0 2024-04-26T04:53:00.934 INFO:tasks.workunit.client.1.smithi078.stdout:4/714: creat d0/d1d/de1/dd8/f102 x:0 0 0 2024-04-26T04:53:00.934 INFO:tasks.workunit.client.0.smithi077.stdout:2/349: dread d9/d15/d1a/d2b/f46 [0,4194304] 0 2024-04-26T04:53:00.935 INFO:tasks.workunit.client.1.smithi078.stdout:3/713: truncate d16/d26/d2b/d66/fc2 223595 0 2024-04-26T04:53:00.939 INFO:tasks.workunit.client.0.smithi077.stdout:6/373: dwrite d8/d55/f62 [4194304,4194304] 0 2024-04-26T04:53:00.939 INFO:tasks.workunit.client.1.smithi078.stdout:5/642: creat d7/da/d1e/d91/dd6/fd7 x:0 0 0 2024-04-26T04:53:00.944 INFO:tasks.workunit.client.0.smithi077.stdout:1/375: symlink d9/db/dd/l71 0 2024-04-26T04:53:00.944 INFO:tasks.workunit.client.0.smithi077.stdout:2/350: dread d9/d15/d1a/f30 [0,4194304] 0 2024-04-26T04:53:00.945 INFO:tasks.workunit.client.0.smithi077.stdout:2/351: read - d9/d15/d44/f58 zero size 2024-04-26T04:53:00.945 INFO:tasks.workunit.client.1.smithi078.stdout:0/711: mknod d2/d9/d60/cee 0 2024-04-26T04:53:00.946 INFO:tasks.workunit.client.1.smithi078.stdout:0/712: fsync d2/d9/d18/db7/d92/fd3 0 2024-04-26T04:53:00.948 INFO:tasks.workunit.client.1.smithi078.stdout:6/764: creat d2/de/d27/da9/d2a/dbf/d5f/fef x:0 0 0 2024-04-26T04:53:00.949 INFO:tasks.workunit.client.1.smithi078.stdout:6/765: readlink d2/de/d27/ldb 0 2024-04-26T04:53:00.949 INFO:tasks.workunit.client.1.smithi078.stdout:6/766: stat d2/f71 0 2024-04-26T04:53:00.950 INFO:tasks.workunit.client.0.smithi077.stdout:0/260: read d0/f15 [459,59069] 0 2024-04-26T04:53:00.950 INFO:tasks.workunit.client.1.smithi078.stdout:6/767: stat d2/de/d27/da9/d2a/dbf 0 2024-04-26T04:53:00.952 INFO:tasks.workunit.client.1.smithi078.stdout:9/635: creat dc/df/d8a/fb9 x:0 0 0 2024-04-26T04:53:00.952 INFO:tasks.workunit.client.0.smithi077.stdout:7/353: write da/d15/f44 [1507048,4243] 0 2024-04-26T04:53:00.955 INFO:tasks.workunit.client.0.smithi077.stdout:9/308: mknod d9/d43/c70 0 2024-04-26T04:53:00.955 INFO:tasks.workunit.client.0.smithi077.stdout:4/303: write d4/de/f2b [201555,5713] 0 2024-04-26T04:53:00.956 INFO:tasks.workunit.client.0.smithi077.stdout:9/309: write d9/f1c [2822602,64242] 0 2024-04-26T04:53:00.963 INFO:tasks.workunit.client.0.smithi077.stdout:9/310: dwrite d9/f22 [4194304,4194304] 0 2024-04-26T04:53:00.963 INFO:tasks.workunit.client.0.smithi077.stdout:9/311: readlink d9/dd/d1b/d2b/l60 0 2024-04-26T04:53:00.964 INFO:tasks.workunit.client.1.smithi078.stdout:4/715: write d0/d1d/de1/d20/d31/d47/db2/d44/d69/fa4 [1593736,61861] 0 2024-04-26T04:53:00.966 INFO:tasks.workunit.client.0.smithi077.stdout:6/374: symlink d8/d52/d53/d5e/l83 0 2024-04-26T04:53:00.970 INFO:tasks.workunit.client.1.smithi078.stdout:3/714: rmdir d16/d26/d2b 39 2024-04-26T04:53:00.973 INFO:tasks.workunit.client.1.smithi078.stdout:3/715: dread d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/fcd [0,4194304] 0 2024-04-26T04:53:00.973 INFO:tasks.workunit.client.0.smithi077.stdout:6/375: dwrite d8/d1c/f4f [0,4194304] 0 2024-04-26T04:53:00.975 INFO:tasks.workunit.client.0.smithi077.stdout:1/376: read d9/db/d2b/f3d [6464,61580] 0 2024-04-26T04:53:00.977 INFO:tasks.workunit.client.1.smithi078.stdout:8/656: rename d0/d19/d52/d3c/f60 to d0/d19/d25/fd6 0 2024-04-26T04:53:00.980 INFO:tasks.workunit.client.0.smithi077.stdout:2/352: dwrite d9/dc/f5c [0,4194304] 0 2024-04-26T04:53:00.981 INFO:tasks.workunit.client.1.smithi078.stdout:8/657: chown d0/d19/d52/d35/d44/f93 3 1 2024-04-26T04:53:00.982 INFO:tasks.workunit.client.1.smithi078.stdout:0/713: dwrite d2/d9/d18/f49 [0,4194304] 0 2024-04-26T04:53:00.985 INFO:tasks.workunit.client.1.smithi078.stdout:2/675: dwrite d8/d11/f74 [0,4194304] 0 2024-04-26T04:53:00.989 INFO:tasks.workunit.client.1.smithi078.stdout:3/716: symlink d16/d26/d2e/d74/l100 0 2024-04-26T04:53:00.991 INFO:tasks.workunit.client.1.smithi078.stdout:0/714: dwrite d2/d9/d18/f8a [0,4194304] 0 2024-04-26T04:53:00.992 INFO:tasks.workunit.client.1.smithi078.stdout:7/664: rename d9/da/d11/d13/c48 to d9/da/d11/d13/d1a/d52/d5a/d41/d68/cd8 0 2024-04-26T04:53:00.994 INFO:tasks.workunit.client.1.smithi078.stdout:6/768: unlink d2/de/d27/da9/da/d36/db9/fd0 0 2024-04-26T04:53:00.998 INFO:tasks.workunit.client.1.smithi078.stdout:1/734: creat d7/d1e/d9e/db/d37/d58/fef x:0 0 0 2024-04-26T04:53:00.999 INFO:tasks.workunit.client.1.smithi078.stdout:1/735: read - d7/d1e/d9e/d92/fe5 zero size 2024-04-26T04:53:00.999 INFO:tasks.workunit.client.1.smithi078.stdout:1/736: readlink d7/d1e/l42 0 2024-04-26T04:53:01.001 INFO:tasks.workunit.client.0.smithi077.stdout:0/261: creat d0/d28/d39/f5c x:0 0 0 2024-04-26T04:53:01.002 INFO:tasks.workunit.client.0.smithi077.stdout:0/262: readlink d0/d47/l3f 0 2024-04-26T04:53:01.004 INFO:tasks.workunit.client.0.smithi077.stdout:7/354: symlink da/d15/l7f 0 2024-04-26T04:53:01.005 INFO:tasks.workunit.client.1.smithi078.stdout:4/716: symlink d0/d1d/de1/l103 0 2024-04-26T04:53:01.006 INFO:tasks.workunit.client.1.smithi078.stdout:4/717: read - d0/d1d/de1/d20/d31/d47/db2/d4c/dba/fcb zero size 2024-04-26T04:53:01.006 INFO:tasks.workunit.client.1.smithi078.stdout:5/643: creat d7/d48/d57/d65/d70/d72/fd8 x:0 0 0 2024-04-26T04:53:01.007 INFO:tasks.workunit.client.1.smithi078.stdout:3/717: dread - d16/d3c/f61 zero size 2024-04-26T04:53:01.007 INFO:tasks.workunit.client.1.smithi078.stdout:5/644: readlink d7/da/d1e/lc0 0 2024-04-26T04:53:01.008 INFO:tasks.workunit.client.0.smithi077.stdout:7/355: read da/f59 [285921,14754] 0 2024-04-26T04:53:01.008 INFO:tasks.workunit.client.1.smithi078.stdout:4/718: readlink d0/d1d/de1/d20/d31/d47/db2/d44/d69/lbc 0 2024-04-26T04:53:01.010 INFO:tasks.workunit.client.1.smithi078.stdout:2/676: dread d8/d11/d14/f90 [0,4194304] 0 2024-04-26T04:53:01.011 INFO:tasks.workunit.client.0.smithi077.stdout:8/366: creat d4/df/d22/d2d/d2b/f7b x:0 0 0 2024-04-26T04:53:01.012 INFO:tasks.workunit.client.1.smithi078.stdout:7/665: rename d9/d57/fc9 to d9/da/d11/d13/d1a/d52/d5a/d28/fd9 0 2024-04-26T04:53:01.013 INFO:tasks.workunit.client.0.smithi077.stdout:9/312: creat d9/d45/f71 x:0 0 0 2024-04-26T04:53:01.013 INFO:tasks.workunit.client.1.smithi078.stdout:3/718: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f68 [0,4194304] 0 2024-04-26T04:53:01.014 INFO:tasks.workunit.client.0.smithi077.stdout:9/313: chown d9/dd/d28/f33 528 1 2024-04-26T04:53:01.014 INFO:tasks.workunit.client.1.smithi078.stdout:9/636: symlink dc/df/d37/lba 0 2024-04-26T04:53:01.022 INFO:tasks.workunit.client.1.smithi078.stdout:8/658: link d0/d19/d52/d35/d44/d6b/fd5 d0/d7a/d83/fd7 0 2024-04-26T04:53:01.024 INFO:tasks.workunit.client.1.smithi078.stdout:1/737: dread d7/d1e/d9e/db/f15 [0,4194304] 0 2024-04-26T04:53:01.027 INFO:tasks.workunit.client.0.smithi077.stdout:4/304: truncate d4/d17/d18/f5d 875664 0 2024-04-26T04:53:01.029 INFO:tasks.workunit.client.0.smithi077.stdout:7/356: mkdir da/d13/d22/d7d/d80 0 2024-04-26T04:53:01.031 INFO:tasks.workunit.client.0.smithi077.stdout:8/367: creat d4/df/d18/d56/f7c x:0 0 0 2024-04-26T04:53:01.033 INFO:tasks.workunit.client.0.smithi077.stdout:9/314: creat d9/dd/d1b/d2b/f72 x:0 0 0 2024-04-26T04:53:01.038 INFO:tasks.workunit.client.0.smithi077.stdout:2/353: mknod d9/dc/d1d/d34/d39/c76 0 2024-04-26T04:53:01.038 INFO:tasks.workunit.client.0.smithi077.stdout:2/354: chown d9/dc/d1d/d1e/c4d 828 1 2024-04-26T04:53:01.040 INFO:tasks.workunit.client.0.smithi077.stdout:0/263: mknod d0/d19/d1e/d59/d5a/c5d 0 2024-04-26T04:53:01.041 INFO:tasks.workunit.client.0.smithi077.stdout:0/264: write d0/d47/f31 [2629655,120929] 0 2024-04-26T04:53:01.044 INFO:tasks.workunit.client.0.smithi077.stdout:7/357: creat da/d6b/d63/f81 x:0 0 0 2024-04-26T04:53:01.050 INFO:tasks.workunit.client.1.smithi078.stdout:4/719: mkdir d0/de/d22/deb/d104 0 2024-04-26T04:53:01.053 INFO:tasks.workunit.client.1.smithi078.stdout:0/715: dwrite d2/d1e/f9e [0,4194304] 0 2024-04-26T04:53:01.053 INFO:tasks.workunit.client.1.smithi078.stdout:6/769: dread d2/de/d27/da9/d2a/d4d/fd1 [0,4194304] 0 2024-04-26T04:53:01.054 INFO:tasks.workunit.client.1.smithi078.stdout:6/770: read - d2/de/d27/da9/d3a/d69/daa/fda zero size 2024-04-26T04:53:01.055 INFO:tasks.workunit.client.0.smithi077.stdout:7/358: mknod da/d13/d22/d34/c82 0 2024-04-26T04:53:01.057 INFO:tasks.workunit.client.0.smithi077.stdout:8/368: symlink d4/df/d22/l7d 0 2024-04-26T04:53:01.057 INFO:tasks.workunit.client.0.smithi077.stdout:6/376: write d8/f37 [1510273,20258] 0 2024-04-26T04:53:01.059 INFO:tasks.workunit.client.0.smithi077.stdout:8/369: dread d4/f42 [0,4194304] 0 2024-04-26T04:53:01.061 INFO:tasks.workunit.client.0.smithi077.stdout:8/370: chown d4/d17/f2c 103 1 2024-04-26T04:53:01.063 INFO:tasks.workunit.client.1.smithi078.stdout:8/659: creat d0/d7a/dbe/fd8 x:0 0 0 2024-04-26T04:53:01.063 INFO:tasks.workunit.client.1.smithi078.stdout:8/660: chown d0/l6f 212 1 2024-04-26T04:53:01.065 INFO:tasks.workunit.client.0.smithi077.stdout:6/377: dwrite d8/d1c/f5c [0,4194304] 0 2024-04-26T04:53:01.067 INFO:tasks.workunit.client.0.smithi077.stdout:7/359: creat da/d6b/d63/d64/f83 x:0 0 0 2024-04-26T04:53:01.068 INFO:tasks.workunit.client.1.smithi078.stdout:2/677: write d8/d11/d14/d69/d8f/d34/dd3/de4/f43 [431202,118301] 0 2024-04-26T04:53:01.073 INFO:tasks.workunit.client.0.smithi077.stdout:9/315: rename d9/dd/f30 to d9/dd/d1b/f73 0 2024-04-26T04:53:01.073 INFO:tasks.workunit.client.0.smithi077.stdout:9/316: readlink d9/dd/l29 0 2024-04-26T04:53:01.074 INFO:tasks.workunit.client.1.smithi078.stdout:9/637: write dc/df/d27/d81/da8/fac [4537622,28194] 0 2024-04-26T04:53:01.075 INFO:tasks.workunit.client.0.smithi077.stdout:9/317: truncate d9/d43/f56 1004437 0 2024-04-26T04:53:01.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/318: chown d9/dd/d1b/f73 490 1 2024-04-26T04:53:01.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/319: chown d9/d3a/c3b 481119 1 2024-04-26T04:53:01.079 INFO:tasks.workunit.client.0.smithi077.stdout:9/320: write d9/dd/d1b/d4e/f6a [19375,2298] 0 2024-04-26T04:53:01.079 INFO:tasks.workunit.client.1.smithi078.stdout:0/716: mknod d2/d9/d31/cef 0 2024-04-26T04:53:01.080 INFO:tasks.workunit.client.1.smithi078.stdout:0/717: write d2/d1e/f56 [1902848,83503] 0 2024-04-26T04:53:01.080 INFO:tasks.workunit.client.1.smithi078.stdout:3/719: write d16/d26/d2b/d31/f3d [3188671,51070] 0 2024-04-26T04:53:01.082 INFO:tasks.workunit.client.0.smithi077.stdout:2/355: dwrite d9/dc/d1d/d1e/f55 [0,4194304] 0 2024-04-26T04:53:01.083 INFO:tasks.workunit.client.0.smithi077.stdout:2/356: dread - d9/f72 zero size 2024-04-26T04:53:01.084 INFO:tasks.workunit.client.0.smithi077.stdout:2/357: stat d9/dc/d1d/d34/d39/d63/l48 0 2024-04-26T04:53:01.085 INFO:tasks.workunit.client.0.smithi077.stdout:6/378: write f4 [3409690,38858] 0 2024-04-26T04:53:01.086 INFO:tasks.workunit.client.1.smithi078.stdout:7/666: link d9/da/d11/d13/d1a/c8c d9/da/d33/d4b/d4c/cda 0 2024-04-26T04:53:01.086 INFO:tasks.workunit.client.0.smithi077.stdout:6/379: dread - d8/f6d zero size 2024-04-26T04:53:01.087 INFO:tasks.workunit.client.0.smithi077.stdout:6/380: write d8/d55/f76 [129231,57146] 0 2024-04-26T04:53:01.089 INFO:tasks.workunit.client.0.smithi077.stdout:6/381: truncate d8/d1c/f4f 4840185 0 2024-04-26T04:53:01.091 INFO:tasks.workunit.client.0.smithi077.stdout:6/382: dread d8/d52/f6e [0,4194304] 0 2024-04-26T04:53:01.092 INFO:tasks.workunit.client.1.smithi078.stdout:1/738: dwrite d7/f57 [0,4194304] 0 2024-04-26T04:53:01.093 INFO:tasks.workunit.client.1.smithi078.stdout:1/739: chown d7/d1e/d9e/db/d37 0 1 2024-04-26T04:53:01.094 INFO:tasks.workunit.client.0.smithi077.stdout:8/371: write d4/df/d22/f43 [500729,13991] 0 2024-04-26T04:53:01.095 INFO:tasks.workunit.client.1.smithi078.stdout:1/740: write d7/d1e/d9e/dd/d20/f50 [3631931,38632] 0 2024-04-26T04:53:01.095 INFO:tasks.workunit.client.0.smithi077.stdout:6/383: dread d8/d2a/f66 [0,4194304] 0 2024-04-26T04:53:01.096 INFO:tasks.workunit.client.1.smithi078.stdout:1/741: fdatasync d7/d1e/d9e/db/d49/fb7 0 2024-04-26T04:53:01.096 INFO:tasks.workunit.client.0.smithi077.stdout:6/384: write d8/d52/f70 [855635,24608] 0 2024-04-26T04:53:01.097 INFO:tasks.workunit.client.1.smithi078.stdout:1/742: write d7/d1e/d9e/d4c/d3d/f91 [3332005,6485] 0 2024-04-26T04:53:01.098 INFO:tasks.workunit.client.1.smithi078.stdout:4/720: mkdir d0/de/d22/deb/d104/d105 0 2024-04-26T04:53:01.098 INFO:tasks.workunit.client.0.smithi077.stdout:0/265: link d0/d2c/l37 d0/d19/l5e 0 2024-04-26T04:53:01.099 INFO:tasks.workunit.client.0.smithi077.stdout:0/266: stat d0/d19/f35 0 2024-04-26T04:53:01.100 INFO:tasks.workunit.client.0.smithi077.stdout:2/358: fdatasync d9/f12 0 2024-04-26T04:53:01.100 INFO:tasks.workunit.client.0.smithi077.stdout:0/267: stat d0/d28 0 2024-04-26T04:53:01.103 INFO:tasks.workunit.client.0.smithi077.stdout:7/360: link da/d13/d22/f46 da/d15/d42/f84 0 2024-04-26T04:53:01.107 INFO:tasks.workunit.client.1.smithi078.stdout:3/720: unlink d16/d26/d2b/d31/d71/d8a/d98/fa1 0 2024-04-26T04:53:01.107 INFO:tasks.workunit.client.0.smithi077.stdout:2/359: dwrite d9/dc/d1d/d34/f61 [0,4194304] 0 2024-04-26T04:53:01.108 INFO:tasks.workunit.client.1.smithi078.stdout:3/721: chown d16/d26/d2b/d31/d71/d8a/dbc/fe7 984 1 2024-04-26T04:53:01.111 INFO:tasks.workunit.client.1.smithi078.stdout:6/771: truncate d2/de/d27/f55 538647 0 2024-04-26T04:53:01.112 INFO:tasks.workunit.client.0.smithi077.stdout:8/372: creat d4/df/d6f/f7e x:0 0 0 2024-04-26T04:53:01.114 INFO:tasks.workunit.client.0.smithi077.stdout:8/373: truncate d4/d17/d35/d5b/f63 5074790 0 2024-04-26T04:53:01.115 INFO:tasks.workunit.client.0.smithi077.stdout:8/374: fdatasync d4/df/d6f/f7e 0 2024-04-26T04:53:01.119 INFO:tasks.workunit.client.1.smithi078.stdout:5/645: link d7/d11/f47 d7/d21/d2f/db6/fd9 0 2024-04-26T04:53:01.126 INFO:tasks.workunit.client.1.smithi078.stdout:7/667: write d9/da/d11/d13/d1a/f40 [900823,97604] 0 2024-04-26T04:53:01.127 INFO:tasks.workunit.client.0.smithi077.stdout:6/385: write d8/d52/d53/d5e/f69 [170161,93169] 0 2024-04-26T04:53:01.127 INFO:tasks.workunit.client.0.smithi077.stdout:7/361: creat da/d13/d22/d7d/f85 x:0 0 0 2024-04-26T04:53:01.128 INFO:tasks.workunit.client.0.smithi077.stdout:6/386: chown d8/d2a/d57/f77 78204 1 2024-04-26T04:53:01.128 INFO:tasks.workunit.client.0.smithi077.stdout:6/387: chown f4 15 1 2024-04-26T04:53:01.129 INFO:tasks.workunit.client.0.smithi077.stdout:2/360: mkdir d9/dc/d1d/d1e/d77 0 2024-04-26T04:53:01.130 INFO:tasks.workunit.client.1.smithi078.stdout:9/638: readlink dc/df/d37/la6 0 2024-04-26T04:53:01.131 INFO:tasks.workunit.client.0.smithi077.stdout:2/361: chown d9/d15/d1a/d2b/f46 6 1 2024-04-26T04:53:01.132 INFO:tasks.workunit.client.1.smithi078.stdout:4/721: symlink d0/de/d22/d27/d7b/l106 0 2024-04-26T04:53:01.133 INFO:tasks.workunit.client.0.smithi077.stdout:0/268: write d0/f3 [7159310,109502] 0 2024-04-26T04:53:01.133 INFO:tasks.workunit.client.0.smithi077.stdout:2/362: read d9/f10 [2078749,42147] 0 2024-04-26T04:53:01.133 INFO:tasks.workunit.client.1.smithi078.stdout:4/722: chown d0/d1d/de1/d20/d31/d47/db2/f5b 945641791 1 2024-04-26T04:53:01.133 INFO:tasks.workunit.client.1.smithi078.stdout:4/723: read - d0/d1d/de1/d20/d31/d47/db2/ff9 zero size 2024-04-26T04:53:01.134 INFO:tasks.workunit.client.1.smithi078.stdout:4/724: dread - d0/d1d/db4/fd3 zero size 2024-04-26T04:53:01.135 INFO:tasks.workunit.client.0.smithi077.stdout:2/363: fdatasync d9/dc/d1d/d34/f61 0 2024-04-26T04:53:01.136 INFO:tasks.workunit.client.0.smithi077.stdout:8/375: creat d4/d17/d35/d5b/f7f x:0 0 0 2024-04-26T04:53:01.136 INFO:tasks.workunit.client.0.smithi077.stdout:2/364: read - d9/f3d zero size 2024-04-26T04:53:01.137 INFO:tasks.workunit.client.0.smithi077.stdout:7/362: dwrite da/d6b/d63/f81 [0,4194304] 0 2024-04-26T04:53:01.137 INFO:tasks.workunit.client.0.smithi077.stdout:8/376: chown d4/f5 4 1 2024-04-26T04:53:01.138 INFO:tasks.workunit.client.1.smithi078.stdout:3/722: truncate d16/d26/d2e/d74/f76 2680754 0 2024-04-26T04:53:01.139 INFO:tasks.workunit.client.1.smithi078.stdout:8/661: creat d0/d19/d25/d36/d58/d5b/d61/fd9 x:0 0 0 2024-04-26T04:53:01.141 INFO:tasks.workunit.client.0.smithi077.stdout:9/321: rename d9/dd/d28/c40 to d9/dd/d28/c74 0 2024-04-26T04:53:01.142 INFO:tasks.workunit.client.1.smithi078.stdout:2/678: creat d8/d11/d14/d69/d8f/ff0 x:0 0 0 2024-04-26T04:53:01.144 INFO:tasks.workunit.client.1.smithi078.stdout:7/668: creat d9/da/d33/fdb x:0 0 0 2024-04-26T04:53:01.145 INFO:tasks.workunit.client.0.smithi077.stdout:0/269: stat d0/d19/l5e 0 2024-04-26T04:53:01.146 INFO:tasks.workunit.client.1.smithi078.stdout:1/743: mkdir d7/d1e/d9e/dd/df0 0 2024-04-26T04:53:01.147 INFO:tasks.workunit.client.0.smithi077.stdout:3/278: sync 2024-04-26T04:53:01.147 INFO:tasks.workunit.client.0.smithi077.stdout:5/369: sync 2024-04-26T04:53:01.151 INFO:tasks.workunit.client.0.smithi077.stdout:7/363: dread da/d15/f78 [0,4194304] 0 2024-04-26T04:53:01.151 INFO:tasks.workunit.client.0.smithi077.stdout:2/365: mkdir d9/dc/d5f/d78 0 2024-04-26T04:53:01.151 INFO:tasks.workunit.client.0.smithi077.stdout:7/364: readlink da/d13/l4d 0 2024-04-26T04:53:01.152 INFO:tasks.workunit.client.0.smithi077.stdout:2/366: chown d9/f17 1878574 1 2024-04-26T04:53:01.155 INFO:tasks.workunit.client.0.smithi077.stdout:6/388: creat d8/d52/d53/d5e/d6c/f84 x:0 0 0 2024-04-26T04:53:01.156 INFO:tasks.workunit.client.0.smithi077.stdout:6/389: write d8/d2a/d57/f7d [173992,104044] 0 2024-04-26T04:53:01.158 INFO:tasks.workunit.client.1.smithi078.stdout:8/662: chown d0/d19/d25/d36/d58/f70 53704859 1 2024-04-26T04:53:01.160 INFO:tasks.workunit.client.0.smithi077.stdout:0/270: write d0/f4 [138771,72775] 0 2024-04-26T04:53:01.161 INFO:tasks.workunit.client.1.smithi078.stdout:2/679: creat d8/d11/d18/d5d/d8b/db0/db4/ff1 x:0 0 0 2024-04-26T04:53:01.162 INFO:tasks.workunit.client.1.smithi078.stdout:8/663: read d0/d19/d52/d35/f8f [215980,105717] 0 2024-04-26T04:53:01.163 INFO:tasks.workunit.client.0.smithi077.stdout:5/370: creat d1/d42/d62/f89 x:0 0 0 2024-04-26T04:53:01.164 INFO:tasks.workunit.client.0.smithi077.stdout:5/371: truncate d1/d12/f60 914076 0 2024-04-26T04:53:01.164 INFO:tasks.workunit.client.0.smithi077.stdout:3/279: truncate d0/f38 844689 0 2024-04-26T04:53:01.166 INFO:tasks.workunit.client.1.smithi078.stdout:1/744: write d7/d1e/d9e/db/d49/f6c [1109068,87695] 0 2024-04-26T04:53:01.166 INFO:tasks.workunit.client.0.smithi077.stdout:5/372: write d1/d30/d33/f76 [721519,54612] 0 2024-04-26T04:53:01.166 INFO:tasks.workunit.client.0.smithi077.stdout:3/280: write d0/d9/d32/f36 [1741732,120199] 0 2024-04-26T04:53:01.167 INFO:tasks.workunit.client.1.smithi078.stdout:9/639: mkdir dc/df/d4b/d22/dbb 0 2024-04-26T04:53:01.167 INFO:tasks.workunit.client.0.smithi077.stdout:3/281: chown d0/d9/f2b 21345202 1 2024-04-26T04:53:01.167 INFO:tasks.workunit.client.1.smithi078.stdout:0/718: link d2/d1e/fe6 d2/d9/d18/d7d/da9/ff0 0 2024-04-26T04:53:01.168 INFO:tasks.workunit.client.1.smithi078.stdout:2/680: dwrite d8/fc6 [0,4194304] 0 2024-04-26T04:53:01.170 INFO:tasks.workunit.client.0.smithi077.stdout:7/365: unlink da/d15/d1a/f49 0 2024-04-26T04:53:01.173 INFO:tasks.workunit.client.0.smithi077.stdout:2/367: dread d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:53:01.173 INFO:tasks.workunit.client.1.smithi078.stdout:0/719: write d2/d9/d18/d7d/fcf [3996804,21430] 0 2024-04-26T04:53:01.173 INFO:tasks.workunit.client.1.smithi078.stdout:3/723: creat d16/d26/d2e/d5f/d49/dcc/de2/f101 x:0 0 0 2024-04-26T04:53:01.174 INFO:tasks.workunit.client.0.smithi077.stdout:7/366: write da/d15/d42/f50 [668502,129108] 0 2024-04-26T04:53:01.174 INFO:tasks.workunit.client.0.smithi077.stdout:8/377: mkdir d4/df/d22/d80 0 2024-04-26T04:53:01.175 INFO:tasks.workunit.client.1.smithi078.stdout:7/669: dread d9/da/d11/d13/f91 [0,4194304] 0 2024-04-26T04:53:01.176 INFO:tasks.workunit.client.0.smithi077.stdout:5/373: dwrite d1/d42/d62/d6b/f77 [0,4194304] 0 2024-04-26T04:53:01.176 INFO:tasks.workunit.client.1.smithi078.stdout:5/646: rename d7/d18/d32/d37/f41 to d7/d18/fda 0 2024-04-26T04:53:01.177 INFO:tasks.workunit.client.1.smithi078.stdout:2/681: dwrite d8/d11/d14/fbf [4194304,4194304] 0 2024-04-26T04:53:01.179 INFO:tasks.workunit.client.0.smithi077.stdout:6/390: chown d8/d2a/d57/l6b 69969223 1 2024-04-26T04:53:01.180 INFO:tasks.workunit.client.0.smithi077.stdout:0/271: read d0/d47/f30 [149464,51145] 0 2024-04-26T04:53:01.187 INFO:tasks.workunit.client.0.smithi077.stdout:5/374: dwrite d1/d12/d21/d25/d31/d3c/f4e [0,4194304] 0 2024-04-26T04:53:01.188 INFO:tasks.workunit.client.0.smithi077.stdout:3/282: rename d0/d9/d32/f28 to d0/d7/d1f/d4d/f56 0 2024-04-26T04:53:01.189 INFO:tasks.workunit.client.1.smithi078.stdout:9/640: read dc/df/d4b/d22/d2a/d57/f73 [3432969,51089] 0 2024-04-26T04:53:01.196 INFO:tasks.workunit.client.0.smithi077.stdout:8/378: symlink d4/df/d22/d2d/l81 0 2024-04-26T04:53:01.197 INFO:tasks.workunit.client.1.smithi078.stdout:0/720: fdatasync d2/f6 0 2024-04-26T04:53:01.198 INFO:tasks.workunit.client.0.smithi077.stdout:8/379: read d4/d17/d35/f52 [3971407,9383] 0 2024-04-26T04:53:01.201 INFO:tasks.workunit.client.0.smithi077.stdout:7/367: read da/d15/f44 [645302,60065] 0 2024-04-26T04:53:01.204 INFO:tasks.workunit.client.1.smithi078.stdout:5/647: creat d7/d18/d1c/d5a/fdb x:0 0 0 2024-04-26T04:53:01.205 INFO:tasks.workunit.client.0.smithi077.stdout:6/391: creat d8/d55/f85 x:0 0 0 2024-04-26T04:53:01.207 INFO:tasks.workunit.client.1.smithi078.stdout:8/664: creat d0/d19/d25/d32/d8a/d92/da4/da7/fda x:0 0 0 2024-04-26T04:53:01.207 INFO:tasks.workunit.client.0.smithi077.stdout:0/272: mknod d0/d2c/d4e/c5f 0 2024-04-26T04:53:01.208 INFO:tasks.workunit.client.1.smithi078.stdout:8/665: write d0/fae [2763336,14053] 0 2024-04-26T04:53:01.210 INFO:tasks.workunit.client.1.smithi078.stdout:9/641: mkdir dc/df/d27/dbc 0 2024-04-26T04:53:01.211 INFO:tasks.workunit.client.1.smithi078.stdout:4/725: getdents d0/d1d/de1/d20/d31/d47/db2/d4c/d8e 0 2024-04-26T04:53:01.212 INFO:tasks.workunit.client.1.smithi078.stdout:8/666: dread d0/d19/d25/d32/f76 [0,4194304] 0 2024-04-26T04:53:01.213 INFO:tasks.workunit.client.1.smithi078.stdout:3/724: write d16/d26/d2b/d31/d71/f8c [512739,81715] 0 2024-04-26T04:53:01.214 INFO:tasks.workunit.client.0.smithi077.stdout:2/368: rename d9/d15/d1a to d9/dc/d1d/d34/d39/d63/d79 0 2024-04-26T04:53:01.216 INFO:tasks.workunit.client.0.smithi077.stdout:2/369: chown d9/dc/d1d/d34/d39/d63/d6a/c70 708 1 2024-04-26T04:53:01.216 INFO:tasks.workunit.client.0.smithi077.stdout:3/283: truncate d0/d7/d15/f22 694025 0 2024-04-26T04:53:01.218 INFO:tasks.workunit.client.0.smithi077.stdout:2/370: write d9/dc/d1d/d34/d39/d63/d79/f3f [598547,70832] 0 2024-04-26T04:53:01.220 INFO:tasks.workunit.client.1.smithi078.stdout:5/648: unlink d7/d48/d57/d65/f53 0 2024-04-26T04:53:01.220 INFO:tasks.workunit.client.1.smithi078.stdout:5/649: chown d7/d48/d57 36 1 2024-04-26T04:53:01.221 INFO:tasks.workunit.client.0.smithi077.stdout:9/322: getdents d9/d3a 0 2024-04-26T04:53:01.223 INFO:tasks.workunit.client.0.smithi077.stdout:8/380: mknod d4/df/d22/d2d/d2b/d33/c82 0 2024-04-26T04:53:01.224 INFO:tasks.workunit.client.0.smithi077.stdout:8/381: chown d4/d17/c5c 655847 1 2024-04-26T04:53:01.226 INFO:tasks.workunit.client.1.smithi078.stdout:9/642: creat dc/df/d4b/d22/d2d/d98/fbd x:0 0 0 2024-04-26T04:53:01.229 INFO:tasks.workunit.client.1.smithi078.stdout:0/721: dwrite d2/f15 [0,4194304] 0 2024-04-26T04:53:01.230 INFO:tasks.workunit.client.1.smithi078.stdout:8/667: creat d0/d19/d52/d35/d44/fdb x:0 0 0 2024-04-26T04:53:01.230 INFO:tasks.workunit.client.0.smithi077.stdout:0/273: chown d0/d47/f16 2 1 2024-04-26T04:53:01.232 INFO:tasks.workunit.client.0.smithi077.stdout:5/375: fdatasync d1/d42/d62/f6f 0 2024-04-26T04:53:01.232 INFO:tasks.workunit.client.1.smithi078.stdout:9/643: dread dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:53:01.233 INFO:tasks.workunit.client.1.smithi078.stdout:9/644: chown dc/f26 6 1 2024-04-26T04:53:01.234 INFO:tasks.workunit.client.1.smithi078.stdout:3/725: creat d16/d26/d2e/d5f/d49/dad/db0/db7/f102 x:0 0 0 2024-04-26T04:53:01.234 INFO:tasks.workunit.client.1.smithi078.stdout:9/645: dread - dc/df/d4b/d22/d2a/d57/d69/fa0 zero size 2024-04-26T04:53:01.234 INFO:tasks.workunit.client.1.smithi078.stdout:3/726: chown d16/d26/d2e/d5f/ff5 0 1 2024-04-26T04:53:01.235 INFO:tasks.workunit.client.0.smithi077.stdout:3/284: unlink d0/d9/d32/cf 0 2024-04-26T04:53:01.236 INFO:tasks.workunit.client.1.smithi078.stdout:3/727: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f5d [2100179,107302] 0 2024-04-26T04:53:01.236 INFO:tasks.workunit.client.1.smithi078.stdout:6/772: sync 2024-04-26T04:53:01.239 INFO:tasks.workunit.client.1.smithi078.stdout:2/682: creat d8/ff2 x:0 0 0 2024-04-26T04:53:01.242 INFO:tasks.workunit.client.0.smithi077.stdout:8/382: symlink d4/df/l83 0 2024-04-26T04:53:01.243 INFO:tasks.workunit.client.1.smithi078.stdout:1/745: getdents d7/d1e/d9e/db/d49/d93 0 2024-04-26T04:53:01.245 INFO:tasks.workunit.client.1.smithi078.stdout:1/746: write d7/d1e/d9e/dd/d20/fbc [42740,75509] 0 2024-04-26T04:53:01.246 INFO:tasks.workunit.client.0.smithi077.stdout:2/371: write d9/dc/d1d/d34/d39/d63/d79/f22 [78271,39547] 0 2024-04-26T04:53:01.246 INFO:tasks.workunit.client.1.smithi078.stdout:1/747: dread d7/d1e/d9e/db/d37/fa9 [0,4194304] 0 2024-04-26T04:53:01.247 INFO:tasks.workunit.client.0.smithi077.stdout:2/372: write d9/d15/f37 [3471478,124770] 0 2024-04-26T04:53:01.248 INFO:tasks.workunit.client.0.smithi077.stdout:6/392: mkdir d8/d2a/d86 0 2024-04-26T04:53:01.252 INFO:tasks.workunit.client.0.smithi077.stdout:5/376: rename d1/d8/c65 to d1/d12/d87/c8a 0 2024-04-26T04:53:01.254 INFO:tasks.workunit.client.0.smithi077.stdout:2/373: dwrite d9/dc/d1d/d34/f69 [0,4194304] 0 2024-04-26T04:53:01.257 INFO:tasks.workunit.client.0.smithi077.stdout:9/323: mknod d9/dd/d1b/c75 0 2024-04-26T04:53:01.259 INFO:tasks.workunit.client.1.smithi078.stdout:2/683: dread d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/d91/fb2 [0,4194304] 0 2024-04-26T04:53:01.259 INFO:tasks.workunit.client.0.smithi077.stdout:9/324: write d9/f11 [6668045,93870] 0 2024-04-26T04:53:01.259 INFO:tasks.workunit.client.1.smithi078.stdout:6/773: creat d2/d1a/d5c/ff0 x:0 0 0 2024-04-26T04:53:01.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/383: mknod d4/df/d22/d2d/d2b/d33/c84 0 2024-04-26T04:53:01.262 INFO:tasks.workunit.client.0.smithi077.stdout:4/305: sync 2024-04-26T04:53:01.263 INFO:tasks.workunit.client.1.smithi078.stdout:3/728: write d16/d26/f29 [97054,130843] 0 2024-04-26T04:53:01.263 INFO:tasks.workunit.client.0.smithi077.stdout:1/377: sync 2024-04-26T04:53:01.263 INFO:tasks.workunit.client.0.smithi077.stdout:8/384: fdatasync d4/df/d22/d2d/d2b/f7b 0 2024-04-26T04:53:01.264 INFO:tasks.workunit.client.1.smithi078.stdout:7/670: sync 2024-04-26T04:53:01.265 INFO:tasks.workunit.client.0.smithi077.stdout:3/285: mknod d0/d9/c57 0 2024-04-26T04:53:01.269 INFO:tasks.workunit.client.0.smithi077.stdout:4/306: dwrite d4/de/d15/d1f/d35/f52 [0,4194304] 0 2024-04-26T04:53:01.273 INFO:tasks.workunit.client.0.smithi077.stdout:6/393: creat d8/d52/f87 x:0 0 0 2024-04-26T04:53:01.274 INFO:tasks.workunit.client.0.smithi077.stdout:6/394: write d8/d52/d53/f80 [240735,48994] 0 2024-04-26T04:53:01.274 INFO:tasks.workunit.client.1.smithi078.stdout:5/650: truncate d7/d18/d1c/d5a/f9d 2261355 0 2024-04-26T04:53:01.274 INFO:tasks.workunit.client.0.smithi077.stdout:6/395: dread - d8/d2a/f63 zero size 2024-04-26T04:53:01.275 INFO:tasks.workunit.client.0.smithi077.stdout:6/396: read - d8/d52/d53/f72 zero size 2024-04-26T04:53:01.275 INFO:tasks.workunit.client.1.smithi078.stdout:5/651: chown d7/d48/d57/d65/d70/d72/fd8 48 1 2024-04-26T04:53:01.276 INFO:tasks.workunit.client.0.smithi077.stdout:0/274: symlink d0/d28/d40/l60 0 2024-04-26T04:53:01.277 INFO:tasks.workunit.client.1.smithi078.stdout:5/652: write d7/d48/d57/fab [797565,37060] 0 2024-04-26T04:53:01.277 INFO:tasks.workunit.client.0.smithi077.stdout:0/275: write d0/d28/d39/f5c [54810,129506] 0 2024-04-26T04:53:01.277 INFO:tasks.workunit.client.1.smithi078.stdout:4/726: creat d0/d1d/de1/f107 x:0 0 0 2024-04-26T04:53:01.279 INFO:tasks.workunit.client.1.smithi078.stdout:1/748: mknod d7/d1e/cf1 0 2024-04-26T04:53:01.281 INFO:tasks.workunit.client.1.smithi078.stdout:8/668: mknod d0/d19/d52/d5c/da2/dd3/cdc 0 2024-04-26T04:53:01.282 INFO:tasks.workunit.client.0.smithi077.stdout:5/377: creat d1/d12/d28/d38/f8b x:0 0 0 2024-04-26T04:53:01.283 INFO:tasks.workunit.client.0.smithi077.stdout:5/378: stat d1/d12/d21/f17 0 2024-04-26T04:53:01.283 INFO:tasks.workunit.client.1.smithi078.stdout:9/646: mkdir dc/df/d4b/d22/dbe 0 2024-04-26T04:53:01.284 INFO:tasks.workunit.client.1.smithi078.stdout:0/722: dread d2/d9/d31/d4a/fa6 [0,4194304] 0 2024-04-26T04:53:01.288 INFO:tasks.workunit.client.0.smithi077.stdout:9/325: mknod d9/dd/d3d/c76 0 2024-04-26T04:53:01.290 INFO:tasks.workunit.client.1.smithi078.stdout:6/774: rename d2/de/d27/da9/d3a/d69/daa/ld7 to d2/de/d27/da9/da/d36/lf1 0 2024-04-26T04:53:01.290 INFO:tasks.workunit.client.1.smithi078.stdout:6/775: chown d2/de/f82 45799590 1 2024-04-26T04:53:01.300 INFO:tasks.workunit.client.0.smithi077.stdout:7/368: getdents da/d13/d22/d7d 0 2024-04-26T04:53:01.303 INFO:tasks.workunit.client.0.smithi077.stdout:4/307: mknod d4/de/d15/d1f/d25/c68 0 2024-04-26T04:53:01.309 INFO:tasks.workunit.client.1.smithi078.stdout:8/669: unlink d0/d19/d52/d35/d44/ca6 0 2024-04-26T04:53:01.310 INFO:tasks.workunit.client.1.smithi078.stdout:2/684: dwrite d8/d11/d6e/f80 [0,4194304] 0 2024-04-26T04:53:01.310 INFO:tasks.workunit.client.1.smithi078.stdout:8/670: chown d0/d19/d25/d36/d58 878118808 1 2024-04-26T04:53:01.312 INFO:tasks.workunit.client.1.smithi078.stdout:8/671: chown d0/d19/d52/d95/d9d/lb8 12185 1 2024-04-26T04:53:01.319 INFO:tasks.workunit.client.1.smithi078.stdout:6/776: write d2/de/d27/da9/d3a/d69/daa/dba/fec [998234,121107] 0 2024-04-26T04:53:01.320 INFO:tasks.workunit.client.1.smithi078.stdout:7/671: creat d9/da/d33/d4b/d4c/d5b/dc3/fdc x:0 0 0 2024-04-26T04:53:01.322 INFO:tasks.workunit.client.1.smithi078.stdout:1/749: write d7/d1e/d9e/db/d47/f73 [1186202,111420] 0 2024-04-26T04:53:01.327 INFO:tasks.workunit.client.1.smithi078.stdout:6/777: mkdir d2/de/d27/da9/d2a/dbf/d5f/df2 0 2024-04-26T04:53:01.329 INFO:tasks.workunit.client.1.smithi078.stdout:7/672: dwrite d9/da/d33/d4b/d4c/d6c/f81 [4194304,4194304] 0 2024-04-26T04:53:01.330 INFO:tasks.workunit.client.1.smithi078.stdout:3/729: sync 2024-04-26T04:53:01.330 INFO:tasks.workunit.client.1.smithi078.stdout:8/672: symlink d0/d19/d52/d35/ldd 0 2024-04-26T04:53:01.332 INFO:tasks.workunit.client.1.smithi078.stdout:1/750: mkdir d7/d1e/d9e/db/d37/d58/d5c/df2 0 2024-04-26T04:53:01.332 INFO:tasks.workunit.client.1.smithi078.stdout:3/730: readlink d16/d26/d2b/l6a 0 2024-04-26T04:53:01.333 INFO:tasks.workunit.client.1.smithi078.stdout:9/647: dwrite dc/f3d [0,4194304] 0 2024-04-26T04:53:01.337 INFO:tasks.workunit.client.1.smithi078.stdout:1/751: dread - d7/d1e/d9e/db/d49/fb4 zero size 2024-04-26T04:53:01.337 INFO:tasks.workunit.client.1.smithi078.stdout:6/778: getdents d2/de/d27/da9/d2a/dbf 0 2024-04-26T04:53:01.338 INFO:tasks.workunit.client.1.smithi078.stdout:9/648: write dc/df/d4b/d22/d2a/d92/fa3 [4877207,122356] 0 2024-04-26T04:53:01.339 INFO:tasks.workunit.client.1.smithi078.stdout:8/673: creat d0/d19/d25/d36/d58/d5b/d61/fde x:0 0 0 2024-04-26T04:53:01.340 INFO:tasks.workunit.client.1.smithi078.stdout:4/727: write d0/d1d/de1/d20/f1b [2301684,23650] 0 2024-04-26T04:53:01.341 INFO:tasks.workunit.client.1.smithi078.stdout:9/649: fdatasync dc/df/d27/d81/da8/fac 0 2024-04-26T04:53:01.341 INFO:tasks.workunit.client.1.smithi078.stdout:7/673: symlink d9/da/d11/d13/d1a/d52/d5a/d41/d68/ldd 0 2024-04-26T04:53:01.341 INFO:tasks.workunit.client.1.smithi078.stdout:0/723: truncate d2/d9/d31/d73/f90 3774090 0 2024-04-26T04:53:01.342 INFO:tasks.workunit.client.1.smithi078.stdout:4/728: stat d0/d1d/de1/d20/d31/d47/db2/d4c/df6 0 2024-04-26T04:53:01.344 INFO:tasks.workunit.client.1.smithi078.stdout:5/653: chown d7/d18/d1c/d5a/f9d 867086 1 2024-04-26T04:53:01.347 INFO:tasks.workunit.client.1.smithi078.stdout:2/685: write d8/d11/d14/f67 [2837206,27375] 0 2024-04-26T04:53:01.349 INFO:tasks.workunit.client.1.smithi078.stdout:4/729: dwrite d0/de/d22/d71/fed [0,4194304] 0 2024-04-26T04:53:01.350 INFO:tasks.workunit.client.1.smithi078.stdout:7/674: unlink d9/f71 0 2024-04-26T04:53:01.357 INFO:tasks.workunit.client.1.smithi078.stdout:5/654: truncate d7/d48/f6f 307479 0 2024-04-26T04:53:01.357 INFO:tasks.workunit.client.1.smithi078.stdout:5/655: chown d7/d48/d57/d65/d46/f4f 2 1 2024-04-26T04:53:01.358 INFO:tasks.workunit.client.1.smithi078.stdout:2/686: unlink d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/d91/fb2 0 2024-04-26T04:53:01.361 INFO:tasks.workunit.client.1.smithi078.stdout:3/731: link d16/d26/d2b/d31/d71/d8a/cab d16/d26/d2e/d5f/d49/dad/c103 0 2024-04-26T04:53:01.361 INFO:tasks.workunit.client.1.smithi078.stdout:4/730: read d0/d1d/de1/d20/d31/d47/db2/d44/d69/fa4 [1562729,127852] 0 2024-04-26T04:53:01.362 INFO:tasks.workunit.client.1.smithi078.stdout:5/656: getdents d7/d48/d57/d65/d46/dce 0 2024-04-26T04:53:01.363 INFO:tasks.workunit.client.1.smithi078.stdout:2/687: symlink d8/d11/d14/lf3 0 2024-04-26T04:53:01.363 INFO:tasks.workunit.client.1.smithi078.stdout:4/731: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/l67 43 1 2024-04-26T04:53:01.366 INFO:tasks.workunit.client.1.smithi078.stdout:4/732: read - d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fe6 zero size 2024-04-26T04:53:01.367 INFO:tasks.workunit.client.1.smithi078.stdout:2/688: unlink d8/d11/d14/f8d 0 2024-04-26T04:53:01.369 INFO:tasks.workunit.client.1.smithi078.stdout:5/657: dwrite d7/da/d8a/fd0 [0,4194304] 0 2024-04-26T04:53:01.370 INFO:tasks.workunit.client.1.smithi078.stdout:3/732: link d16/d26/d2e/d74/l38 d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/l104 0 2024-04-26T04:53:01.372 INFO:tasks.workunit.client.1.smithi078.stdout:5/658: mknod d7/d21/d2f/cdc 0 2024-04-26T04:53:01.373 INFO:tasks.workunit.client.1.smithi078.stdout:5/659: truncate d7/d18/d32/d84/fd3 820542 0 2024-04-26T04:53:01.373 INFO:tasks.workunit.client.1.smithi078.stdout:4/733: creat d0/f108 x:0 0 0 2024-04-26T04:53:01.374 INFO:tasks.workunit.client.1.smithi078.stdout:1/752: dread d7/d1e/d9e/db/d37/d58/d5c/dc0/fa1 [0,4194304] 0 2024-04-26T04:53:01.374 INFO:tasks.workunit.client.1.smithi078.stdout:2/689: truncate d8/d11/d14/f40 2284066 0 2024-04-26T04:53:01.378 INFO:tasks.workunit.client.1.smithi078.stdout:5/660: read d7/d48/d57/fcf [606400,103855] 0 2024-04-26T04:53:01.383 INFO:tasks.workunit.client.1.smithi078.stdout:5/661: fdatasync d7/d18/f2e 0 2024-04-26T04:53:01.384 INFO:tasks.workunit.client.1.smithi078.stdout:8/674: dread d0/d19/d25/d32/d8a/db1/f88 [0,4194304] 0 2024-04-26T04:53:01.385 INFO:tasks.workunit.client.1.smithi078.stdout:1/753: mknod d7/d1e/d9e/db/d37/d58/d5c/d68/de8/cf3 0 2024-04-26T04:53:01.386 INFO:tasks.workunit.client.1.smithi078.stdout:5/662: chown d7/da/d1e/d91/dd6/fd7 2617051 1 2024-04-26T04:53:01.387 INFO:tasks.workunit.client.1.smithi078.stdout:2/690: link d8/d11/d14/d69/d8f/ff0 d8/d11/d6e/ff4 0 2024-04-26T04:53:01.387 INFO:tasks.workunit.client.1.smithi078.stdout:6/779: dwrite d2/de/d27/da9/d3a/d42/f5a [0,4194304] 0 2024-04-26T04:53:01.388 INFO:tasks.workunit.client.0.smithi077.stdout:5/379: unlink d1/d30/c43 0 2024-04-26T04:53:01.388 INFO:tasks.workunit.client.0.smithi077.stdout:9/326: mknod d9/dd/d1b/d2b/d32/d36/c77 0 2024-04-26T04:53:01.388 INFO:tasks.workunit.client.0.smithi077.stdout:8/385: mknod d4/df/d22/d80/c85 0 2024-04-26T04:53:01.389 INFO:tasks.workunit.client.0.smithi077.stdout:3/286: mknod d0/c58 0 2024-04-26T04:53:01.389 INFO:tasks.workunit.client.1.smithi078.stdout:3/733: getdents d16/d26/d2e/d5f/d49/dad 0 2024-04-26T04:53:01.390 INFO:tasks.workunit.client.0.smithi077.stdout:7/369: mknod da/d6b/d63/c86 0 2024-04-26T04:53:01.394 INFO:tasks.workunit.client.1.smithi078.stdout:8/675: rmdir d0/d19/d25/d36/d58 39 2024-04-26T04:53:01.394 INFO:tasks.workunit.client.1.smithi078.stdout:2/691: read d8/d11/f48 [1044997,74811] 0 2024-04-26T04:53:01.394 INFO:tasks.workunit.client.0.smithi077.stdout:2/374: creat d9/dc/f7a x:0 0 0 2024-04-26T04:53:01.395 INFO:tasks.workunit.client.0.smithi077.stdout:2/375: chown d9/dc/d5f/d78 625 1 2024-04-26T04:53:01.396 INFO:tasks.workunit.client.0.smithi077.stdout:0/276: dwrite d0/f15 [0,4194304] 0 2024-04-26T04:53:01.397 INFO:tasks.workunit.client.1.smithi078.stdout:9/650: dread dc/df/d4b/f86 [0,4194304] 0 2024-04-26T04:53:01.397 INFO:tasks.workunit.client.1.smithi078.stdout:0/724: dwrite d2/f57 [4194304,4194304] 0 2024-04-26T04:53:01.398 INFO:tasks.workunit.client.0.smithi077.stdout:0/277: write d0/d47/ff [2721757,5883] 0 2024-04-26T04:53:01.401 INFO:tasks.workunit.client.1.smithi078.stdout:2/692: write d8/d11/d14/f67 [2477645,64482] 0 2024-04-26T04:53:01.403 INFO:tasks.workunit.client.0.smithi077.stdout:5/380: mkdir d1/d8c 0 2024-04-26T04:53:01.403 INFO:tasks.workunit.client.1.smithi078.stdout:0/725: truncate d2/f21 5729234 0 2024-04-26T04:53:01.404 INFO:tasks.workunit.client.1.smithi078.stdout:2/693: fdatasync d8/d53/f7a 0 2024-04-26T04:53:01.405 INFO:tasks.workunit.client.1.smithi078.stdout:6/780: dread d2/de/d27/da9/d2a/dbf/d5f/f9f [0,4194304] 0 2024-04-26T04:53:01.406 INFO:tasks.workunit.client.1.smithi078.stdout:0/726: fdatasync d2/d1e/f56 0 2024-04-26T04:53:01.408 INFO:tasks.workunit.client.0.smithi077.stdout:9/327: mknod d9/dd/d1b/d4e/c78 0 2024-04-26T04:53:01.408 INFO:tasks.workunit.client.1.smithi078.stdout:2/694: readlink d8/d11/l65 0 2024-04-26T04:53:01.408 INFO:tasks.workunit.client.1.smithi078.stdout:5/663: dwrite d7/d21/f4e [0,4194304] 0 2024-04-26T04:53:01.408 INFO:tasks.workunit.client.0.smithi077.stdout:9/328: chown d9/dd/d1b/d2b/d32/c6f 219030594 1 2024-04-26T04:53:01.409 INFO:tasks.workunit.client.0.smithi077.stdout:9/329: chown d9/dd/d28/f33 157791 1 2024-04-26T04:53:01.411 INFO:tasks.workunit.client.1.smithi078.stdout:7/675: dwrite d9/da/d11/d13/d1a/d2d/f3d [0,4194304] 0 2024-04-26T04:53:01.413 INFO:tasks.workunit.client.1.smithi078.stdout:3/734: symlink d16/d26/d2b/d66/l105 0 2024-04-26T04:53:01.414 INFO:tasks.workunit.client.1.smithi078.stdout:7/676: dread - d9/da/d33/fd7 zero size 2024-04-26T04:53:01.414 INFO:tasks.workunit.client.1.smithi078.stdout:2/695: truncate d8/d11/d6e/fdd 469503 0 2024-04-26T04:53:01.416 INFO:tasks.workunit.client.1.smithi078.stdout:3/735: truncate d16/d26/d2b/d31/fe5 248196 0 2024-04-26T04:53:01.420 INFO:tasks.workunit.client.0.smithi077.stdout:8/386: mkdir d4/df/d22/d80/d86 0 2024-04-26T04:53:01.420 INFO:tasks.workunit.client.1.smithi078.stdout:2/696: read d8/d11/d14/d69/d8f/d34/d70/f75 [2458076,2400] 0 2024-04-26T04:53:01.420 INFO:tasks.workunit.client.0.smithi077.stdout:8/387: chown d4/df 163411 1 2024-04-26T04:53:01.421 INFO:tasks.workunit.client.0.smithi077.stdout:8/388: fdatasync d4/df/d22/d2d/d2b/f7b 0 2024-04-26T04:53:01.422 INFO:tasks.workunit.client.0.smithi077.stdout:8/389: fsync d4/d17/f3c 0 2024-04-26T04:53:01.425 INFO:tasks.workunit.client.1.smithi078.stdout:7/677: dwrite d9/da/d11/fad [0,4194304] 0 2024-04-26T04:53:01.425 INFO:tasks.workunit.client.1.smithi078.stdout:8/676: dwrite d0/d19/d25/d32/d8a/db1/f88 [0,4194304] 0 2024-04-26T04:53:01.426 INFO:tasks.workunit.client.1.smithi078.stdout:9/651: creat dc/df/d4b/d22/d2a/d57/fbf x:0 0 0 2024-04-26T04:53:01.428 INFO:tasks.workunit.client.1.smithi078.stdout:9/652: chown dc/df/d27/c41 3621 1 2024-04-26T04:53:01.430 INFO:tasks.workunit.client.0.smithi077.stdout:1/378: getdents d9/d53/d6e 0 2024-04-26T04:53:01.433 INFO:tasks.workunit.client.0.smithi077.stdout:0/278: mknod d0/d19/d1e/d59/c61 0 2024-04-26T04:53:01.433 INFO:tasks.workunit.client.1.smithi078.stdout:6/781: dread d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:53:01.436 INFO:tasks.workunit.client.1.smithi078.stdout:0/727: symlink d2/d9/d18/d7d/da9/dab/dc6/lf1 0 2024-04-26T04:53:01.438 INFO:tasks.workunit.client.0.smithi077.stdout:7/370: write da/d15/f44 [1468315,93605] 0 2024-04-26T04:53:01.439 INFO:tasks.workunit.client.1.smithi078.stdout:4/734: sync 2024-04-26T04:53:01.439 INFO:tasks.workunit.client.1.smithi078.stdout:0/728: dread d2/d39/f9d [0,4194304] 0 2024-04-26T04:53:01.440 INFO:tasks.workunit.client.0.smithi077.stdout:4/308: write d4/f8 [7878260,389] 0 2024-04-26T04:53:01.443 INFO:tasks.workunit.client.0.smithi077.stdout:9/330: rename d9/dd/d28 to d9/d3a/d79 0 2024-04-26T04:53:01.446 INFO:tasks.workunit.client.1.smithi078.stdout:4/735: dwrite d0/d1d/d46/d100/fd1 [0,4194304] 0 2024-04-26T04:53:01.447 INFO:tasks.workunit.client.0.smithi077.stdout:8/390: truncate d4/f10 939 0 2024-04-26T04:53:01.450 INFO:tasks.workunit.client.1.smithi078.stdout:7/678: creat d9/da/d20/fde x:0 0 0 2024-04-26T04:53:01.450 INFO:tasks.workunit.client.1.smithi078.stdout:7/679: readlink d9/da/d33/l38 0 2024-04-26T04:53:01.452 INFO:tasks.workunit.client.0.smithi077.stdout:5/381: chown d1/d12/d21/f34 32522309 1 2024-04-26T04:53:01.453 INFO:tasks.workunit.client.0.smithi077.stdout:5/382: readlink d1/d12/l52 0 2024-04-26T04:53:01.453 INFO:tasks.workunit.client.1.smithi078.stdout:7/680: dread d9/da/d11/fad [0,4194304] 0 2024-04-26T04:53:01.454 INFO:tasks.workunit.client.1.smithi078.stdout:7/681: stat d9/da/d33/d4b/d4c/d5b/d83/fac 0 2024-04-26T04:53:01.455 INFO:tasks.workunit.client.1.smithi078.stdout:1/754: link d7/d1e/d9e/dd/d20/fbc d7/d1e/d9e/db/d37/d58/ff4 0 2024-04-26T04:53:01.457 INFO:tasks.workunit.client.0.smithi077.stdout:1/379: dwrite d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:53:01.457 INFO:tasks.workunit.client.0.smithi077.stdout:7/371: truncate da/d13/d22/f27 546002 0 2024-04-26T04:53:01.458 INFO:tasks.workunit.client.0.smithi077.stdout:7/372: read - da/d13/d22/f58 zero size 2024-04-26T04:53:01.461 INFO:tasks.workunit.client.0.smithi077.stdout:9/331: rmdir d9/d3a 39 2024-04-26T04:53:01.463 INFO:tasks.workunit.client.0.smithi077.stdout:9/332: chown d9/dd/d3d/f59 31 1 2024-04-26T04:53:01.465 INFO:tasks.workunit.client.0.smithi077.stdout:0/279: rename d0/d47/f56 to d0/d19/d1e/d2f/d52/f62 0 2024-04-26T04:53:01.466 INFO:tasks.workunit.client.0.smithi077.stdout:4/309: dwrite d4/de/d15/f62 [4194304,4194304] 0 2024-04-26T04:53:01.467 INFO:tasks.workunit.client.0.smithi077.stdout:0/280: read d0/d19/f2d [450970,100389] 0 2024-04-26T04:53:01.471 INFO:tasks.workunit.client.1.smithi078.stdout:9/653: write dc/df/f9f [552725,90901] 0 2024-04-26T04:53:01.472 INFO:tasks.workunit.client.1.smithi078.stdout:9/654: truncate f1 1041785 0 2024-04-26T04:53:01.472 INFO:tasks.workunit.client.0.smithi077.stdout:5/383: unlink d1/d12/d28/f1c 0 2024-04-26T04:53:01.474 INFO:tasks.workunit.client.0.smithi077.stdout:5/384: dread d1/d12/f60 [0,4194304] 0 2024-04-26T04:53:01.476 INFO:tasks.workunit.client.0.smithi077.stdout:3/287: getdents d0 0 2024-04-26T04:53:01.479 INFO:tasks.workunit.client.0.smithi077.stdout:7/373: creat da/f87 x:0 0 0 2024-04-26T04:53:01.479 INFO:tasks.workunit.client.0.smithi077.stdout:7/374: stat da/c28 0 2024-04-26T04:53:01.489 INFO:tasks.workunit.client.0.smithi077.stdout:2/376: getdents d9/dc 0 2024-04-26T04:53:01.489 INFO:tasks.workunit.client.0.smithi077.stdout:6/397: sync 2024-04-26T04:53:01.490 INFO:tasks.workunit.client.0.smithi077.stdout:0/281: write d0/d47/f16 [2148381,107493] 0 2024-04-26T04:53:01.491 INFO:tasks.workunit.client.1.smithi078.stdout:5/664: getdents d7/d48/d57/d65/d7a 0 2024-04-26T04:53:01.493 INFO:tasks.workunit.client.0.smithi077.stdout:0/282: chown d0/l57 9545442 1 2024-04-26T04:53:01.494 INFO:tasks.workunit.client.1.smithi078.stdout:3/736: creat d16/f106 x:0 0 0 2024-04-26T04:53:01.494 INFO:tasks.workunit.client.0.smithi077.stdout:2/377: write d9/dc/d1d/d1e/f5e [274991,106914] 0 2024-04-26T04:53:01.496 INFO:tasks.workunit.client.0.smithi077.stdout:9/333: dwrite d9/dd/d1b/f66 [0,4194304] 0 2024-04-26T04:53:01.496 INFO:tasks.workunit.client.0.smithi077.stdout:1/380: mknod d9/db/dd/c72 0 2024-04-26T04:53:01.496 INFO:tasks.workunit.client.0.smithi077.stdout:0/283: readlink d0/d19/l5e 0 2024-04-26T04:53:01.498 INFO:tasks.workunit.client.1.smithi078.stdout:2/697: mkdir d8/d11/d18/d5d/d61/df5 0 2024-04-26T04:53:01.500 INFO:tasks.workunit.client.0.smithi077.stdout:4/310: symlink d4/d17/d18/d43/d51/l69 0 2024-04-26T04:53:01.501 INFO:tasks.workunit.client.0.smithi077.stdout:1/381: chown d9/db/d2b/c65 121672 1 2024-04-26T04:53:01.503 INFO:tasks.workunit.client.0.smithi077.stdout:8/391: dwrite d4/f13 [0,4194304] 0 2024-04-26T04:53:01.503 INFO:tasks.workunit.client.0.smithi077.stdout:6/398: dread d8/d1c/f4f [0,4194304] 0 2024-04-26T04:53:01.505 INFO:tasks.workunit.client.1.smithi078.stdout:1/755: creat d7/d1e/d9e/dd/df0/ff5 x:0 0 0 2024-04-26T04:53:01.506 INFO:tasks.workunit.client.0.smithi077.stdout:6/399: fdatasync d8/f11 0 2024-04-26T04:53:01.507 INFO:tasks.workunit.client.1.smithi078.stdout:6/782: link d2/de/d27/da9/da/d35/f66 d2/d1a/d38/dd5/ff3 0 2024-04-26T04:53:01.508 INFO:tasks.workunit.client.1.smithi078.stdout:1/756: truncate d7/d1e/d9e/db/d49/d93/fd9 332788 0 2024-04-26T04:53:01.509 INFO:tasks.workunit.client.0.smithi077.stdout:2/378: dwrite d9/d15/d44/d4e/f5d [0,4194304] 0 2024-04-26T04:53:01.513 INFO:tasks.workunit.client.0.smithi077.stdout:5/385: mkdir d1/d8d 0 2024-04-26T04:53:01.515 INFO:tasks.workunit.client.1.smithi078.stdout:0/729: rmdir d2/d9/d60/dc3 0 2024-04-26T04:53:01.518 INFO:tasks.workunit.client.0.smithi077.stdout:2/379: dwrite d9/dc/d1d/d1e/f55 [4194304,4194304] 0 2024-04-26T04:53:01.526 INFO:tasks.workunit.client.0.smithi077.stdout:3/288: dread d0/d9/d32/f21 [0,4194304] 0 2024-04-26T04:53:01.526 INFO:tasks.workunit.client.0.smithi077.stdout:3/289: dread - d0/d7/d15/f3f zero size 2024-04-26T04:53:01.529 INFO:tasks.workunit.client.0.smithi077.stdout:0/284: mknod d0/d19/d1e/d59/d5a/c63 0 2024-04-26T04:53:01.530 INFO:tasks.workunit.client.0.smithi077.stdout:0/285: write d0/d2c/d4e/f4c [383658,48149] 0 2024-04-26T04:53:01.536 INFO:tasks.workunit.client.0.smithi077.stdout:5/386: fsync d1/d30/d35/f57 0 2024-04-26T04:53:01.538 INFO:tasks.workunit.client.0.smithi077.stdout:2/380: rename d9/d15/d44/d4e/f5d to d9/dc/d1d/d34/d39/d63/d79/d2b/f7b 0 2024-04-26T04:53:01.539 INFO:tasks.workunit.client.0.smithi077.stdout:7/375: creat da/d13/d22/f88 x:0 0 0 2024-04-26T04:53:01.539 INFO:tasks.workunit.client.0.smithi077.stdout:2/381: stat d9/dc/d1d/d34/d39/d63/d79/d2b/f46 0 2024-04-26T04:53:01.543 INFO:tasks.workunit.client.0.smithi077.stdout:9/334: write d9/f62 [547438,60899] 0 2024-04-26T04:53:01.545 INFO:tasks.workunit.client.0.smithi077.stdout:8/392: mkdir d4/d87 0 2024-04-26T04:53:01.548 INFO:tasks.workunit.client.1.smithi078.stdout:8/677: getdents d0/d19/d52/d95/d9d 0 2024-04-26T04:53:01.548 INFO:tasks.workunit.client.1.smithi078.stdout:2/698: rename d8/d11/d14/d69/d8f/d34/d70/c78 to d8/d11/d18/d5d/d8b/db0/db4/cf6 0 2024-04-26T04:53:01.548 INFO:tasks.workunit.client.1.smithi078.stdout:7/682: rmdir d9/da/d11/d13/d1a/dc4 0 2024-04-26T04:53:01.548 INFO:tasks.workunit.client.1.smithi078.stdout:5/665: creat d7/d18/d1c/d5a/d96/fdd x:0 0 0 2024-04-26T04:53:01.549 INFO:tasks.workunit.client.0.smithi077.stdout:5/387: dread - d1/d42/d62/d6b/f85 zero size 2024-04-26T04:53:01.549 INFO:tasks.workunit.client.1.smithi078.stdout:4/736: getdents d0/d1d/de1/d20/d31/d47/db2/d4c/dba 0 2024-04-26T04:53:01.550 INFO:tasks.workunit.client.1.smithi078.stdout:2/699: truncate d8/f25 5234172 0 2024-04-26T04:53:01.552 INFO:tasks.workunit.client.1.smithi078.stdout:6/783: mkdir d2/de/d27/da9/de5/df4 0 2024-04-26T04:53:01.555 INFO:tasks.workunit.client.1.smithi078.stdout:2/700: dwrite d8/d11/d6e/f7b [0,4194304] 0 2024-04-26T04:53:01.556 INFO:tasks.workunit.client.0.smithi077.stdout:3/290: write d0/d9/d32/d29/f31 [3685684,93872] 0 2024-04-26T04:53:01.559 INFO:tasks.workunit.client.1.smithi078.stdout:4/737: truncate d0/d1d/de1/f19 4054367 0 2024-04-26T04:53:01.560 INFO:tasks.workunit.client.0.smithi077.stdout:8/393: mknod d4/d17/d35/c88 0 2024-04-26T04:53:01.561 INFO:tasks.workunit.client.1.smithi078.stdout:0/730: dread d2/d1e/d5a/da3/fe3 [0,4194304] 0 2024-04-26T04:53:01.563 INFO:tasks.workunit.client.0.smithi077.stdout:6/400: creat d8/f88 x:0 0 0 2024-04-26T04:53:01.564 INFO:tasks.workunit.client.1.smithi078.stdout:8/678: rmdir d0/d19/d25/d36/d58/d5b 39 2024-04-26T04:53:01.564 INFO:tasks.workunit.client.0.smithi077.stdout:7/376: dwrite da/d15/f78 [0,4194304] 0 2024-04-26T04:53:01.565 INFO:tasks.workunit.client.1.smithi078.stdout:8/679: write d0/d19/d52/d35/d44/d6b/f9b [167365,83152] 0 2024-04-26T04:53:01.569 INFO:tasks.workunit.client.1.smithi078.stdout:4/738: creat d0/de/d22/d71/f109 x:0 0 0 2024-04-26T04:53:01.570 INFO:tasks.workunit.client.1.smithi078.stdout:6/784: mkdir d2/d1a/d38/dd5/ddf/df5 0 2024-04-26T04:53:01.571 INFO:tasks.workunit.client.0.smithi077.stdout:2/382: symlink d9/dc/d1d/d34/d39/d6b/l7c 0 2024-04-26T04:53:01.572 INFO:tasks.workunit.client.1.smithi078.stdout:0/731: mkdir d2/d9/d31/df2 0 2024-04-26T04:53:01.572 INFO:tasks.workunit.client.1.smithi078.stdout:4/739: chown d0/de/d22/d27/d56/d5a 7470 1 2024-04-26T04:53:01.572 INFO:tasks.workunit.client.0.smithi077.stdout:4/311: link d4/l6 d4/d4e/l6a 0 2024-04-26T04:53:01.573 INFO:tasks.workunit.client.1.smithi078.stdout:7/683: dwrite d9/da/d11/fad [0,4194304] 0 2024-04-26T04:53:01.574 INFO:tasks.workunit.client.0.smithi077.stdout:3/291: symlink d0/d7/d1f/l59 0 2024-04-26T04:53:01.575 INFO:tasks.workunit.client.0.smithi077.stdout:1/382: getdents d9/db 0 2024-04-26T04:53:01.577 INFO:tasks.workunit.client.0.smithi077.stdout:8/394: unlink d4/f40 0 2024-04-26T04:53:01.578 INFO:tasks.workunit.client.1.smithi078.stdout:7/684: truncate f4 3669521 0 2024-04-26T04:53:01.579 INFO:tasks.workunit.client.0.smithi077.stdout:6/401: unlink d8/d55/l59 0 2024-04-26T04:53:01.580 INFO:tasks.workunit.client.0.smithi077.stdout:6/402: fdatasync f4 0 2024-04-26T04:53:01.580 INFO:tasks.workunit.client.1.smithi078.stdout:0/732: dwrite d2/dc1/fdf [0,4194304] 0 2024-04-26T04:53:01.580 INFO:tasks.workunit.client.1.smithi078.stdout:7/685: write d9/da/d33/d4b/d4c/d5b/dc3/fdc [228610,75307] 0 2024-04-26T04:53:01.581 INFO:tasks.workunit.client.0.smithi077.stdout:6/403: stat d8/d52/d53/c71 0 2024-04-26T04:53:01.581 INFO:tasks.workunit.client.1.smithi078.stdout:2/701: creat d8/d11/d18/d5d/d61/ff7 x:0 0 0 2024-04-26T04:53:01.581 INFO:tasks.workunit.client.0.smithi077.stdout:6/404: readlink d8/d52/d53/l7f 0 2024-04-26T04:53:01.582 INFO:tasks.workunit.client.1.smithi078.stdout:7/686: write d9/fbc [987897,101580] 0 2024-04-26T04:53:01.583 INFO:tasks.workunit.client.0.smithi077.stdout:7/377: chown da/d13/d22/f25 872 1 2024-04-26T04:53:01.584 INFO:tasks.workunit.client.1.smithi078.stdout:9/655: sync 2024-04-26T04:53:01.585 INFO:tasks.workunit.client.0.smithi077.stdout:0/286: rename d0/d28/d39 to d0/d47/d64 0 2024-04-26T04:53:01.587 INFO:tasks.workunit.client.1.smithi078.stdout:6/785: rename d2/de/d27/da9/da/d36/db9/d97/dc3 to d2/de/d27/da9/d2a/dbf/df6 0 2024-04-26T04:53:01.589 INFO:tasks.workunit.client.0.smithi077.stdout:2/383: unlink d9/dc/d1d/d34/d39/d63/d79/d2b/f4b 0 2024-04-26T04:53:01.593 INFO:tasks.workunit.client.0.smithi077.stdout:3/292: mknod d0/d7/d15/c5a 0 2024-04-26T04:53:01.594 INFO:tasks.workunit.client.1.smithi078.stdout:6/786: dwrite d2/de/d27/da9/da/d36/db9/d97/fd2 [0,4194304] 0 2024-04-26T04:53:01.600 INFO:tasks.workunit.client.0.smithi077.stdout:8/395: mknod d4/df/d18/c89 0 2024-04-26T04:53:01.601 INFO:tasks.workunit.client.1.smithi078.stdout:0/733: read d2/d9/d18/db7/f95 [115821,109773] 0 2024-04-26T04:53:01.601 INFO:tasks.workunit.client.1.smithi078.stdout:4/740: getdents d0/d1d/de1/d20/dd5/de4 0 2024-04-26T04:53:01.601 INFO:tasks.workunit.client.1.smithi078.stdout:8/680: write d0/d19/d52/d35/d44/fc8 [665110,39241] 0 2024-04-26T04:53:01.602 INFO:tasks.workunit.client.1.smithi078.stdout:6/787: fsync d2/de/d27/da9/da/f4a 0 2024-04-26T04:53:01.607 INFO:tasks.workunit.client.0.smithi077.stdout:7/378: creat da/f89 x:0 0 0 2024-04-26T04:53:01.608 INFO:tasks.workunit.client.0.smithi077.stdout:7/379: write da/d6b/d63/f70 [4306485,80858] 0 2024-04-26T04:53:01.610 INFO:tasks.workunit.client.1.smithi078.stdout:9/656: rename dc/df/d4b/d22/d2a/d57/d69/db5/fb6 to dc/df/d27/d81/da8/fc0 0 2024-04-26T04:53:01.612 INFO:tasks.workunit.client.0.smithi077.stdout:5/388: rename d1/d8/f70 to d1/d12/d21/d25/d31/d3c/f8e 0 2024-04-26T04:53:01.613 INFO:tasks.workunit.client.0.smithi077.stdout:5/389: fsync d1/d12/d21/d25/d31/d3c/f4e 0 2024-04-26T04:53:01.614 INFO:tasks.workunit.client.0.smithi077.stdout:5/390: write d1/d8/f22 [2873909,57164] 0 2024-04-26T04:53:01.615 INFO:tasks.workunit.client.0.smithi077.stdout:5/391: readlink d1/d30/d35/l54 0 2024-04-26T04:53:01.617 INFO:tasks.workunit.client.1.smithi078.stdout:2/702: write d8/d11/d6e/ff4 [795954,53620] 0 2024-04-26T04:53:01.620 INFO:tasks.workunit.client.0.smithi077.stdout:3/293: chown d0/d9/d32/d2a 1930247 1 2024-04-26T04:53:01.622 INFO:tasks.workunit.client.0.smithi077.stdout:9/335: getdents d9/dd 0 2024-04-26T04:53:01.623 INFO:tasks.workunit.client.0.smithi077.stdout:1/383: dwrite d9/d53/d6e/f28 [0,4194304] 0 2024-04-26T04:53:01.625 INFO:tasks.workunit.client.0.smithi077.stdout:1/384: chown d9/d53/d6e/c3c 1417 1 2024-04-26T04:53:01.626 INFO:tasks.workunit.client.0.smithi077.stdout:8/396: mknod d4/df/d22/d80/c8a 0 2024-04-26T04:53:01.626 INFO:tasks.workunit.client.0.smithi077.stdout:0/287: write d0/d19/f2d [1551139,49488] 0 2024-04-26T04:53:01.628 INFO:tasks.workunit.client.0.smithi077.stdout:6/405: mknod d8/d2a/d86/c89 0 2024-04-26T04:53:01.631 INFO:tasks.workunit.client.0.smithi077.stdout:8/397: dread d4/df/d22/f68 [0,4194304] 0 2024-04-26T04:53:01.632 INFO:tasks.workunit.client.0.smithi077.stdout:7/380: unlink da/l10 0 2024-04-26T04:53:01.633 INFO:tasks.workunit.client.0.smithi077.stdout:7/381: write da/d15/f2f [2871608,56599] 0 2024-04-26T04:53:01.634 INFO:tasks.workunit.client.0.smithi077.stdout:2/384: rename d9/dc/d1d/d34/d39/d63/d79/d2b/c74 to d9/dc/d1d/d34/d39/d63/c7d 0 2024-04-26T04:53:01.635 INFO:tasks.workunit.client.0.smithi077.stdout:7/382: dread da/d15/d1a/f65 [0,4194304] 0 2024-04-26T04:53:01.638 INFO:tasks.workunit.client.0.smithi077.stdout:4/312: creat d4/f6b x:0 0 0 2024-04-26T04:53:01.642 INFO:tasks.workunit.client.0.smithi077.stdout:9/336: symlink d9/dd/l7a 0 2024-04-26T04:53:01.644 INFO:tasks.workunit.client.0.smithi077.stdout:4/313: dwrite d4/de/d15/f16 [4194304,4194304] 0 2024-04-26T04:53:01.645 INFO:tasks.workunit.client.0.smithi077.stdout:1/385: truncate d9/db/dd/f49 2388080 0 2024-04-26T04:53:01.648 INFO:tasks.workunit.client.0.smithi077.stdout:6/406: creat d8/d1c/f8a x:0 0 0 2024-04-26T04:53:01.654 INFO:tasks.workunit.client.1.smithi078.stdout:3/737: sync 2024-04-26T04:53:01.655 INFO:tasks.workunit.client.0.smithi077.stdout:8/398: dwrite d4/df/f15 [0,4194304] 0 2024-04-26T04:53:01.657 INFO:tasks.workunit.client.0.smithi077.stdout:8/399: write d4/df/d22/d2d/f1b [347589,59187] 0 2024-04-26T04:53:01.658 INFO:tasks.workunit.client.0.smithi077.stdout:0/288: rename d0/d47/d64/c4f to d0/d2c/d4e/c65 0 2024-04-26T04:53:01.664 INFO:tasks.workunit.client.0.smithi077.stdout:5/392: mkdir d1/d8f 0 2024-04-26T04:53:01.665 INFO:tasks.workunit.client.0.smithi077.stdout:5/393: readlink d1/d8/l3f 0 2024-04-26T04:53:01.672 INFO:tasks.workunit.client.1.smithi078.stdout:1/757: sync 2024-04-26T04:53:01.672 INFO:tasks.workunit.client.1.smithi078.stdout:5/666: sync 2024-04-26T04:53:01.673 INFO:tasks.workunit.client.1.smithi078.stdout:1/758: chown d7/d1e/d9e/dd/d20/c6a 1110 1 2024-04-26T04:53:01.683 INFO:tasks.workunit.client.1.smithi078.stdout:1/759: dread d7/fc [0,4194304] 0 2024-04-26T04:53:01.684 INFO:tasks.workunit.client.0.smithi077.stdout:0/289: dread d0/d47/ff [0,4194304] 0 2024-04-26T04:53:01.684 INFO:tasks.workunit.client.0.smithi077.stdout:2/385: rename d9/d15/d44/c4f to d9/dc/d1d/d34/d39/d63/d6a/c7e 0 2024-04-26T04:53:01.685 INFO:tasks.workunit.client.0.smithi077.stdout:2/386: readlink d9/dc/d1d/d34/d39/d6b/l7c 0 2024-04-26T04:53:01.691 INFO:tasks.workunit.client.0.smithi077.stdout:0/290: dwrite d0/d19/f32 [0,4194304] 0 2024-04-26T04:53:01.691 INFO:tasks.workunit.client.0.smithi077.stdout:5/394: write d1/d12/d21/d25/d31/d3c/f8e [1857255,113009] 0 2024-04-26T04:53:01.693 INFO:tasks.workunit.client.0.smithi077.stdout:3/294: link d0/d9/c57 d0/d9/d32/d2a/c5b 0 2024-04-26T04:53:01.697 INFO:tasks.workunit.client.0.smithi077.stdout:9/337: symlink d9/d3a/l7b 0 2024-04-26T04:53:01.704 INFO:tasks.workunit.client.0.smithi077.stdout:6/407: truncate d8/f11 3329234 0 2024-04-26T04:53:01.707 INFO:tasks.workunit.client.1.smithi078.stdout:7/687: truncate d9/da/d20/f9b 1184823 0 2024-04-26T04:53:01.709 INFO:tasks.workunit.client.0.smithi077.stdout:0/291: dwrite d0/d19/d1e/f26 [0,4194304] 0 2024-04-26T04:53:01.711 INFO:tasks.workunit.client.0.smithi077.stdout:3/295: dwrite d0/fb [4194304,4194304] 0 2024-04-26T04:53:01.711 INFO:tasks.workunit.client.0.smithi077.stdout:5/395: dwrite d1/d12/d28/d38/f8b [0,4194304] 0 2024-04-26T04:53:01.713 INFO:tasks.workunit.client.1.smithi078.stdout:0/734: truncate d2/d1e/f61 1357967 0 2024-04-26T04:53:01.713 INFO:tasks.workunit.client.0.smithi077.stdout:1/386: dwrite d9/db/d2b/d3f/f50 [0,4194304] 0 2024-04-26T04:53:01.714 INFO:tasks.workunit.client.0.smithi077.stdout:3/296: stat d0/d9 0 2024-04-26T04:53:01.715 INFO:tasks.workunit.client.1.smithi078.stdout:4/741: mknod d0/de/d22/d27/d56/d5a/ddc/c10a 0 2024-04-26T04:53:01.726 INFO:tasks.workunit.client.0.smithi077.stdout:7/383: dwrite da/d13/d22/f27 [0,4194304] 0 2024-04-26T04:53:01.727 INFO:tasks.workunit.client.0.smithi077.stdout:1/387: dwrite d9/f1d [0,4194304] 0 2024-04-26T04:53:01.728 INFO:tasks.workunit.client.0.smithi077.stdout:7/384: chown da/d13/c7c 132 1 2024-04-26T04:53:01.729 INFO:tasks.workunit.client.0.smithi077.stdout:0/292: dread d0/d19/d1e/f26 [0,4194304] 0 2024-04-26T04:53:01.729 INFO:tasks.workunit.client.1.smithi078.stdout:8/681: write d0/d19/f1c [4047544,35730] 0 2024-04-26T04:53:01.729 INFO:tasks.workunit.client.1.smithi078.stdout:8/682: chown d0/cf 373 1 2024-04-26T04:53:01.732 INFO:tasks.workunit.client.1.smithi078.stdout:3/738: unlink d16/d26/d2e/d74/lb5 0 2024-04-26T04:53:01.733 INFO:tasks.workunit.client.1.smithi078.stdout:5/667: rmdir d7/d11 39 2024-04-26T04:53:01.740 INFO:tasks.workunit.client.0.smithi077.stdout:2/387: unlink d9/c2a 0 2024-04-26T04:53:01.741 INFO:tasks.workunit.client.0.smithi077.stdout:7/385: dwrite da/d15/d1a/f79 [0,4194304] 0 2024-04-26T04:53:01.742 INFO:tasks.workunit.client.1.smithi078.stdout:7/688: unlink d9/f10 0 2024-04-26T04:53:01.744 INFO:tasks.workunit.client.0.smithi077.stdout:6/408: dwrite d8/f74 [4194304,4194304] 0 2024-04-26T04:53:01.748 INFO:tasks.workunit.client.1.smithi078.stdout:4/742: creat d0/de/d22/d71/dae/f10b x:0 0 0 2024-04-26T04:53:01.749 INFO:tasks.workunit.client.1.smithi078.stdout:0/735: dwrite d2/d9/d18/fb8 [0,4194304] 0 2024-04-26T04:53:01.750 INFO:tasks.workunit.client.1.smithi078.stdout:4/743: write d0/de/f7e [569528,27277] 0 2024-04-26T04:53:01.754 INFO:tasks.workunit.client.1.smithi078.stdout:6/788: link d2/de/d27/da9/da/d36/db9/d97/fd2 d2/de/d27/da9/d11/ff7 0 2024-04-26T04:53:01.759 INFO:tasks.workunit.client.1.smithi078.stdout:2/703: dread d8/d11/d18/d5d/d61/f76 [0,4194304] 0 2024-04-26T04:53:01.761 INFO:tasks.workunit.client.0.smithi077.stdout:4/314: getdents d4/d17/d18/d43 0 2024-04-26T04:53:01.763 INFO:tasks.workunit.client.1.smithi078.stdout:3/739: creat d16/d26/d2b/d31/d71/d8a/f107 x:0 0 0 2024-04-26T04:53:01.764 INFO:tasks.workunit.client.1.smithi078.stdout:2/704: dread d8/d11/d6e/ff4 [0,4194304] 0 2024-04-26T04:53:01.766 INFO:tasks.workunit.client.1.smithi078.stdout:5/668: creat d7/d48/d57/d65/d7a/d9c/fde x:0 0 0 2024-04-26T04:53:01.767 INFO:tasks.workunit.client.0.smithi077.stdout:0/293: creat d0/d2c/d4e/f66 x:0 0 0 2024-04-26T04:53:01.769 INFO:tasks.workunit.client.0.smithi077.stdout:8/400: getdents d4/df 0 2024-04-26T04:53:01.769 INFO:tasks.workunit.client.1.smithi078.stdout:1/760: dwrite d7/d1e/d9e/f3e [0,4194304] 0 2024-04-26T04:53:01.770 INFO:tasks.workunit.client.0.smithi077.stdout:2/388: write d9/f10 [3688439,56906] 0 2024-04-26T04:53:01.771 INFO:tasks.workunit.client.1.smithi078.stdout:3/740: dwrite d16/d26/d2b/fa9 [0,4194304] 0 2024-04-26T04:53:01.774 INFO:tasks.workunit.client.0.smithi077.stdout:6/409: symlink d8/d2a/d57/l8b 0 2024-04-26T04:53:01.775 INFO:tasks.workunit.client.0.smithi077.stdout:6/410: dread - d8/d1c/f8a zero size 2024-04-26T04:53:01.776 INFO:tasks.workunit.client.1.smithi078.stdout:0/736: truncate d2/d9/d31/d4a/f82 5048662 0 2024-04-26T04:53:01.777 INFO:tasks.workunit.client.1.smithi078.stdout:4/744: dread d0/d1d/de1/d20/d31/d47/db2/fc5 [0,4194304] 0 2024-04-26T04:53:01.778 INFO:tasks.workunit.client.1.smithi078.stdout:4/745: fdatasync d0/de/f91 0 2024-04-26T04:53:01.782 INFO:tasks.workunit.client.0.smithi077.stdout:8/401: dwrite d4/df/f15 [4194304,4194304] 0 2024-04-26T04:53:01.783 INFO:tasks.workunit.client.1.smithi078.stdout:0/737: dwrite d2/d9/d18/d7d/da9/dab/dc6/fb2 [4194304,4194304] 0 2024-04-26T04:53:01.784 INFO:tasks.workunit.client.0.smithi077.stdout:9/338: mkdir d9/dd/d3d/d7c 0 2024-04-26T04:53:01.790 INFO:tasks.workunit.client.0.smithi077.stdout:9/339: dwrite d9/f1c [4194304,4194304] 0 2024-04-26T04:53:01.797 INFO:tasks.workunit.client.1.smithi078.stdout:2/705: dread d8/d11/d14/f86 [0,4194304] 0 2024-04-26T04:53:01.799 INFO:tasks.workunit.client.1.smithi078.stdout:7/689: creat d9/da/d11/d13/d1a/dd4/fdf x:0 0 0 2024-04-26T04:53:01.801 INFO:tasks.workunit.client.0.smithi077.stdout:2/389: fdatasync d9/dc/d5f/f62 0 2024-04-26T04:53:01.804 INFO:tasks.workunit.client.0.smithi077.stdout:3/297: dwrite d0/d7/f54 [0,4194304] 0 2024-04-26T04:53:01.805 INFO:tasks.workunit.client.0.smithi077.stdout:4/315: mkdir d4/d17/d18/d43/d53/d58/d6c 0 2024-04-26T04:53:01.807 INFO:tasks.workunit.client.0.smithi077.stdout:9/340: dwrite d9/f62 [0,4194304] 0 2024-04-26T04:53:01.807 INFO:tasks.workunit.client.0.smithi077.stdout:8/402: creat d4/d17/d35/d5b/f8b x:0 0 0 2024-04-26T04:53:01.810 INFO:tasks.workunit.client.1.smithi078.stdout:6/789: creat d2/de/d27/da9/de5/df4/ff8 x:0 0 0 2024-04-26T04:53:01.813 INFO:tasks.workunit.client.0.smithi077.stdout:3/298: truncate d0/f42 975065 0 2024-04-26T04:53:01.815 INFO:tasks.workunit.client.0.smithi077.stdout:0/294: dwrite d0/d19/d1e/f34 [0,4194304] 0 2024-04-26T04:53:01.817 INFO:tasks.workunit.client.0.smithi077.stdout:5/396: dwrite d1/d12/f19 [0,4194304] 0 2024-04-26T04:53:01.817 INFO:tasks.workunit.client.1.smithi078.stdout:9/657: getdents dc/df/d37 0 2024-04-26T04:53:01.818 INFO:tasks.workunit.client.1.smithi078.stdout:9/658: chown dc/df/d4b/f86 413 1 2024-04-26T04:53:01.819 INFO:tasks.workunit.client.0.smithi077.stdout:1/388: rename d9/db/dd/d34/f5a to d9/f73 0 2024-04-26T04:53:01.820 INFO:tasks.workunit.client.1.smithi078.stdout:0/738: creat d2/d9/d31/d73/ff3 x:0 0 0 2024-04-26T04:53:01.820 INFO:tasks.workunit.client.0.smithi077.stdout:1/389: stat d9/db/dd/l17 0 2024-04-26T04:53:01.823 INFO:tasks.workunit.client.0.smithi077.stdout:6/411: dread d8/fb [0,4194304] 0 2024-04-26T04:53:01.825 INFO:tasks.workunit.client.0.smithi077.stdout:6/412: chown d8/f6d 79036 1 2024-04-26T04:53:01.830 INFO:tasks.workunit.client.1.smithi078.stdout:1/761: creat d7/d1e/d9e/dd/d20/dca/ff6 x:0 0 0 2024-04-26T04:53:01.831 INFO:tasks.workunit.client.0.smithi077.stdout:5/397: creat d1/d12/d28/d38/f90 x:0 0 0 2024-04-26T04:53:01.832 INFO:tasks.workunit.client.0.smithi077.stdout:5/398: readlink d1/d30/l5c 0 2024-04-26T04:53:01.832 INFO:tasks.workunit.client.0.smithi077.stdout:5/399: chown d1/d12/d21/l29 18698236 1 2024-04-26T04:53:01.833 INFO:tasks.workunit.client.0.smithi077.stdout:7/386: rename da/d13/c7c to da/d15/d42/c8a 0 2024-04-26T04:53:01.835 INFO:tasks.workunit.client.0.smithi077.stdout:1/390: unlink d9/db/l1a 0 2024-04-26T04:53:01.836 INFO:tasks.workunit.client.1.smithi078.stdout:8/683: rename d0/c11 to d0/d19/d52/d5c/cdf 0 2024-04-26T04:53:01.837 INFO:tasks.workunit.client.0.smithi077.stdout:2/390: symlink d9/dc/d5f/d78/l7f 0 2024-04-26T04:53:01.837 INFO:tasks.workunit.client.1.smithi078.stdout:8/684: stat d0/d19/d52/d35/d44/f93 0 2024-04-26T04:53:01.838 INFO:tasks.workunit.client.1.smithi078.stdout:0/739: creat d2/d9/d78/d84/dd7/ff4 x:0 0 0 2024-04-26T04:53:01.839 INFO:tasks.workunit.client.0.smithi077.stdout:9/341: mkdir d9/d7d 0 2024-04-26T04:53:01.843 INFO:tasks.workunit.client.1.smithi078.stdout:5/669: link d7/d21/d2f/fc2 d7/d48/d57/d65/d70/d72/fdf 0 2024-04-26T04:53:01.845 INFO:tasks.workunit.client.0.smithi077.stdout:8/403: rmdir d4/d17/d35/d5b 39 2024-04-26T04:53:01.846 INFO:tasks.workunit.client.1.smithi078.stdout:3/741: write d16/f8d [272744,26341] 0 2024-04-26T04:53:01.852 INFO:tasks.workunit.client.0.smithi077.stdout:5/400: unlink d1/d8/c24 0 2024-04-26T04:53:01.853 INFO:tasks.workunit.client.1.smithi078.stdout:9/659: creat dc/df/d4b/dae/fc1 x:0 0 0 2024-04-26T04:53:01.854 INFO:tasks.workunit.client.0.smithi077.stdout:9/342: dread f2 [0,4194304] 0 2024-04-26T04:53:01.855 INFO:tasks.workunit.client.0.smithi077.stdout:3/299: rename d0/d7/d15/f37 to d0/f5c 0 2024-04-26T04:53:01.856 INFO:tasks.workunit.client.0.smithi077.stdout:7/387: dread da/d13/f5c [0,4194304] 0 2024-04-26T04:53:01.856 INFO:tasks.workunit.client.0.smithi077.stdout:9/343: write d9/d3a/d79/f6b [667234,127682] 0 2024-04-26T04:53:01.860 INFO:tasks.workunit.client.1.smithi078.stdout:0/740: rename d2/f57 to d2/d9/d18/d7d/da9/dab/ff5 0 2024-04-26T04:53:01.861 INFO:tasks.workunit.client.1.smithi078.stdout:8/685: dwrite d0/d19/d25/fd6 [0,4194304] 0 2024-04-26T04:53:01.862 INFO:tasks.workunit.client.1.smithi078.stdout:8/686: fdatasync d0/d19/d52/faa 0 2024-04-26T04:53:01.865 INFO:tasks.workunit.client.0.smithi077.stdout:2/391: symlink d9/dc/d5f/d78/l80 0 2024-04-26T04:53:01.865 INFO:tasks.workunit.client.0.smithi077.stdout:2/392: fsync d9/dc/f5c 0 2024-04-26T04:53:01.871 INFO:tasks.workunit.client.0.smithi077.stdout:4/316: creat d4/de/d15/f6d x:0 0 0 2024-04-26T04:53:01.873 INFO:tasks.workunit.client.0.smithi077.stdout:8/404: creat d4/df/f8c x:0 0 0 2024-04-26T04:53:01.873 INFO:tasks.workunit.client.1.smithi078.stdout:2/706: write d8/d11/d18/d5d/d61/f1b [1389468,50088] 0 2024-04-26T04:53:01.874 INFO:tasks.workunit.client.0.smithi077.stdout:8/405: stat d4/df/f16 0 2024-04-26T04:53:01.874 INFO:tasks.workunit.client.1.smithi078.stdout:6/790: link d2/d1a/f9b d2/d1a/d38/dd5/ff9 0 2024-04-26T04:53:01.875 INFO:tasks.workunit.client.0.smithi077.stdout:8/406: write d4/df/d22/d2d/d2b/f7b [155696,97116] 0 2024-04-26T04:53:01.875 INFO:tasks.workunit.client.0.smithi077.stdout:5/401: symlink d1/d12/d28/d38/l91 0 2024-04-26T04:53:01.878 INFO:tasks.workunit.client.1.smithi078.stdout:9/660: unlink dc/df/d4b/d22/d2a/d57/f73 0 2024-04-26T04:53:01.879 INFO:tasks.workunit.client.0.smithi077.stdout:0/295: truncate d0/f4 3277657 0 2024-04-26T04:53:01.879 INFO:tasks.workunit.client.0.smithi077.stdout:0/296: chown d0/d19/d1e/f26 51369 1 2024-04-26T04:53:01.880 INFO:tasks.workunit.client.0.smithi077.stdout:0/297: fdatasync d0/d47/f50 0 2024-04-26T04:53:01.881 INFO:tasks.workunit.client.1.smithi078.stdout:4/746: sync 2024-04-26T04:53:01.884 INFO:tasks.workunit.client.0.smithi077.stdout:6/413: rename d8/f74 to d8/d52/d53/d5e/f8c 0 2024-04-26T04:53:01.888 INFO:tasks.workunit.client.0.smithi077.stdout:9/344: read d9/dd/d1b/d4e/f6a [21037,58849] 0 2024-04-26T04:53:01.892 INFO:tasks.workunit.client.1.smithi078.stdout:3/742: write d16/d26/d2e/f4d [56614,35240] 0 2024-04-26T04:53:01.894 INFO:tasks.workunit.client.1.smithi078.stdout:8/687: truncate d0/d19/d25/d36/d58/d5b/d61/fd9 168425 0 2024-04-26T04:53:01.894 INFO:tasks.workunit.client.1.smithi078.stdout:8/688: dread - d0/d19/d52/d5c/fcd zero size 2024-04-26T04:53:01.900 INFO:tasks.workunit.client.1.smithi078.stdout:8/689: dwrite d0/d19/d52/d35/d73/fc2 [0,4194304] 0 2024-04-26T04:53:01.900 INFO:tasks.workunit.client.1.smithi078.stdout:1/762: creat d7/ff7 x:0 0 0 2024-04-26T04:53:01.900 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:01 smithi077 ceph-mon[26311]: pgmap v20: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 56 MiB/s rd, 213 MiB/s wr, 376 op/s 2024-04-26T04:53:01.902 INFO:tasks.workunit.client.1.smithi078.stdout:8/690: dread - d0/d19/d52/d95/d9d/fc5 zero size 2024-04-26T04:53:01.904 INFO:tasks.workunit.client.1.smithi078.stdout:2/707: symlink d8/db7/lf8 0 2024-04-26T04:53:01.905 INFO:tasks.workunit.client.1.smithi078.stdout:8/691: dread - d0/d7a/dbe/fd8 zero size 2024-04-26T04:53:01.906 INFO:tasks.workunit.client.1.smithi078.stdout:6/791: fdatasync d2/de/d27/da9/da/d36/db9/f47 0 2024-04-26T04:53:01.906 INFO:tasks.workunit.client.1.smithi078.stdout:2/708: stat d8/d11/d14/d69/l82 0 2024-04-26T04:53:01.906 INFO:tasks.workunit.client.0.smithi077.stdout:0/298: mknod d0/d2c/d4e/c67 0 2024-04-26T04:53:01.907 INFO:tasks.workunit.client.1.smithi078.stdout:0/741: dwrite d2/d9/d18/db7/fd1 [4194304,4194304] 0 2024-04-26T04:53:01.907 INFO:tasks.workunit.client.1.smithi078.stdout:2/709: fsync d8/f25 0 2024-04-26T04:53:01.908 INFO:tasks.workunit.client.1.smithi078.stdout:9/661: unlink dc/df/d4b/d22/d2a/d57/d69/laf 0 2024-04-26T04:53:01.908 INFO:tasks.workunit.client.1.smithi078.stdout:2/710: dread - d8/d11/d14/d69/d8f/fe3 zero size 2024-04-26T04:53:01.911 INFO:tasks.workunit.client.1.smithi078.stdout:4/747: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/l10c 0 2024-04-26T04:53:01.911 INFO:tasks.workunit.client.1.smithi078.stdout:4/748: stat d0/de/d22/c88 0 2024-04-26T04:53:01.912 INFO:tasks.workunit.client.1.smithi078.stdout:4/749: stat d0/de/d22/d71/dae/f10b 0 2024-04-26T04:53:01.914 INFO:tasks.workunit.client.0.smithi077.stdout:4/317: rename d4/d17/d18/d43/d53/d58 to d4/d17/d18/d43/d6e 0 2024-04-26T04:53:01.915 INFO:tasks.workunit.client.1.smithi078.stdout:7/690: mknod d9/da/d11/d13/d50/d8a/ce0 0 2024-04-26T04:53:01.920 INFO:tasks.workunit.client.0.smithi077.stdout:9/345: creat d9/d45/f7e x:0 0 0 2024-04-26T04:53:01.921 INFO:tasks.workunit.client.0.smithi077.stdout:5/402: dwrite d1/d12/f60 [0,4194304] 0 2024-04-26T04:53:01.923 INFO:tasks.workunit.client.0.smithi077.stdout:2/393: creat d9/dc/d1d/d1e/d77/f81 x:0 0 0 2024-04-26T04:53:01.925 INFO:tasks.workunit.client.0.smithi077.stdout:6/414: dwrite d8/d52/d53/d5e/f8c [4194304,4194304] 0 2024-04-26T04:53:01.925 INFO:tasks.workunit.client.0.smithi077.stdout:2/394: readlink d9/dc/d1d/d34/d39/d63/d79/l6f 0 2024-04-26T04:53:01.932 INFO:tasks.workunit.client.0.smithi077.stdout:8/407: mkdir d4/d17/d35/d5b/d8d 0 2024-04-26T04:53:01.933 INFO:tasks.workunit.client.0.smithi077.stdout:8/408: stat d4/d17/d35/d5b/f63 0 2024-04-26T04:53:01.935 INFO:tasks.workunit.client.1.smithi078.stdout:6/792: read d2/de/f33 [1452688,8608] 0 2024-04-26T04:53:01.937 INFO:tasks.workunit.client.1.smithi078.stdout:2/711: symlink d8/d11/d14/d5b/lf9 0 2024-04-26T04:53:01.939 INFO:tasks.workunit.client.0.smithi077.stdout:4/318: rmdir d4/d17/d18 39 2024-04-26T04:53:01.941 INFO:tasks.workunit.client.0.smithi077.stdout:3/300: creat d0/d7/d15/f5d x:0 0 0 2024-04-26T04:53:01.943 INFO:tasks.workunit.client.1.smithi078.stdout:7/691: chown d9/da/d11/d13/d1a/d52/d5a/d28/d6d 1 1 2024-04-26T04:53:01.945 INFO:tasks.workunit.client.1.smithi078.stdout:3/743: rename d16/d26/d2e/d74/l38 to d16/d26/d2b/d31/d78/dd1/l108 0 2024-04-26T04:53:01.946 INFO:tasks.workunit.client.1.smithi078.stdout:5/670: link d7/d18/d32/d84/fb2 d7/d21/d2f/fe0 0 2024-04-26T04:53:01.946 INFO:tasks.workunit.client.0.smithi077.stdout:1/391: dwrite d9/db/f1c [4194304,4194304] 0 2024-04-26T04:53:01.947 INFO:tasks.workunit.client.0.smithi077.stdout:5/403: fdatasync d1/f5 0 2024-04-26T04:53:01.948 INFO:tasks.workunit.client.1.smithi078.stdout:8/692: mkdir d0/d19/d25/de0 0 2024-04-26T04:53:01.949 INFO:tasks.workunit.client.1.smithi078.stdout:8/693: fdatasync d0/d19/d52/f69 0 2024-04-26T04:53:01.954 INFO:tasks.workunit.client.0.smithi077.stdout:9/346: unlink d9/dd/d1b/d2b/d32/d36/f5e 0 2024-04-26T04:53:01.957 INFO:tasks.workunit.client.0.smithi077.stdout:6/415: symlink d8/d55/l8d 0 2024-04-26T04:53:01.958 INFO:tasks.workunit.client.1.smithi078.stdout:2/712: stat d8/d11/d14/d5b/c73 0 2024-04-26T04:53:01.961 INFO:tasks.workunit.client.1.smithi078.stdout:4/750: rmdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab 39 2024-04-26T04:53:01.963 INFO:tasks.workunit.client.1.smithi078.stdout:0/742: dwrite d2/d9/d31/d4a/fa2 [0,4194304] 0 2024-04-26T04:53:01.963 INFO:tasks.workunit.client.0.smithi077.stdout:8/409: creat d4/df/d22/d80/f8e x:0 0 0 2024-04-26T04:53:01.964 INFO:tasks.workunit.client.0.smithi077.stdout:6/416: dwrite d8/d52/d53/d5e/f69 [0,4194304] 0 2024-04-26T04:53:01.965 INFO:tasks.workunit.client.0.smithi077.stdout:0/299: mkdir d0/d47/d68 0 2024-04-26T04:53:01.966 INFO:tasks.workunit.client.1.smithi078.stdout:9/662: truncate dc/df/d4b/f1c 1834378 0 2024-04-26T04:53:01.968 INFO:tasks.workunit.client.1.smithi078.stdout:1/763: rename d7/ddd/dab to d7/d1e/d9e/db/d49/df8 0 2024-04-26T04:53:01.970 INFO:tasks.workunit.client.1.smithi078.stdout:3/744: fdatasync d16/d26/f2a 0 2024-04-26T04:53:01.972 INFO:tasks.workunit.client.1.smithi078.stdout:5/671: creat d7/d18/d32/d37/d8f/fe1 x:0 0 0 2024-04-26T04:53:01.974 INFO:tasks.workunit.client.1.smithi078.stdout:8/694: creat d0/d7a/dbe/fe1 x:0 0 0 2024-04-26T04:53:01.978 INFO:tasks.workunit.client.1.smithi078.stdout:6/793: link d2/de/d27/da9/d2a/dbf/d5f/fd4 d2/d1a/d38/dd5/ffa 0 2024-04-26T04:53:01.982 INFO:tasks.workunit.client.1.smithi078.stdout:0/743: symlink d2/d1e/lf6 0 2024-04-26T04:53:01.982 INFO:tasks.workunit.client.1.smithi078.stdout:0/744: stat d2/d9/d31/l63 0 2024-04-26T04:53:01.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/692: truncate d9/da/f66 3386624 0 2024-04-26T04:53:01.985 INFO:tasks.workunit.client.1.smithi078.stdout:4/751: truncate d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 1968515 0 2024-04-26T04:53:01.987 INFO:tasks.workunit.client.1.smithi078.stdout:9/663: symlink dc/df/d27/d81/lc2 0 2024-04-26T04:53:01.988 INFO:tasks.workunit.client.1.smithi078.stdout:9/664: write dc/df/f9f [365034,103793] 0 2024-04-26T04:53:01.989 INFO:tasks.workunit.client.1.smithi078.stdout:1/764: fdatasync d7/d1e/d9e/dd/d20/fb3 0 2024-04-26T04:53:01.993 INFO:tasks.workunit.client.1.smithi078.stdout:7/693: dread d9/da/d11/d13/d1a/d52/d5a/f44 [0,4194304] 0 2024-04-26T04:53:01.995 INFO:tasks.workunit.client.1.smithi078.stdout:3/745: rename d16/d26/d2b/ld5 to d16/d26/d2e/d5f/d49/dcc/de2/l109 0 2024-04-26T04:53:01.998 INFO:tasks.workunit.client.1.smithi078.stdout:8/695: fsync d0/d19/d52/d3c/fa5 0 2024-04-26T04:53:01.999 INFO:tasks.workunit.client.1.smithi078.stdout:5/672: dread d7/d48/d57/d65/d46/f68 [0,4194304] 0 2024-04-26T04:53:02.002 INFO:tasks.workunit.client.0.smithi077.stdout:7/388: getdents da/d15/d42 0 2024-04-26T04:53:02.005 INFO:tasks.workunit.client.1.smithi078.stdout:0/745: dread d2/d39/d96/fe9 [0,4194304] 0 2024-04-26T04:53:02.006 INFO:tasks.workunit.client.1.smithi078.stdout:2/713: dwrite d8/d11/d14/d69/d8f/d34/dd3/de4/f93 [0,4194304] 0 2024-04-26T04:53:02.009 INFO:tasks.workunit.client.0.smithi077.stdout:2/395: symlink d9/dc/d6e/l82 0 2024-04-26T04:53:02.010 INFO:tasks.workunit.client.1.smithi078.stdout:1/765: chown d7/d1e/d9e/db/d37/d58/d5c/dc0/l27 1 1 2024-04-26T04:53:02.012 INFO:tasks.workunit.client.0.smithi077.stdout:8/410: fsync d4/df/f16 0 2024-04-26T04:53:02.013 INFO:tasks.workunit.client.1.smithi078.stdout:7/694: chown d9/da/d33/fb0 55019627 1 2024-04-26T04:53:02.015 INFO:tasks.workunit.client.0.smithi077.stdout:6/417: unlink d8/d52/d53/d5e/l83 0 2024-04-26T04:53:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:01 smithi078 ceph-mon[33746]: pgmap v20: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 56 MiB/s rd, 213 MiB/s wr, 376 op/s 2024-04-26T04:53:02.017 INFO:tasks.workunit.client.0.smithi077.stdout:0/300: mkdir d0/d19/d69 0 2024-04-26T04:53:02.017 INFO:tasks.workunit.client.0.smithi077.stdout:6/418: read d8/f17 [2653491,105070] 0 2024-04-26T04:53:02.018 INFO:tasks.workunit.client.0.smithi077.stdout:0/301: write d0/d47/f31 [1586393,120473] 0 2024-04-26T04:53:02.018 INFO:tasks.workunit.client.1.smithi078.stdout:3/746: stat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/f7d 0 2024-04-26T04:53:02.021 INFO:tasks.workunit.client.0.smithi077.stdout:3/301: mknod d0/c5e 0 2024-04-26T04:53:02.022 INFO:tasks.workunit.client.1.smithi078.stdout:8/696: mknod d0/d19/d25/d32/d8a/ce2 0 2024-04-26T04:53:02.031 INFO:tasks.workunit.client.0.smithi077.stdout:7/389: dread da/d15/d1a/f20 [0,4194304] 0 2024-04-26T04:53:02.031 INFO:tasks.workunit.client.1.smithi078.stdout:9/665: symlink dc/df/d27/dbc/lc3 0 2024-04-26T04:53:02.032 INFO:tasks.workunit.client.0.smithi077.stdout:9/347: dwrite d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:02.033 INFO:tasks.workunit.client.0.smithi077.stdout:7/390: truncate da/d6b/d63/d64/f83 103233 0 2024-04-26T04:53:02.034 INFO:tasks.workunit.client.1.smithi078.stdout:2/714: creat d8/d11/d14/d69/d8f/d34/dd3/dd9/ffa x:0 0 0 2024-04-26T04:53:02.035 INFO:tasks.workunit.client.0.smithi077.stdout:8/411: creat d4/df/d18/d56/f8f x:0 0 0 2024-04-26T04:53:02.036 INFO:tasks.workunit.client.0.smithi077.stdout:8/412: chown d4/f42 5 1 2024-04-26T04:53:02.037 INFO:tasks.workunit.client.1.smithi078.stdout:9/666: dwrite f1 [0,4194304] 0 2024-04-26T04:53:02.038 INFO:tasks.workunit.client.0.smithi077.stdout:8/413: fsync d4/d17/f6a 0 2024-04-26T04:53:02.041 INFO:tasks.workunit.client.0.smithi077.stdout:7/391: dwrite da/d6b/d63/d64/f83 [0,4194304] 0 2024-04-26T04:53:02.041 INFO:tasks.workunit.client.1.smithi078.stdout:4/752: rename d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fb8 to d0/d1d/de1/dd8/f10d 0 2024-04-26T04:53:02.042 INFO:tasks.workunit.client.1.smithi078.stdout:9/667: rename dc/df/d4b/d22/d2a to dc/df/d4b/d22/d2a/d57/d69/dc4 22 2024-04-26T04:53:02.045 INFO:tasks.workunit.client.0.smithi077.stdout:4/319: dwrite d4/f14 [0,4194304] 0 2024-04-26T04:53:02.047 INFO:tasks.workunit.client.0.smithi077.stdout:0/302: unlink d0/d2c/l3a 0 2024-04-26T04:53:02.048 INFO:tasks.workunit.client.1.smithi078.stdout:3/747: rmdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56 39 2024-04-26T04:53:02.050 INFO:tasks.workunit.client.1.smithi078.stdout:3/748: write d16/d7e/fea [653353,121279] 0 2024-04-26T04:53:02.050 INFO:tasks.workunit.client.0.smithi077.stdout:0/303: write d0/d47/d64/f5c [210849,38258] 0 2024-04-26T04:53:02.052 INFO:tasks.workunit.client.1.smithi078.stdout:0/746: write d2/d9/d18/db7/d92/fca [271543,73429] 0 2024-04-26T04:53:02.053 INFO:tasks.workunit.client.0.smithi077.stdout:7/392: dwrite da/d13/d22/f88 [0,4194304] 0 2024-04-26T04:53:02.053 INFO:tasks.workunit.client.1.smithi078.stdout:3/749: read - d16/d26/d2b/d31/d71/d8a/dbc/fdf zero size 2024-04-26T04:53:02.054 INFO:tasks.workunit.client.1.smithi078.stdout:5/673: dwrite d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:53:02.055 INFO:tasks.workunit.client.1.smithi078.stdout:6/794: creat d2/de/d27/da9/da/ffb x:0 0 0 2024-04-26T04:53:02.056 INFO:tasks.workunit.client.0.smithi077.stdout:5/404: creat d1/d12/d21/d25/d31/f92 x:0 0 0 2024-04-26T04:53:02.061 INFO:tasks.workunit.client.0.smithi077.stdout:0/304: dwrite d0/d47/f31 [4194304,4194304] 0 2024-04-26T04:53:02.062 INFO:tasks.workunit.client.0.smithi077.stdout:9/348: stat d9/dd/f52 0 2024-04-26T04:53:02.065 INFO:tasks.workunit.client.1.smithi078.stdout:1/766: symlink d7/lf9 0 2024-04-26T04:53:02.067 INFO:tasks.workunit.client.1.smithi078.stdout:1/767: truncate d7/d1e/d9e/db/d37/d58/fef 114311 0 2024-04-26T04:53:02.069 INFO:tasks.workunit.client.1.smithi078.stdout:4/753: creat d0/de/d22/d27/de7/f10e x:0 0 0 2024-04-26T04:53:02.070 INFO:tasks.workunit.client.1.smithi078.stdout:4/754: readlink d0/de/d22/d27/d7b/l106 0 2024-04-26T04:53:02.071 INFO:tasks.workunit.client.1.smithi078.stdout:9/668: symlink dc/df/d4b/d22/d2d/d98/lc5 0 2024-04-26T04:53:02.072 INFO:tasks.workunit.client.0.smithi077.stdout:6/419: mknod d8/d2a/c8e 0 2024-04-26T04:53:02.073 INFO:tasks.workunit.client.0.smithi077.stdout:8/414: mkdir d4/d17/d90 0 2024-04-26T04:53:02.078 INFO:tasks.workunit.client.0.smithi077.stdout:6/420: dread d8/fb [0,4194304] 0 2024-04-26T04:53:02.079 INFO:tasks.workunit.client.1.smithi078.stdout:2/715: fsync d8/d11/d14/fab 0 2024-04-26T04:53:02.081 INFO:tasks.workunit.client.0.smithi077.stdout:6/421: write d8/d55/f62 [3298635,44535] 0 2024-04-26T04:53:02.081 INFO:tasks.workunit.client.0.smithi077.stdout:1/392: creat d9/db/dd/f74 x:0 0 0 2024-04-26T04:53:02.082 INFO:tasks.workunit.client.0.smithi077.stdout:8/415: dwrite d4/f76 [0,4194304] 0 2024-04-26T04:53:02.083 INFO:tasks.workunit.client.0.smithi077.stdout:7/393: unlink da/d13/d22/f25 0 2024-04-26T04:53:02.085 INFO:tasks.workunit.client.0.smithi077.stdout:7/394: chown da/d13/d22/d7d/d80 1045089811 1 2024-04-26T04:53:02.086 INFO:tasks.workunit.client.1.smithi078.stdout:8/697: link d0/d19/fa1 d0/d19/d25/d36/d58/fe3 0 2024-04-26T04:53:02.087 INFO:tasks.workunit.client.0.smithi077.stdout:7/395: stat da/d13/d22/d7d/d80 0 2024-04-26T04:53:02.089 INFO:tasks.workunit.client.0.smithi077.stdout:2/396: link d9/dc/f5c d9/dc/d1d/f83 0 2024-04-26T04:53:02.089 INFO:tasks.workunit.client.1.smithi078.stdout:0/747: mknod d2/d9/d31/df2/cf7 0 2024-04-26T04:53:02.092 INFO:tasks.workunit.client.0.smithi077.stdout:0/305: symlink d0/d2c/l6a 0 2024-04-26T04:53:02.092 INFO:tasks.workunit.client.0.smithi077.stdout:6/422: dwrite d8/d2a/f2e [4194304,4194304] 0 2024-04-26T04:53:02.093 INFO:tasks.workunit.client.1.smithi078.stdout:5/674: dread d7/d18/d32/f3b [0,4194304] 0 2024-04-26T04:53:02.098 INFO:tasks.workunit.client.0.smithi077.stdout:2/397: dwrite d9/f12 [0,4194304] 0 2024-04-26T04:53:02.100 INFO:tasks.workunit.client.1.smithi078.stdout:6/795: dread d2/de/d27/da9/f6b [0,4194304] 0 2024-04-26T04:53:02.101 INFO:tasks.workunit.client.1.smithi078.stdout:6/796: write d2/de/d27/da9/d3a/d42/f84 [598552,61974] 0 2024-04-26T04:53:02.104 INFO:tasks.workunit.client.0.smithi077.stdout:1/393: dread - d9/db/f60 zero size 2024-04-26T04:53:02.106 INFO:tasks.workunit.client.1.smithi078.stdout:2/716: unlink d8/cd 0 2024-04-26T04:53:02.108 INFO:tasks.workunit.client.1.smithi078.stdout:1/768: truncate d7/d1e/d9e/d4c/d3d/f95 394683 0 2024-04-26T04:53:02.109 INFO:tasks.workunit.client.0.smithi077.stdout:8/416: symlink d4/df/d18/d56/l91 0 2024-04-26T04:53:02.110 INFO:tasks.workunit.client.1.smithi078.stdout:9/669: symlink dc/df/d4b/d22/d2a/d57/d93/lc6 0 2024-04-26T04:53:02.111 INFO:tasks.workunit.client.0.smithi077.stdout:5/405: symlink d1/d8c/l93 0 2024-04-26T04:53:02.112 INFO:tasks.workunit.client.1.smithi078.stdout:8/698: rmdir d0/d7a 39 2024-04-26T04:53:02.112 INFO:tasks.workunit.client.1.smithi078.stdout:2/717: dwrite d8/d11/f74 [0,4194304] 0 2024-04-26T04:53:02.116 INFO:tasks.workunit.client.0.smithi077.stdout:4/320: mknod d4/d17/d18/d43/d6e/d6c/c6f 0 2024-04-26T04:53:02.118 INFO:tasks.workunit.client.1.smithi078.stdout:1/769: dwrite d7/d1e/d9e/db/d47/f73 [0,4194304] 0 2024-04-26T04:53:02.121 INFO:tasks.workunit.client.0.smithi077.stdout:3/302: rename d0/d9/f25 to d0/d7/d1f/d4d/f5f 0 2024-04-26T04:53:02.123 INFO:tasks.workunit.client.0.smithi077.stdout:4/321: dread f2 [4194304,4194304] 0 2024-04-26T04:53:02.125 INFO:tasks.workunit.client.0.smithi077.stdout:4/322: chown d4/de/d15/f40 118 1 2024-04-26T04:53:02.127 INFO:tasks.workunit.client.0.smithi077.stdout:4/323: dread - d4/d17/d18/d43/d53/d64/f65 zero size 2024-04-26T04:53:02.129 INFO:tasks.workunit.client.0.smithi077.stdout:8/417: rmdir d4/d17 39 2024-04-26T04:53:02.131 INFO:tasks.workunit.client.0.smithi077.stdout:6/423: mkdir d8/d8f 0 2024-04-26T04:53:02.133 INFO:tasks.workunit.client.1.smithi078.stdout:7/695: dread d9/da/f66 [0,4194304] 0 2024-04-26T04:53:02.135 INFO:tasks.workunit.client.0.smithi077.stdout:6/424: dread d8/fe [4194304,4194304] 0 2024-04-26T04:53:02.139 INFO:tasks.workunit.client.0.smithi077.stdout:7/396: rename da/d13/f5c to da/d13/d22/d7d/d80/f8b 0 2024-04-26T04:53:02.140 INFO:tasks.workunit.client.0.smithi077.stdout:7/397: truncate da/f87 440540 0 2024-04-26T04:53:02.141 INFO:tasks.workunit.client.0.smithi077.stdout:7/398: read da/d15/d1a/f65 [88691,9267] 0 2024-04-26T04:53:02.141 INFO:tasks.workunit.client.1.smithi078.stdout:3/750: truncate d16/f8d 2189841 0 2024-04-26T04:53:02.143 INFO:tasks.workunit.client.1.smithi078.stdout:3/751: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f68 [4806063,24269] 0 2024-04-26T04:53:02.143 INFO:tasks.workunit.client.0.smithi077.stdout:2/398: dread f5 [0,4194304] 0 2024-04-26T04:53:02.143 INFO:tasks.workunit.client.0.smithi077.stdout:2/399: chown d9/f72 32599 1 2024-04-26T04:53:02.144 INFO:tasks.workunit.client.1.smithi078.stdout:2/718: unlink d8/d11/l97 0 2024-04-26T04:53:02.145 INFO:tasks.workunit.client.1.smithi078.stdout:2/719: write d8/db7/fe2 [566242,105888] 0 2024-04-26T04:53:02.145 INFO:tasks.workunit.client.0.smithi077.stdout:1/394: mkdir d9/db/d2b/d75 0 2024-04-26T04:53:02.147 INFO:tasks.workunit.client.0.smithi077.stdout:1/395: chown d9/db/dd/f2f 27663 1 2024-04-26T04:53:02.148 INFO:tasks.workunit.client.1.smithi078.stdout:1/770: unlink d7/d1e/d9e/d4c/d3d/lda 0 2024-04-26T04:53:02.149 INFO:tasks.workunit.client.0.smithi077.stdout:2/400: dread d9/dc/d1d/d34/d39/d63/d79/f3f [0,4194304] 0 2024-04-26T04:53:02.149 INFO:tasks.workunit.client.0.smithi077.stdout:8/418: dread d4/f5 [0,4194304] 0 2024-04-26T04:53:02.152 INFO:tasks.workunit.client.1.smithi078.stdout:0/748: dread d2/d9/d18/f13 [0,4194304] 0 2024-04-26T04:53:02.152 INFO:tasks.workunit.client.1.smithi078.stdout:5/675: link d7/d18/d1c/d5a/d96/fdd d7/da/d1e/d91/dd6/fe2 0 2024-04-26T04:53:02.158 INFO:tasks.workunit.client.1.smithi078.stdout:7/696: creat d9/da/d33/d4b/d4c/fe1 x:0 0 0 2024-04-26T04:53:02.159 INFO:tasks.workunit.client.1.smithi078.stdout:7/697: dread - d9/da/d11/d13/d1a/d52/fca zero size 2024-04-26T04:53:02.159 INFO:tasks.workunit.client.0.smithi077.stdout:9/349: write d9/dd/d1b/f23 [3051608,103583] 0 2024-04-26T04:53:02.161 INFO:tasks.workunit.client.1.smithi078.stdout:9/670: rename dc/df/d27/l3f to dc/df/d4b/d58/lc7 0 2024-04-26T04:53:02.162 INFO:tasks.workunit.client.1.smithi078.stdout:4/755: truncate d0/d1d/de1/d20/d31/d47/db2/d44/d69/f7a 1225687 0 2024-04-26T04:53:02.163 INFO:tasks.workunit.client.1.smithi078.stdout:4/756: readlink d0/de/d22/d27/d56/d5a/l86 0 2024-04-26T04:53:02.165 INFO:tasks.workunit.client.0.smithi077.stdout:8/419: dread d4/df/d22/d2d/f3d [0,4194304] 0 2024-04-26T04:53:02.167 INFO:tasks.workunit.client.0.smithi077.stdout:5/406: rename d1/d12/d28/l5b to d1/d8/d72/l94 0 2024-04-26T04:53:02.168 INFO:tasks.workunit.client.1.smithi078.stdout:3/752: symlink d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/l10a 0 2024-04-26T04:53:02.169 INFO:tasks.workunit.client.1.smithi078.stdout:2/720: truncate d8/d11/d14/d69/d8f/d34/dd3/de4/d52/f9f 1542395 0 2024-04-26T04:53:02.171 INFO:tasks.workunit.client.0.smithi077.stdout:7/399: rmdir da/d13/d22/d5d 39 2024-04-26T04:53:02.175 INFO:tasks.workunit.client.1.smithi078.stdout:2/721: dwrite d8/d11/f45 [0,4194304] 0 2024-04-26T04:53:02.176 INFO:tasks.workunit.client.0.smithi077.stdout:1/396: dread d9/f44 [0,4194304] 0 2024-04-26T04:53:02.177 INFO:tasks.workunit.client.1.smithi078.stdout:5/676: dread - d7/d18/d32/f60 zero size 2024-04-26T04:53:02.179 INFO:tasks.workunit.client.1.smithi078.stdout:8/699: creat d0/fe4 x:0 0 0 2024-04-26T04:53:02.182 INFO:tasks.workunit.client.0.smithi077.stdout:4/324: dwrite d4/de/d15/f40 [0,4194304] 0 2024-04-26T04:53:02.186 INFO:tasks.workunit.client.0.smithi077.stdout:9/350: mkdir d9/dd/d1b/d2b/d32/d36/d7f 0 2024-04-26T04:53:02.187 INFO:tasks.workunit.client.0.smithi077.stdout:9/351: dread - d9/dd/d1b/d2b/f2f zero size 2024-04-26T04:53:02.187 INFO:tasks.workunit.client.1.smithi078.stdout:9/671: mkdir dc/df/d17/dc8 0 2024-04-26T04:53:02.189 INFO:tasks.workunit.client.0.smithi077.stdout:6/425: link d8/d1c/f4c d8/d55/f90 0 2024-04-26T04:53:02.189 INFO:tasks.workunit.client.1.smithi078.stdout:9/672: write dc/df/d4b/d22/d2a/d57/f90 [1548324,106857] 0 2024-04-26T04:53:02.192 INFO:tasks.workunit.client.0.smithi077.stdout:9/352: dread d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:02.195 INFO:tasks.workunit.client.0.smithi077.stdout:8/420: write d4/df/d22/d2d/d2b/f5d [4360323,85235] 0 2024-04-26T04:53:02.195 INFO:tasks.workunit.client.0.smithi077.stdout:8/421: stat d4/fe 0 2024-04-26T04:53:02.196 INFO:tasks.workunit.client.1.smithi078.stdout:0/749: dread d2/d9/f3e [0,4194304] 0 2024-04-26T04:53:02.196 INFO:tasks.workunit.client.1.smithi078.stdout:3/753: fsync d16/d26/d2b/d31/d71/d8a/dbc/fbd 0 2024-04-26T04:53:02.199 INFO:tasks.workunit.client.0.smithi077.stdout:5/407: creat d1/d8/f95 x:0 0 0 2024-04-26T04:53:02.200 INFO:tasks.workunit.client.1.smithi078.stdout:1/771: creat d7/d1e/d9e/db/d47/da5/ffa x:0 0 0 2024-04-26T04:53:02.205 INFO:tasks.workunit.client.1.smithi078.stdout:5/677: truncate d7/d18/f2e 4411490 0 2024-04-26T04:53:02.207 INFO:tasks.workunit.client.1.smithi078.stdout:7/698: write d9/da/d11/d13/d50/d8a/d8f/fd0 [3526206,25166] 0 2024-04-26T04:53:02.208 INFO:tasks.workunit.client.1.smithi078.stdout:7/699: chown d9/cb3 57466 1 2024-04-26T04:53:02.208 INFO:tasks.workunit.client.1.smithi078.stdout:8/700: write d0/d19/d25/d36/d58/fb7 [6417,60846] 0 2024-04-26T04:53:02.208 INFO:tasks.workunit.client.1.smithi078.stdout:7/700: chown d9/fbc 4205801 1 2024-04-26T04:53:02.210 INFO:tasks.workunit.client.0.smithi077.stdout:4/325: symlink d4/d17/d18/d43/d53/d64/l70 0 2024-04-26T04:53:02.222 INFO:tasks.workunit.client.1.smithi078.stdout:3/754: creat d16/d26/d2e/f10b x:0 0 0 2024-04-26T04:53:02.223 INFO:tasks.workunit.client.0.smithi077.stdout:3/303: getdents d0/d7/d1f/d4d 0 2024-04-26T04:53:02.226 INFO:tasks.workunit.client.1.smithi078.stdout:1/772: symlink d7/d1e/d9e/db/d37/d58/d5c/lfb 0 2024-04-26T04:53:02.228 INFO:tasks.workunit.client.0.smithi077.stdout:6/426: dwrite d8/d2a/f66 [0,4194304] 0 2024-04-26T04:53:02.228 INFO:tasks.workunit.client.1.smithi078.stdout:4/757: dwrite d0/de/d22/f98 [8388608,4194304] 0 2024-04-26T04:53:02.228 INFO:tasks.workunit.client.1.smithi078.stdout:1/773: readlink d7/d1e/d9e/db/d49/d93/lce 0 2024-04-26T04:53:02.228 INFO:tasks.workunit.client.1.smithi078.stdout:2/722: getdents d8/d11/d18/d5d/d61/df5 0 2024-04-26T04:53:02.229 INFO:tasks.workunit.client.1.smithi078.stdout:5/678: creat d7/d18/d1c/d5a/d96/fe3 x:0 0 0 2024-04-26T04:53:02.231 INFO:tasks.workunit.client.1.smithi078.stdout:7/701: creat d9/d57/fe2 x:0 0 0 2024-04-26T04:53:02.232 INFO:tasks.workunit.client.1.smithi078.stdout:7/702: fsync d9/da/d11/d13/d1a/d52/d5a/d41/d68/f7b 0 2024-04-26T04:53:02.234 INFO:tasks.workunit.client.0.smithi077.stdout:7/400: dwrite da/ff [0,4194304] 0 2024-04-26T04:53:02.236 INFO:tasks.workunit.client.0.smithi077.stdout:8/422: link d4/fd d4/df/d22/d2d/d2b/d33/f92 0 2024-04-26T04:53:02.239 INFO:tasks.workunit.client.0.smithi077.stdout:2/401: rename d9/ff to d9/dc/d1d/d34/f84 0 2024-04-26T04:53:02.240 INFO:tasks.workunit.client.0.smithi077.stdout:5/408: symlink d1/d12/l96 0 2024-04-26T04:53:02.241 INFO:tasks.workunit.client.1.smithi078.stdout:5/679: dread f5 [0,4194304] 0 2024-04-26T04:53:02.242 INFO:tasks.workunit.client.0.smithi077.stdout:3/304: rmdir d0/d7/d1f/d4d 39 2024-04-26T04:53:02.244 INFO:tasks.workunit.client.0.smithi077.stdout:1/397: rmdir d9/db/dd/d34/d4a/d6c 0 2024-04-26T04:53:02.246 INFO:tasks.workunit.client.1.smithi078.stdout:2/723: mknod d8/d53/cfb 0 2024-04-26T04:53:02.248 INFO:tasks.workunit.client.1.smithi078.stdout:2/724: dread - d8/ff2 zero size 2024-04-26T04:53:02.249 INFO:tasks.workunit.client.1.smithi078.stdout:2/725: stat d8/d11/d18/d5d/d61 0 2024-04-26T04:53:02.250 INFO:tasks.workunit.client.1.smithi078.stdout:5/680: dwrite d7/d18/d32/d37/d79/fad [0,4194304] 0 2024-04-26T04:53:02.250 INFO:tasks.workunit.client.0.smithi077.stdout:9/353: link d9/d3a/l7b d9/dd/d3d/l80 0 2024-04-26T04:53:02.251 INFO:tasks.workunit.client.0.smithi077.stdout:1/398: dwrite d9/f1b [4194304,4194304] 0 2024-04-26T04:53:02.252 INFO:tasks.workunit.client.0.smithi077.stdout:1/399: write d9/db/dd/f74 [982861,113328] 0 2024-04-26T04:53:02.259 INFO:tasks.workunit.client.1.smithi078.stdout:9/673: creat dc/df/d4b/d22/fc9 x:0 0 0 2024-04-26T04:53:02.263 INFO:tasks.workunit.client.0.smithi077.stdout:2/402: mknod d9/dc/d1d/d34/d39/d63/d79/d2b/c85 0 2024-04-26T04:53:02.264 INFO:tasks.workunit.client.0.smithi077.stdout:2/403: chown d9/dc/d5f/l6d 31361387 1 2024-04-26T04:53:02.265 INFO:tasks.workunit.client.1.smithi078.stdout:6/797: sync 2024-04-26T04:53:02.267 INFO:tasks.workunit.client.1.smithi078.stdout:3/755: symlink d16/d26/d2b/l10c 0 2024-04-26T04:53:02.269 INFO:tasks.workunit.client.1.smithi078.stdout:4/758: mknod d0/d1d/de1/c10f 0 2024-04-26T04:53:02.270 INFO:tasks.workunit.client.1.smithi078.stdout:4/759: write d0/de/f7e [570495,81826] 0 2024-04-26T04:53:02.271 INFO:tasks.workunit.client.1.smithi078.stdout:1/774: mkdir d7/d1e/d9e/dfc 0 2024-04-26T04:53:02.275 INFO:tasks.workunit.client.1.smithi078.stdout:7/703: mknod d9/da/d20/ce3 0 2024-04-26T04:53:02.276 INFO:tasks.workunit.client.0.smithi077.stdout:1/400: read d9/db/dd/d34/f51 [103977,107184] 0 2024-04-26T04:53:02.277 INFO:tasks.workunit.client.1.smithi078.stdout:9/674: creat dc/d55/fca x:0 0 0 2024-04-26T04:53:02.278 INFO:tasks.workunit.client.0.smithi077.stdout:8/423: mknod d4/df/d22/d75/c93 0 2024-04-26T04:53:02.278 INFO:tasks.workunit.client.0.smithi077.stdout:1/401: readlink d9/db/d2b/d5c/l67 0 2024-04-26T04:53:02.279 INFO:tasks.workunit.client.1.smithi078.stdout:0/750: getdents d2/d1e 0 2024-04-26T04:53:02.279 INFO:tasks.workunit.client.1.smithi078.stdout:0/751: chown d2/f12 33800 1 2024-04-26T04:53:02.280 INFO:tasks.workunit.client.1.smithi078.stdout:0/752: write d2/d9/d60/d6a/f87 [2578459,84397] 0 2024-04-26T04:53:02.284 INFO:tasks.workunit.client.0.smithi077.stdout:7/401: dwrite da/d13/d22/d5d/f60 [0,4194304] 0 2024-04-26T04:53:02.284 INFO:tasks.workunit.client.1.smithi078.stdout:3/756: fdatasync d16/d26/d2e/d5f/d49/f4b 0 2024-04-26T04:53:02.287 INFO:tasks.workunit.client.0.smithi077.stdout:5/409: dwrite d1/d42/d62/d6b/f85 [0,4194304] 0 2024-04-26T04:53:02.288 INFO:tasks.workunit.client.1.smithi078.stdout:3/757: dread d16/d26/d2b/d31/f93 [0,4194304] 0 2024-04-26T04:53:02.291 INFO:tasks.workunit.client.0.smithi077.stdout:1/402: dread d9/db/d2b/d3f/f50 [0,4194304] 0 2024-04-26T04:53:02.293 INFO:tasks.workunit.client.0.smithi077.stdout:3/305: unlink d0/f42 0 2024-04-26T04:53:02.294 INFO:tasks.workunit.client.0.smithi077.stdout:1/403: readlink d9/db/dd/l70 0 2024-04-26T04:53:02.296 INFO:tasks.workunit.client.1.smithi078.stdout:2/726: write d8/d53/f71 [161933,47967] 0 2024-04-26T04:53:02.297 INFO:tasks.workunit.client.0.smithi077.stdout:1/404: write d9/db/f60 [155668,122190] 0 2024-04-26T04:53:02.297 INFO:tasks.workunit.client.0.smithi077.stdout:4/326: getdents d4/d17/d18/d43/d51 0 2024-04-26T04:53:02.298 INFO:tasks.workunit.client.1.smithi078.stdout:2/727: stat d8/d11/d14/d69/d8f/f5c 0 2024-04-26T04:53:02.298 INFO:tasks.workunit.client.0.smithi077.stdout:2/404: dread d9/dc/d1d/d34/d39/d63/d79/d2b/f46 [4194304,4194304] 0 2024-04-26T04:53:02.298 INFO:tasks.workunit.client.1.smithi078.stdout:4/760: mkdir d0/de/d22/d27/d56/d5a/d110 0 2024-04-26T04:53:02.299 INFO:tasks.workunit.client.0.smithi077.stdout:7/402: dwrite da/f89 [0,4194304] 0 2024-04-26T04:53:02.299 INFO:tasks.workunit.client.0.smithi077.stdout:7/403: readlink da/l4c 0 2024-04-26T04:53:02.301 INFO:tasks.workunit.client.0.smithi077.stdout:6/427: creat d8/d2a/f91 x:0 0 0 2024-04-26T04:53:02.302 INFO:tasks.workunit.client.0.smithi077.stdout:6/428: write d8/d1c/f79 [659756,76422] 0 2024-04-26T04:53:02.303 INFO:tasks.workunit.client.1.smithi078.stdout:1/775: creat d7/ddd/ffd x:0 0 0 2024-04-26T04:53:02.305 INFO:tasks.workunit.client.1.smithi078.stdout:6/798: write d2/de/d27/da9/da8/fb7 [142013,90169] 0 2024-04-26T04:53:02.307 INFO:tasks.workunit.client.1.smithi078.stdout:8/701: sync 2024-04-26T04:53:02.309 INFO:tasks.workunit.client.1.smithi078.stdout:7/704: mknod d9/da/d11/d13/d1a/d65/ce4 0 2024-04-26T04:53:02.310 INFO:tasks.workunit.client.1.smithi078.stdout:9/675: read dc/f60 [173156,28908] 0 2024-04-26T04:53:02.318 INFO:tasks.workunit.client.1.smithi078.stdout:9/676: dread dc/f39 [4194304,4194304] 0 2024-04-26T04:53:02.318 INFO:tasks.workunit.client.0.smithi077.stdout:3/306: chown d0/d7/d1f/d4d/f56 25994184 1 2024-04-26T04:53:02.318 INFO:tasks.workunit.client.0.smithi077.stdout:4/327: mknod d4/de/d15/d1f/c71 0 2024-04-26T04:53:02.319 INFO:tasks.workunit.client.1.smithi078.stdout:3/758: mknod d16/d26/d2e/c10d 0 2024-04-26T04:53:02.320 INFO:tasks.workunit.client.0.smithi077.stdout:5/410: dwrite d1/d12/f50 [0,4194304] 0 2024-04-26T04:53:02.321 INFO:tasks.workunit.client.0.smithi077.stdout:7/404: mkdir da/d15/d42/d8c 0 2024-04-26T04:53:02.323 INFO:tasks.workunit.client.1.smithi078.stdout:2/728: rmdir d8/d11/d18/d5d/d8b/db0 39 2024-04-26T04:53:02.328 INFO:tasks.workunit.client.0.smithi077.stdout:6/429: rmdir d8/d2a/d57 39 2024-04-26T04:53:02.331 INFO:tasks.workunit.client.1.smithi078.stdout:6/799: symlink d2/de/d27/da9/da8/lfc 0 2024-04-26T04:53:02.331 INFO:tasks.workunit.client.1.smithi078.stdout:5/681: link d7/d48/d57/d65/d46/f54 d7/da/d1e/d91/dd6/fe4 0 2024-04-26T04:53:02.333 INFO:tasks.workunit.client.1.smithi078.stdout:7/705: creat d9/da/d33/d4b/d4c/fe5 x:0 0 0 2024-04-26T04:53:02.334 INFO:tasks.workunit.client.0.smithi077.stdout:4/328: chown d4/de/d15/d1f/d35/c42 1572 1 2024-04-26T04:53:02.337 INFO:tasks.workunit.client.0.smithi077.stdout:7/405: mkdir da/d13/d22/d5d/d8d 0 2024-04-26T04:53:02.338 INFO:tasks.workunit.client.0.smithi077.stdout:8/424: write d4/f42 [558903,17156] 0 2024-04-26T04:53:02.342 INFO:tasks.workunit.client.0.smithi077.stdout:6/430: symlink d8/d1c/l92 0 2024-04-26T04:53:02.343 INFO:tasks.workunit.client.1.smithi078.stdout:2/729: readlink d8/d11/d14/d69/d8f/l44 0 2024-04-26T04:53:02.344 INFO:tasks.workunit.client.0.smithi077.stdout:6/431: truncate d8/d2a/f91 945397 0 2024-04-26T04:53:02.344 INFO:tasks.workunit.client.0.smithi077.stdout:7/406: dwrite da/ff [0,4194304] 0 2024-04-26T04:53:02.346 INFO:tasks.workunit.client.0.smithi077.stdout:9/354: rename c1 to d9/c81 0 2024-04-26T04:53:02.346 INFO:tasks.workunit.client.0.smithi077.stdout:9/355: stat d9/dd/d1b/d4e/f6a 0 2024-04-26T04:53:02.347 INFO:tasks.workunit.client.0.smithi077.stdout:9/356: chown d9/d43/f56 55 1 2024-04-26T04:53:02.348 INFO:tasks.workunit.client.1.smithi078.stdout:1/776: mknod d7/d1e/d9e/dfc/cfe 0 2024-04-26T04:53:02.348 INFO:tasks.workunit.client.0.smithi077.stdout:9/357: write d9/d45/f71 [905625,9661] 0 2024-04-26T04:53:02.350 INFO:tasks.workunit.client.0.smithi077.stdout:9/358: truncate d9/d45/f7e 365035 0 2024-04-26T04:53:02.351 INFO:tasks.workunit.client.1.smithi078.stdout:1/777: truncate d7/d1e/d9e/db/d49/fd8 656332 0 2024-04-26T04:53:02.355 INFO:tasks.workunit.client.1.smithi078.stdout:7/706: mkdir d9/da/d33/d4b/d4c/d5b/d83/de6 0 2024-04-26T04:53:02.356 INFO:tasks.workunit.client.0.smithi077.stdout:8/425: creat d4/df/d22/d2d/f94 x:0 0 0 2024-04-26T04:53:02.356 INFO:tasks.workunit.client.1.smithi078.stdout:6/800: dread d2/de/d27/da9/d3a/d69/daa/fea [0,4194304] 0 2024-04-26T04:53:02.357 INFO:tasks.workunit.client.0.smithi077.stdout:8/426: chown d4/df/d18/d56/f72 10 1 2024-04-26T04:53:02.357 INFO:tasks.workunit.client.1.smithi078.stdout:0/753: dwrite d2/d9/d31/fce [0,4194304] 0 2024-04-26T04:53:02.359 INFO:tasks.workunit.client.0.smithi077.stdout:6/432: read d8/d1c/f3e [130867,117589] 0 2024-04-26T04:53:02.361 INFO:tasks.workunit.client.1.smithi078.stdout:2/730: dread d8/d11/f31 [4194304,4194304] 0 2024-04-26T04:53:02.366 INFO:tasks.workunit.client.0.smithi077.stdout:2/405: dwrite d9/dc/d1d/d34/d39/d63/d79/f1c [0,4194304] 0 2024-04-26T04:53:02.367 INFO:tasks.workunit.client.0.smithi077.stdout:5/411: link d1/d8/l3f d1/d12/d28/l97 0 2024-04-26T04:53:02.370 INFO:tasks.workunit.client.0.smithi077.stdout:6/433: dwrite d8/d2a/f63 [0,4194304] 0 2024-04-26T04:53:02.371 INFO:tasks.workunit.client.0.smithi077.stdout:6/434: chown d8/d1c/c61 31 1 2024-04-26T04:53:02.371 INFO:tasks.workunit.client.0.smithi077.stdout:4/329: link d4/d17/l36 d4/d17/d18/l72 0 2024-04-26T04:53:02.372 INFO:tasks.workunit.client.1.smithi078.stdout:4/761: write d0/d1d/de1/d20/f2b [46600,126573] 0 2024-04-26T04:53:02.373 INFO:tasks.workunit.client.0.smithi077.stdout:3/307: truncate d0/d9/d32/d2a/f4c 2726714 0 2024-04-26T04:53:02.374 INFO:tasks.workunit.client.0.smithi077.stdout:7/407: dread da/f36 [0,4194304] 0 2024-04-26T04:53:02.374 INFO:tasks.workunit.client.0.smithi077.stdout:3/308: stat d0/d9/d32/d29/f31 0 2024-04-26T04:53:02.378 INFO:tasks.workunit.client.1.smithi078.stdout:7/707: fsync d9/da/f66 0 2024-04-26T04:53:02.379 INFO:tasks.workunit.client.1.smithi078.stdout:5/682: write d7/d48/d57/fcf [1087368,45209] 0 2024-04-26T04:53:02.380 INFO:tasks.workunit.client.1.smithi078.stdout:7/708: write d9/da/d33/fd7 [94295,31432] 0 2024-04-26T04:53:02.381 INFO:tasks.workunit.client.0.smithi077.stdout:2/406: symlink d9/d15/l86 0 2024-04-26T04:53:02.383 INFO:tasks.workunit.client.1.smithi078.stdout:6/801: creat d2/d1a/d38/dd5/ffd x:0 0 0 2024-04-26T04:53:02.384 INFO:tasks.workunit.client.0.smithi077.stdout:5/412: symlink d1/d12/d28/l98 0 2024-04-26T04:53:02.384 INFO:tasks.workunit.client.1.smithi078.stdout:8/702: truncate d0/d19/d25/d32/d8a/db1/f88 2826383 0 2024-04-26T04:53:02.385 INFO:tasks.workunit.client.1.smithi078.stdout:0/754: mknod d2/d9/d18/d7d/da9/dab/cf8 0 2024-04-26T04:53:02.386 INFO:tasks.workunit.client.0.smithi077.stdout:6/435: truncate d8/f27 125116 0 2024-04-26T04:53:02.387 INFO:tasks.workunit.client.1.smithi078.stdout:2/731: mkdir d8/d11/d14/d69/d8f/d34/dd3/dd9/dfc 0 2024-04-26T04:53:02.387 INFO:tasks.workunit.client.0.smithi077.stdout:6/436: write d8/d1c/f5c [4905041,30313] 0 2024-04-26T04:53:02.388 INFO:tasks.workunit.client.1.smithi078.stdout:9/677: link dc/df/d17/l50 dc/df/d4b/d22/d2a/lcb 0 2024-04-26T04:53:02.391 INFO:tasks.workunit.client.1.smithi078.stdout:8/703: dwrite d0/fe4 [0,4194304] 0 2024-04-26T04:53:02.392 INFO:tasks.workunit.client.1.smithi078.stdout:8/704: chown d0/d19/d25/fd6 106 1 2024-04-26T04:53:02.395 INFO:tasks.workunit.client.1.smithi078.stdout:4/762: symlink d0/de/d22/d27/d56/l111 0 2024-04-26T04:53:02.395 INFO:tasks.workunit.client.1.smithi078.stdout:9/678: dread dc/df/d27/d81/f87 [0,4194304] 0 2024-04-26T04:53:02.395 INFO:tasks.workunit.client.0.smithi077.stdout:9/359: rename d9/dd/f52 to d9/f82 0 2024-04-26T04:53:02.396 INFO:tasks.workunit.client.1.smithi078.stdout:8/705: read d0/d19/d52/d35/d44/d6b/f9b [133967,7074] 0 2024-04-26T04:53:02.397 INFO:tasks.workunit.client.0.smithi077.stdout:9/360: write d9/dd/d1b/d2b/d32/d36/f6d [4777477,85868] 0 2024-04-26T04:53:02.398 INFO:tasks.workunit.client.0.smithi077.stdout:9/361: fsync d9/d43/f61 0 2024-04-26T04:53:02.399 INFO:tasks.workunit.client.1.smithi078.stdout:1/778: mknod d7/d1e/d9e/db/d37/d58/cff 0 2024-04-26T04:53:02.400 INFO:tasks.workunit.client.1.smithi078.stdout:3/759: sync 2024-04-26T04:53:02.402 INFO:tasks.workunit.client.1.smithi078.stdout:1/779: write d7/d1e/d9e/dd/d20/fb0 [1695934,31757] 0 2024-04-26T04:53:02.404 INFO:tasks.workunit.client.1.smithi078.stdout:8/706: dwrite d0/d19/f2a [0,4194304] 0 2024-04-26T04:53:02.407 INFO:tasks.workunit.client.1.smithi078.stdout:6/802: mkdir d2/de/d27/da9/da8/ddc/dfe 0 2024-04-26T04:53:02.409 INFO:tasks.workunit.client.1.smithi078.stdout:3/760: dread d16/d26/d2e/f4d [0,4194304] 0 2024-04-26T04:53:02.410 INFO:tasks.workunit.client.1.smithi078.stdout:1/780: dwrite d7/f13 [0,4194304] 0 2024-04-26T04:53:02.430 INFO:tasks.workunit.client.1.smithi078.stdout:8/707: readlink d0/d19/d25/ld2 0 2024-04-26T04:53:02.431 INFO:tasks.workunit.client.1.smithi078.stdout:8/708: chown d0/d19/d52/d35/cb0 29 1 2024-04-26T04:53:02.432 INFO:tasks.workunit.client.1.smithi078.stdout:0/755: dwrite d2/d9/d60/f68 [0,4194304] 0 2024-04-26T04:53:02.432 INFO:tasks.workunit.client.1.smithi078.stdout:2/732: readlink d8/d11/d14/d5b/l63 0 2024-04-26T04:53:02.434 INFO:tasks.workunit.client.1.smithi078.stdout:2/733: stat d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/d91/cc9 0 2024-04-26T04:53:02.437 INFO:tasks.workunit.client.0.smithi077.stdout:7/408: write da/d15/f48 [23230,40859] 0 2024-04-26T04:53:02.437 INFO:tasks.workunit.client.0.smithi077.stdout:9/362: creat d9/dd/d1b/d2b/f83 x:0 0 0 2024-04-26T04:53:02.441 INFO:tasks.workunit.client.0.smithi077.stdout:5/413: dread d1/d8/ff [0,4194304] 0 2024-04-26T04:53:02.441 INFO:tasks.workunit.client.1.smithi078.stdout:7/709: creat d9/da/d11/fe7 x:0 0 0 2024-04-26T04:53:02.442 INFO:tasks.workunit.client.1.smithi078.stdout:7/710: read d9/da/d11/d13/d1a/d52/fb5 [23816,2712] 0 2024-04-26T04:53:02.443 INFO:tasks.workunit.client.0.smithi077.stdout:7/409: chown da/d13/d22/f88 315 1 2024-04-26T04:53:02.446 INFO:tasks.workunit.client.1.smithi078.stdout:5/683: truncate d7/d48/d57/d65/fb0 354709 0 2024-04-26T04:53:02.447 INFO:tasks.workunit.client.1.smithi078.stdout:5/684: readlink d7/da/d1e/l20 0 2024-04-26T04:53:02.447 INFO:tasks.workunit.client.1.smithi078.stdout:1/781: write d7/f8a [3695669,55707] 0 2024-04-26T04:53:02.448 INFO:tasks.workunit.client.0.smithi077.stdout:4/330: dwrite d4/f39 [0,4194304] 0 2024-04-26T04:53:02.449 INFO:tasks.workunit.client.1.smithi078.stdout:5/685: readlink d7/da/d1e/lb3 0 2024-04-26T04:53:02.450 INFO:tasks.workunit.client.0.smithi077.stdout:4/331: dread - d4/de/d15/f6d zero size 2024-04-26T04:53:02.450 INFO:tasks.workunit.client.0.smithi077.stdout:2/407: link d9/dc/d1d/d34/d39/d63/d79/d2b/f46 d9/dc/d1d/d34/f87 0 2024-04-26T04:53:02.450 INFO:tasks.workunit.client.0.smithi077.stdout:3/309: dwrite d0/d9/f30 [4194304,4194304] 0 2024-04-26T04:53:02.451 INFO:tasks.workunit.client.1.smithi078.stdout:7/711: dwrite d9/da/d11/d13/d50/d8a/d8f/fd0 [8388608,4194304] 0 2024-04-26T04:53:02.454 INFO:tasks.workunit.client.0.smithi077.stdout:5/414: creat d1/d8/d72/f99 x:0 0 0 2024-04-26T04:53:02.456 INFO:tasks.workunit.client.0.smithi077.stdout:6/437: link d8/d52/c5f d8/d2a/d86/c93 0 2024-04-26T04:53:02.456 INFO:tasks.workunit.client.0.smithi077.stdout:5/415: read d1/d12/d45/f63 [3566492,129643] 0 2024-04-26T04:53:02.457 INFO:tasks.workunit.client.0.smithi077.stdout:5/416: rename d1/d8c to d1/d8c/d9a 22 2024-04-26T04:53:02.457 INFO:tasks.workunit.client.0.smithi077.stdout:5/417: stat d1/d12/l52 0 2024-04-26T04:53:02.463 INFO:tasks.workunit.client.1.smithi078.stdout:4/763: dwrite d0/de/d22/d27/d56/f60 [0,4194304] 0 2024-04-26T04:53:02.464 INFO:tasks.workunit.client.0.smithi077.stdout:7/410: mknod da/d13/d22/d7d/d80/c8e 0 2024-04-26T04:53:02.464 INFO:tasks.workunit.client.0.smithi077.stdout:2/408: dread d9/dc/d1d/d34/d39/f68 [0,4194304] 0 2024-04-26T04:53:02.465 INFO:tasks.workunit.client.1.smithi078.stdout:4/764: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa 11758 1 2024-04-26T04:53:02.465 INFO:tasks.workunit.client.1.smithi078.stdout:6/803: write d2/d1a/f6 [6029147,13440] 0 2024-04-26T04:53:02.467 INFO:tasks.workunit.client.1.smithi078.stdout:4/765: chown d0/de/d22/d27/d56/d5a 22145560 1 2024-04-26T04:53:02.468 INFO:tasks.workunit.client.1.smithi078.stdout:8/709: truncate d0/f23 265227 0 2024-04-26T04:53:02.468 INFO:tasks.workunit.client.1.smithi078.stdout:6/804: chown d2/de/d27/d58/l87 20962 1 2024-04-26T04:53:02.471 INFO:tasks.workunit.client.1.smithi078.stdout:9/679: getdents dc/df/d27/d81/da8 0 2024-04-26T04:53:02.472 INFO:tasks.workunit.client.1.smithi078.stdout:1/782: dread d7/fc [0,4194304] 0 2024-04-26T04:53:02.473 INFO:tasks.workunit.client.0.smithi077.stdout:9/363: dread d9/f22 [8388608,4194304] 0 2024-04-26T04:53:02.473 INFO:tasks.workunit.client.1.smithi078.stdout:2/734: creat d8/d11/d18/d5d/d8b/db0/ffd x:0 0 0 2024-04-26T04:53:02.474 INFO:tasks.workunit.client.1.smithi078.stdout:5/686: rmdir d7/d11 39 2024-04-26T04:53:02.475 INFO:tasks.workunit.client.1.smithi078.stdout:5/687: dread - d7/da/d1e/d91/dd6/fe2 zero size 2024-04-26T04:53:02.477 INFO:tasks.workunit.client.0.smithi077.stdout:7/411: mknod da/d15/d1a/c8f 0 2024-04-26T04:53:02.480 INFO:tasks.workunit.client.0.smithi077.stdout:9/364: write d9/dd/d1b/d2b/f2f [725572,52876] 0 2024-04-26T04:53:02.481 INFO:tasks.workunit.client.0.smithi077.stdout:3/310: creat d0/d7/d1f/f60 x:0 0 0 2024-04-26T04:53:02.481 INFO:tasks.workunit.client.0.smithi077.stdout:9/365: chown d9/d43 133555713 1 2024-04-26T04:53:02.482 INFO:tasks.workunit.client.1.smithi078.stdout:1/783: mknod d7/d1e/d9e/db/d49/c100 0 2024-04-26T04:53:02.483 INFO:tasks.workunit.client.0.smithi077.stdout:2/409: link d9/dc/d1d/d1e/c4d d9/dc/d1d/d34/d39/d63/d79/c88 0 2024-04-26T04:53:02.485 INFO:tasks.workunit.client.0.smithi077.stdout:7/412: dwrite da/f68 [0,4194304] 0 2024-04-26T04:53:02.488 INFO:tasks.workunit.client.0.smithi077.stdout:9/366: dread d9/dd/d3d/f3e [0,4194304] 0 2024-04-26T04:53:02.490 INFO:tasks.workunit.client.1.smithi078.stdout:5/688: creat d7/d18/d1c/d5a/fe5 x:0 0 0 2024-04-26T04:53:02.490 INFO:tasks.workunit.client.0.smithi077.stdout:3/311: dread d0/f3d [0,4194304] 0 2024-04-26T04:53:02.491 INFO:tasks.workunit.client.1.smithi078.stdout:3/761: sync 2024-04-26T04:53:02.491 INFO:tasks.workunit.client.0.smithi077.stdout:2/410: symlink d9/dc/d6e/l89 0 2024-04-26T04:53:02.491 INFO:tasks.workunit.client.1.smithi078.stdout:3/762: stat f6 0 2024-04-26T04:53:02.492 INFO:tasks.workunit.client.1.smithi078.stdout:7/712: rename d9/da/d11/d13/d50/l7d to d9/da/d11/d13/d1a/d52/d5a/d41/le8 0 2024-04-26T04:53:02.493 INFO:tasks.workunit.client.1.smithi078.stdout:0/756: getdents d2/d9/d78/deb 0 2024-04-26T04:53:02.493 INFO:tasks.workunit.client.0.smithi077.stdout:7/413: rmdir da/d13/d22/d34 39 2024-04-26T04:53:02.494 INFO:tasks.workunit.client.1.smithi078.stdout:3/763: dread - d16/d26/d2e/d5f/d49/dad/db0/db7/de4/ff1 zero size 2024-04-26T04:53:02.495 INFO:tasks.workunit.client.1.smithi078.stdout:6/805: symlink d2/de/d27/da9/da/d36/db6/lff 0 2024-04-26T04:53:02.499 INFO:tasks.workunit.client.1.smithi078.stdout:4/766: creat d0/f112 x:0 0 0 2024-04-26T04:53:02.501 INFO:tasks.workunit.client.1.smithi078.stdout:5/689: truncate d7/d21/f42 5928 0 2024-04-26T04:53:02.502 INFO:tasks.workunit.client.1.smithi078.stdout:6/806: dwrite d2/de/d27/d58/d5b/fd6 [0,4194304] 0 2024-04-26T04:53:02.502 INFO:tasks.workunit.client.0.smithi077.stdout:3/312: rename d0/f5c to d0/d7/d1f/d4d/f61 0 2024-04-26T04:53:02.503 INFO:tasks.workunit.client.1.smithi078.stdout:4/767: dread d0/d1d/de1/d20/d31/d47/db2/fc5 [0,4194304] 0 2024-04-26T04:53:02.505 INFO:tasks.workunit.client.1.smithi078.stdout:7/713: fdatasync d9/da/d11/d13/d1a/d52/d5a/d28/fd9 0 2024-04-26T04:53:02.505 INFO:tasks.workunit.client.1.smithi078.stdout:6/807: write d2/de/d27/da9/da/d36/fe7 [789830,36280] 0 2024-04-26T04:53:02.508 INFO:tasks.workunit.client.1.smithi078.stdout:9/680: link dc/df/d27/f13 dc/df/d4b/fcc 0 2024-04-26T04:53:02.508 INFO:tasks.workunit.client.1.smithi078.stdout:0/757: dread d2/f37 [4194304,4194304] 0 2024-04-26T04:53:02.508 INFO:tasks.workunit.client.1.smithi078.stdout:6/808: truncate d2/de/fca 866806 0 2024-04-26T04:53:02.509 INFO:tasks.workunit.client.0.smithi077.stdout:7/414: rename da/d15/c3a to da/d15/d1a/c90 0 2024-04-26T04:53:02.511 INFO:tasks.workunit.client.1.smithi078.stdout:2/735: dread d8/f4e [8388608,4194304] 0 2024-04-26T04:53:02.512 INFO:tasks.workunit.client.1.smithi078.stdout:8/710: getdents d0/d19/d52/d35/d73 0 2024-04-26T04:53:02.514 INFO:tasks.workunit.client.1.smithi078.stdout:2/736: write d8/d11/d18/d5d/d8b/db0/fda [19926,3473] 0 2024-04-26T04:53:02.515 INFO:tasks.workunit.client.0.smithi077.stdout:7/415: dwrite da/d15/f78 [0,4194304] 0 2024-04-26T04:53:02.517 INFO:tasks.workunit.client.1.smithi078.stdout:4/768: mkdir d0/d1d/de1/d20/d31/d47/db2/d113 0 2024-04-26T04:53:02.517 INFO:tasks.workunit.client.0.smithi077.stdout:4/332: write d4/d17/f3b [670070,27351] 0 2024-04-26T04:53:02.521 INFO:tasks.workunit.client.0.smithi077.stdout:4/333: symlink d4/d17/d18/l73 0 2024-04-26T04:53:02.522 INFO:tasks.workunit.client.1.smithi078.stdout:1/784: creat d7/d1e/d9e/d4c/d3d/f101 x:0 0 0 2024-04-26T04:53:02.524 INFO:tasks.workunit.client.0.smithi077.stdout:6/438: dwrite d8/f60 [0,4194304] 0 2024-04-26T04:53:02.524 INFO:tasks.workunit.client.0.smithi077.stdout:7/416: fsync da/d13/f30 0 2024-04-26T04:53:02.525 INFO:tasks.workunit.client.1.smithi078.stdout:0/758: readlink d2/d9/d18/l41 0 2024-04-26T04:53:02.528 INFO:tasks.workunit.client.0.smithi077.stdout:4/334: creat d4/d17/d18/d43/f74 x:0 0 0 2024-04-26T04:53:02.529 INFO:tasks.workunit.client.0.smithi077.stdout:6/439: dread d8/fb [0,4194304] 0 2024-04-26T04:53:02.530 INFO:tasks.workunit.client.1.smithi078.stdout:1/785: dwrite d7/d1e/d9e/db/fe9 [0,4194304] 0 2024-04-26T04:53:02.530 INFO:tasks.workunit.client.0.smithi077.stdout:6/440: fdatasync d8/d55/f76 0 2024-04-26T04:53:02.532 INFO:tasks.workunit.client.0.smithi077.stdout:6/441: write d8/d52/d53/f72 [834720,72346] 0 2024-04-26T04:53:02.536 INFO:tasks.workunit.client.0.smithi077.stdout:4/335: mkdir d4/d17/d18/d43/d53/d75 0 2024-04-26T04:53:02.540 INFO:tasks.workunit.client.1.smithi078.stdout:8/711: rename d0/f2f to d0/d19/d52/d35/d44/fe5 0 2024-04-26T04:53:02.541 INFO:tasks.workunit.client.1.smithi078.stdout:8/712: write d0/d19/d25/d36/f7f [61582,45326] 0 2024-04-26T04:53:02.542 INFO:tasks.workunit.client.0.smithi077.stdout:6/442: creat d8/d2a/d86/f94 x:0 0 0 2024-04-26T04:53:02.542 INFO:tasks.workunit.client.1.smithi078.stdout:8/713: fdatasync d0/d19/d25/d32/fba 0 2024-04-26T04:53:02.543 INFO:tasks.workunit.client.1.smithi078.stdout:2/737: mkdir d8/d11/d18/dfe 0 2024-04-26T04:53:02.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/714: truncate d0/d19/d25/f72 4881199 0 2024-04-26T04:53:02.544 INFO:tasks.workunit.client.1.smithi078.stdout:2/738: write d8/d11/d6e/f80 [3541962,39861] 0 2024-04-26T04:53:02.545 INFO:tasks.workunit.client.1.smithi078.stdout:2/739: chown d8/d53/c7d 214 1 2024-04-26T04:53:02.545 INFO:tasks.workunit.client.1.smithi078.stdout:2/740: stat d8/d11/d14/f1c 0 2024-04-26T04:53:02.546 INFO:tasks.workunit.client.1.smithi078.stdout:4/769: mknod d0/d1d/d46/d100/c114 0 2024-04-26T04:53:02.546 INFO:tasks.workunit.client.1.smithi078.stdout:2/741: chown d8/d11/d14/d69/d8f/d34/dd3/dd9 5 1 2024-04-26T04:53:02.547 INFO:tasks.workunit.client.0.smithi077.stdout:9/367: truncate d9/f46 20772 0 2024-04-26T04:53:02.549 INFO:tasks.workunit.client.0.smithi077.stdout:6/443: mknod d8/d52/d53/d5e/c95 0 2024-04-26T04:53:02.550 INFO:tasks.workunit.client.0.smithi077.stdout:6/444: stat d8/d1c/f8a 0 2024-04-26T04:53:02.552 INFO:tasks.workunit.client.0.smithi077.stdout:7/417: link da/d6b/d63/d64/c69 da/d13/d22/d5d/c91 0 2024-04-26T04:53:02.553 INFO:tasks.workunit.client.1.smithi078.stdout:0/759: mkdir d2/d1e/d5a/df9 0 2024-04-26T04:53:02.557 INFO:tasks.workunit.client.1.smithi078.stdout:5/690: write d7/d18/d1c/d8e/fcd [1283483,101119] 0 2024-04-26T04:53:02.558 INFO:tasks.workunit.client.1.smithi078.stdout:6/809: truncate d2/d1a/f43 2655653 0 2024-04-26T04:53:02.559 INFO:tasks.workunit.client.1.smithi078.stdout:5/691: readlink d7/d18/d32/l6c 0 2024-04-26T04:53:02.561 INFO:tasks.workunit.client.1.smithi078.stdout:6/810: write d2/d1a/d5c/ff0 [546016,61729] 0 2024-04-26T04:53:02.561 INFO:tasks.workunit.client.1.smithi078.stdout:1/786: creat d7/d1e/d9e/db/d47/f102 x:0 0 0 2024-04-26T04:53:02.561 INFO:tasks.workunit.client.1.smithi078.stdout:0/760: dwrite d2/d9/d31/d4a/fa2 [4194304,4194304] 0 2024-04-26T04:53:02.561 INFO:tasks.workunit.client.1.smithi078.stdout:5/692: chown d7/da/d1e/d91/dd6 5 1 2024-04-26T04:53:02.562 INFO:tasks.workunit.client.1.smithi078.stdout:5/693: fsync d7/d21/f4e 0 2024-04-26T04:53:02.562 INFO:tasks.workunit.client.1.smithi078.stdout:9/681: mkdir dc/df/d4b/d22/dbe/dcd 0 2024-04-26T04:53:02.564 INFO:tasks.workunit.client.1.smithi078.stdout:5/694: chown d7/d48/d57/d65/l3e 90183 1 2024-04-26T04:53:02.565 INFO:tasks.workunit.client.1.smithi078.stdout:5/695: readlink d7/da/d1e/l20 0 2024-04-26T04:53:02.566 INFO:tasks.workunit.client.0.smithi077.stdout:7/418: creat da/d13/d22/d34/f92 x:0 0 0 2024-04-26T04:53:02.572 INFO:tasks.workunit.client.0.smithi077.stdout:7/419: dwrite da/d15/d1a/f79 [0,4194304] 0 2024-04-26T04:53:02.572 INFO:tasks.workunit.client.1.smithi078.stdout:2/742: mknod d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/cff 0 2024-04-26T04:53:02.573 INFO:tasks.workunit.client.1.smithi078.stdout:2/743: write d8/d11/f54 [3108975,105743] 0 2024-04-26T04:53:02.577 INFO:tasks.workunit.client.1.smithi078.stdout:3/764: getdents d16/d26/d2e/d5f/dbb 0 2024-04-26T04:53:02.579 INFO:tasks.workunit.client.1.smithi078.stdout:2/744: read d8/d11/d14/d69/d8f/f5c [3659417,55665] 0 2024-04-26T04:53:02.580 INFO:tasks.workunit.client.0.smithi077.stdout:7/420: dwrite da/f59 [0,4194304] 0 2024-04-26T04:53:02.582 INFO:tasks.workunit.client.0.smithi077.stdout:9/368: link d9/dd/c27 d9/dd/c84 0 2024-04-26T04:53:02.585 INFO:tasks.workunit.client.0.smithi077.stdout:4/336: truncate d4/d17/f3d 3884541 0 2024-04-26T04:53:02.585 INFO:tasks.workunit.client.0.smithi077.stdout:6/445: dread d8/d1c/f24 [0,4194304] 0 2024-04-26T04:53:02.586 INFO:tasks.workunit.client.0.smithi077.stdout:6/446: chown d8/f50 196 1 2024-04-26T04:53:02.586 INFO:tasks.workunit.client.0.smithi077.stdout:4/337: truncate d4/d17/f57 403334 0 2024-04-26T04:53:02.586 INFO:tasks.workunit.client.0.smithi077.stdout:7/421: rename da/d13/d22/f88 to da/d6b/f93 0 2024-04-26T04:53:02.589 INFO:tasks.workunit.client.1.smithi078.stdout:1/787: creat d7/d1e/d9e/db/d37/d58/d5c/f103 x:0 0 0 2024-04-26T04:53:02.593 INFO:tasks.workunit.client.1.smithi078.stdout:0/761: rename d2/d39/d96/fa4 to d2/d9/d31/ffa 0 2024-04-26T04:53:02.594 INFO:tasks.workunit.client.0.smithi077.stdout:7/422: dwrite da/d15/d42/f77 [0,4194304] 0 2024-04-26T04:53:02.595 INFO:tasks.workunit.client.1.smithi078.stdout:0/762: write d2/d9/d18/d7d/da9/dab/dc6/fb2 [8816049,101241] 0 2024-04-26T04:53:02.596 INFO:tasks.workunit.client.0.smithi077.stdout:7/423: chown da/d13/f4b 36233935 1 2024-04-26T04:53:02.596 INFO:tasks.workunit.client.1.smithi078.stdout:0/763: fsync d2/d9/d31/fce 0 2024-04-26T04:53:02.596 INFO:tasks.workunit.client.1.smithi078.stdout:0/764: stat d2/d1e/d5a/dda 0 2024-04-26T04:53:02.597 INFO:tasks.workunit.client.0.smithi077.stdout:7/424: chown da/d15/d42/l6e 51 1 2024-04-26T04:53:02.597 INFO:tasks.workunit.client.0.smithi077.stdout:7/425: chown da/d6b/d63/f70 99583751 1 2024-04-26T04:53:02.598 INFO:tasks.workunit.client.0.smithi077.stdout:6/447: mknod d8/d2a/d86/c96 0 2024-04-26T04:53:02.601 INFO:tasks.workunit.client.1.smithi078.stdout:9/682: unlink dc/df/d17/l67 0 2024-04-26T04:53:02.604 INFO:tasks.workunit.client.0.smithi077.stdout:9/369: creat d9/dd/d1b/d2b/d32/d36/d7f/f85 x:0 0 0 2024-04-26T04:53:02.605 INFO:tasks.workunit.client.0.smithi077.stdout:9/370: read d9/dd/d3d/f3e [585024,2406] 0 2024-04-26T04:53:02.606 INFO:tasks.workunit.client.1.smithi078.stdout:5/696: dread - d7/d48/d57/f99 zero size 2024-04-26T04:53:02.606 INFO:tasks.workunit.client.1.smithi078.stdout:4/770: mknod d0/d1d/de1/d20/d31/d47/db2/dc3/c115 0 2024-04-26T04:53:02.606 INFO:tasks.workunit.client.0.smithi077.stdout:9/371: readlink d9/d3a/d79/d50/l5c 0 2024-04-26T04:53:02.609 INFO:tasks.workunit.client.0.smithi077.stdout:7/426: rename da/d13/d22/d7d/f7e to da/d15/f94 0 2024-04-26T04:53:02.610 INFO:tasks.workunit.client.0.smithi077.stdout:7/427: fdatasync da/d6b/d63/d64/f67 0 2024-04-26T04:53:02.614 INFO:tasks.workunit.client.0.smithi077.stdout:6/448: unlink d8/d1c/c3b 0 2024-04-26T04:53:02.614 INFO:tasks.workunit.client.0.smithi077.stdout:6/449: readlink l7 0 2024-04-26T04:53:02.616 INFO:tasks.workunit.client.1.smithi078.stdout:7/714: link d9/da/d11/d13/d1a/d52/d5a/d41/f51 d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fe9 0 2024-04-26T04:53:02.620 INFO:tasks.workunit.client.1.smithi078.stdout:1/788: mknod d7/d2e/c104 0 2024-04-26T04:53:02.623 INFO:tasks.workunit.client.0.smithi077.stdout:7/428: mknod da/d13/d22/d7d/d80/c95 0 2024-04-26T04:53:02.629 INFO:tasks.workunit.client.0.smithi077.stdout:9/372: fdatasync d9/f62 0 2024-04-26T04:53:02.633 INFO:tasks.workunit.client.0.smithi077.stdout:4/338: getdents d4/d17/d18 0 2024-04-26T04:53:02.640 INFO:tasks.workunit.client.0.smithi077.stdout:7/429: write da/d15/d1a/f20 [950494,105114] 0 2024-04-26T04:53:02.641 INFO:tasks.workunit.client.0.smithi077.stdout:6/450: dwrite d8/d55/f90 [0,4194304] 0 2024-04-26T04:53:02.643 INFO:tasks.workunit.client.0.smithi077.stdout:4/339: unlink d4/de/d15/f16 0 2024-04-26T04:53:02.651 INFO:tasks.workunit.client.1.smithi078.stdout:8/715: sync 2024-04-26T04:53:02.654 INFO:tasks.workunit.client.0.smithi077.stdout:7/430: mkdir da/d15/d1a/d39/d96 0 2024-04-26T04:53:02.655 INFO:tasks.workunit.client.0.smithi077.stdout:4/340: mkdir d4/de/d15/d1f/d76 0 2024-04-26T04:53:02.657 INFO:tasks.workunit.client.0.smithi077.stdout:9/373: link d9/dd/d1b/d2b/l60 d9/dd/d1b/d2b/d32/l86 0 2024-04-26T04:53:02.657 INFO:tasks.workunit.client.0.smithi077.stdout:9/374: dread - d9/dd/d1b/d2b/f72 zero size 2024-04-26T04:53:02.659 INFO:tasks.workunit.client.0.smithi077.stdout:9/375: read d9/f26 [3089495,119511] 0 2024-04-26T04:53:02.660 INFO:tasks.workunit.client.0.smithi077.stdout:7/431: symlink da/d6b/l97 0 2024-04-26T04:53:02.662 INFO:tasks.workunit.client.0.smithi077.stdout:4/341: fsync d4/de/f44 0 2024-04-26T04:53:02.663 INFO:tasks.workunit.client.0.smithi077.stdout:4/342: write d4/f39 [3336577,3526] 0 2024-04-26T04:53:02.675 INFO:tasks.workunit.client.0.smithi077.stdout:4/343: unlink d4/d17/d18/d43/d53/d56/l2e 0 2024-04-26T04:53:02.675 INFO:tasks.workunit.client.0.smithi077.stdout:4/344: chown d4/l6 7742583 1 2024-04-26T04:53:02.676 INFO:tasks.workunit.client.0.smithi077.stdout:4/345: dread - d4/de/d15/d1f/f5e zero size 2024-04-26T04:53:02.677 INFO:tasks.workunit.client.0.smithi077.stdout:6/451: link d8/d2a/l2d d8/d52/d53/d5e/l97 0 2024-04-26T04:53:02.679 INFO:tasks.workunit.client.0.smithi077.stdout:9/376: mknod d9/dd/d3d/d7c/c87 0 2024-04-26T04:53:02.681 INFO:tasks.workunit.client.0.smithi077.stdout:7/432: fsync da/f3c 0 2024-04-26T04:53:02.686 INFO:tasks.workunit.client.0.smithi077.stdout:9/377: dwrite d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:53:02.690 INFO:tasks.workunit.client.0.smithi077.stdout:7/433: fdatasync da/d6b/f93 0 2024-04-26T04:53:02.692 INFO:tasks.workunit.client.0.smithi077.stdout:7/434: dread - da/d13/d22/f58 zero size 2024-04-26T04:53:02.693 INFO:tasks.workunit.client.0.smithi077.stdout:7/435: dread - da/d15/d1a/d39/f71 zero size 2024-04-26T04:53:02.694 INFO:tasks.workunit.client.0.smithi077.stdout:7/436: fdatasync da/d15/d42/f50 0 2024-04-26T04:53:02.694 INFO:tasks.workunit.client.0.smithi077.stdout:7/437: read da/d13/d22/d34/f51 [334479,43800] 0 2024-04-26T04:53:02.695 INFO:tasks.workunit.client.0.smithi077.stdout:6/452: creat d8/d2a/d57/f98 x:0 0 0 2024-04-26T04:53:02.696 INFO:tasks.workunit.client.0.smithi077.stdout:6/453: chown d8/f48 75 1 2024-04-26T04:53:02.699 INFO:tasks.workunit.client.0.smithi077.stdout:4/346: dwrite d4/d17/d18/d43/d53/d56/f2d [0,4194304] 0 2024-04-26T04:53:02.700 INFO:tasks.workunit.client.0.smithi077.stdout:7/438: creat da/d13/f98 x:0 0 0 2024-04-26T04:53:02.701 INFO:tasks.workunit.client.0.smithi077.stdout:6/454: mknod d8/d52/c99 0 2024-04-26T04:53:02.703 INFO:tasks.workunit.client.0.smithi077.stdout:9/378: dread d9/d35/f4c [0,4194304] 0 2024-04-26T04:53:02.703 INFO:tasks.workunit.client.0.smithi077.stdout:4/347: creat d4/d4e/f77 x:0 0 0 2024-04-26T04:53:02.705 INFO:tasks.workunit.client.0.smithi077.stdout:4/348: truncate d4/de/d15/f40 5195661 0 2024-04-26T04:53:02.706 INFO:tasks.workunit.client.0.smithi077.stdout:6/455: creat d8/d2a/d57/f9a x:0 0 0 2024-04-26T04:53:02.710 INFO:tasks.workunit.client.0.smithi077.stdout:6/456: creat d8/d55/f9b x:0 0 0 2024-04-26T04:53:02.713 INFO:tasks.workunit.client.0.smithi077.stdout:7/439: rename da/f11 to da/d13/d22/f99 0 2024-04-26T04:53:02.714 INFO:tasks.workunit.client.0.smithi077.stdout:7/440: chown da/d13/d22/d7d/d80/f8b 1 1 2024-04-26T04:53:02.715 INFO:tasks.workunit.client.0.smithi077.stdout:4/349: symlink d4/de/d15/d1f/d76/l78 0 2024-04-26T04:53:02.717 INFO:tasks.workunit.client.1.smithi078.stdout:5/697: fsync d7/d48/d57/d65/d7a/f98 0 2024-04-26T04:53:02.718 INFO:tasks.workunit.client.1.smithi078.stdout:5/698: stat d7/d18/d1c 0 2024-04-26T04:53:02.719 INFO:tasks.workunit.client.1.smithi078.stdout:2/745: creat d8/d11/d18/dfe/f100 x:0 0 0 2024-04-26T04:53:02.721 INFO:tasks.workunit.client.0.smithi077.stdout:6/457: symlink d8/l9c 0 2024-04-26T04:53:02.722 INFO:tasks.workunit.client.1.smithi078.stdout:7/715: dread d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e [0,4194304] 0 2024-04-26T04:53:02.726 INFO:tasks.workunit.client.1.smithi078.stdout:3/765: rename d16/l24 to d16/d26/d2e/l10e 0 2024-04-26T04:53:02.727 INFO:tasks.workunit.client.1.smithi078.stdout:6/811: link d2/de/d27/l3f d2/de/d27/da9/de5/l100 0 2024-04-26T04:53:02.730 INFO:tasks.workunit.client.0.smithi077.stdout:4/350: dread d4/d17/f3b [0,4194304] 0 2024-04-26T04:53:02.730 INFO:tasks.workunit.client.1.smithi078.stdout:9/683: creat dc/df/d4b/d3e/d80/fce x:0 0 0 2024-04-26T04:53:02.732 INFO:tasks.workunit.client.1.smithi078.stdout:4/771: truncate d0/de/f34 1674641 0 2024-04-26T04:53:02.733 INFO:tasks.workunit.client.1.smithi078.stdout:4/772: write d0/de/d22/d27/de7/f10e [127034,99999] 0 2024-04-26T04:53:02.735 INFO:tasks.workunit.client.1.smithi078.stdout:5/699: symlink d7/d21/d2f/d90/le6 0 2024-04-26T04:53:02.737 INFO:tasks.workunit.client.0.smithi077.stdout:4/351: dwrite d4/de/d15/d1f/f5e [0,4194304] 0 2024-04-26T04:53:02.737 INFO:tasks.workunit.client.1.smithi078.stdout:2/746: creat d8/d11/d18/dbb/f101 x:0 0 0 2024-04-26T04:53:02.741 INFO:tasks.workunit.client.0.smithi077.stdout:6/458: getdents d8/d2a/d57 0 2024-04-26T04:53:02.742 INFO:tasks.workunit.client.1.smithi078.stdout:0/765: rename d2/d9/d78/cde to d2/d9/d60/dd5/cfb 0 2024-04-26T04:53:02.743 INFO:tasks.workunit.client.0.smithi077.stdout:4/352: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:53:02.745 INFO:tasks.workunit.client.0.smithi077.stdout:6/459: mknod d8/d55/c9d 0 2024-04-26T04:53:02.746 INFO:tasks.workunit.client.1.smithi078.stdout:6/812: chown d2/d1a/cb2 74 1 2024-04-26T04:53:02.748 INFO:tasks.workunit.client.0.smithi077.stdout:7/441: getdents da/d13/d22 0 2024-04-26T04:53:02.750 INFO:tasks.workunit.client.1.smithi078.stdout:8/716: creat d0/d7a/fe6 x:0 0 0 2024-04-26T04:53:02.751 INFO:tasks.workunit.client.1.smithi078.stdout:8/717: chown d0/d7a/dbe/fe1 1974283 1 2024-04-26T04:53:02.751 INFO:tasks.workunit.client.0.smithi077.stdout:4/353: dread - d4/de/d15/f50 zero size 2024-04-26T04:53:02.753 INFO:tasks.workunit.client.0.smithi077.stdout:6/460: mknod d8/d1c/c9e 0 2024-04-26T04:53:02.757 INFO:tasks.workunit.client.1.smithi078.stdout:4/773: truncate d0/d1d/de1/d20/d31/d47/db2/fc5 1488514 0 2024-04-26T04:53:02.763 INFO:tasks.workunit.client.1.smithi078.stdout:1/789: dwrite d7/d1e/d9e/db/d37/d58/d5c/dc0/fa1 [0,4194304] 0 2024-04-26T04:53:02.763 INFO:tasks.workunit.client.1.smithi078.stdout:7/716: rename d9/da/d11/d13/d1a/d52/d5a/d41/d68/fbb to d9/da/d11/d13/d50/d8a/fea 0 2024-04-26T04:53:02.764 INFO:tasks.workunit.client.1.smithi078.stdout:0/766: creat d2/d1e/d5a/da3/ffc x:0 0 0 2024-04-26T04:53:02.767 INFO:tasks.workunit.client.0.smithi077.stdout:6/461: creat d8/d8f/f9f x:0 0 0 2024-04-26T04:53:02.767 INFO:tasks.workunit.client.0.smithi077.stdout:6/462: dread - d8/d1c/f8a zero size 2024-04-26T04:53:02.768 INFO:tasks.workunit.client.1.smithi078.stdout:6/813: mkdir d2/de/d27/d58/d5b/d101 0 2024-04-26T04:53:02.771 INFO:tasks.workunit.client.1.smithi078.stdout:9/684: dread dc/df/d27/f53 [0,4194304] 0 2024-04-26T04:53:02.771 INFO:tasks.workunit.client.1.smithi078.stdout:2/747: mkdir d8/d11/d18/d5d/d61/df5/d102 0 2024-04-26T04:53:02.775 INFO:tasks.workunit.client.1.smithi078.stdout:7/717: symlink d9/da/d20/leb 0 2024-04-26T04:53:02.777 INFO:tasks.workunit.client.1.smithi078.stdout:1/790: rmdir d7/d1e/d9e/db/d49/dba/d87 39 2024-04-26T04:53:02.778 INFO:tasks.workunit.client.0.smithi077.stdout:7/442: dwrite da/f3c [0,4194304] 0 2024-04-26T04:53:02.780 INFO:tasks.workunit.client.1.smithi078.stdout:2/748: dread d8/d11/d14/d69/d8f/f49 [0,4194304] 0 2024-04-26T04:53:02.780 INFO:tasks.workunit.client.1.smithi078.stdout:5/700: dwrite d7/d18/d32/f60 [0,4194304] 0 2024-04-26T04:53:02.780 INFO:tasks.workunit.client.0.smithi077.stdout:4/354: dread d4/de/d15/d1f/d35/f52 [0,4194304] 0 2024-04-26T04:53:02.783 INFO:tasks.workunit.client.1.smithi078.stdout:3/766: write d16/d26/d2b/fe6 [352477,99686] 0 2024-04-26T04:53:02.783 INFO:tasks.workunit.client.1.smithi078.stdout:2/749: write d8/d53/f71 [699519,6197] 0 2024-04-26T04:53:02.784 INFO:tasks.workunit.client.0.smithi077.stdout:7/443: dread da/d15/f44 [0,4194304] 0 2024-04-26T04:53:02.787 INFO:tasks.workunit.client.0.smithi077.stdout:4/355: rmdir d4/de/d15/d1f/d25 39 2024-04-26T04:53:02.787 INFO:tasks.workunit.client.1.smithi078.stdout:8/718: mkdir d0/d19/d52/de7 0 2024-04-26T04:53:02.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/356: dread - d4/de/d15/f50 zero size 2024-04-26T04:53:02.789 INFO:tasks.workunit.client.1.smithi078.stdout:4/774: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/de0/f116 x:0 0 0 2024-04-26T04:53:02.791 INFO:tasks.workunit.client.1.smithi078.stdout:9/685: readlink dc/df/d17/l50 0 2024-04-26T04:53:02.793 INFO:tasks.workunit.client.1.smithi078.stdout:7/718: mknod d9/da/d11/d13/d1a/d2d/cec 0 2024-04-26T04:53:02.794 INFO:tasks.workunit.client.1.smithi078.stdout:1/791: dwrite d7/d1e/d9e/db/d49/f61 [4194304,4194304] 0 2024-04-26T04:53:02.795 INFO:tasks.workunit.client.1.smithi078.stdout:2/750: creat d8/d11/d14/d69/d8f/d34/d70/f103 x:0 0 0 2024-04-26T04:53:02.795 INFO:tasks.workunit.client.1.smithi078.stdout:9/686: dread dc/df/d27/d81/f9e [0,4194304] 0 2024-04-26T04:53:02.797 INFO:tasks.workunit.client.1.smithi078.stdout:5/701: rename d7/d48/d57/d65/d70/d72 to d7/d48/d57/d65/d46/de7 0 2024-04-26T04:53:02.799 INFO:tasks.workunit.client.0.smithi077.stdout:4/357: getdents d4/d17/d18/d43/d6e/d6c 0 2024-04-26T04:53:02.799 INFO:tasks.workunit.client.1.smithi078.stdout:8/719: creat d0/d19/d52/d35/d73/fe8 x:0 0 0 2024-04-26T04:53:02.800 INFO:tasks.workunit.client.1.smithi078.stdout:4/775: creat d0/de/d22/d27/d7b/f117 x:0 0 0 2024-04-26T04:53:02.801 INFO:tasks.workunit.client.1.smithi078.stdout:4/776: read - d0/de/d22/d71/dae/f10b zero size 2024-04-26T04:53:02.805 INFO:tasks.workunit.client.1.smithi078.stdout:1/792: rmdir d7/d1e/d9e/db/d37 39 2024-04-26T04:53:02.806 INFO:tasks.workunit.client.1.smithi078.stdout:6/814: dread d2/d1a/f43 [0,4194304] 0 2024-04-26T04:53:02.807 INFO:tasks.workunit.client.0.smithi077.stdout:4/358: dwrite d4/d4e/f5a [0,4194304] 0 2024-04-26T04:53:02.808 INFO:tasks.workunit.client.1.smithi078.stdout:9/687: creat dc/df/d17/fcf x:0 0 0 2024-04-26T04:53:02.809 INFO:tasks.workunit.client.1.smithi078.stdout:0/767: dread d2/d9/f20 [0,4194304] 0 2024-04-26T04:53:02.812 INFO:tasks.workunit.client.0.smithi077.stdout:4/359: creat d4/d17/d18/d43/d53/f79 x:0 0 0 2024-04-26T04:53:02.814 INFO:tasks.workunit.client.1.smithi078.stdout:8/720: creat d0/d19/d52/d35/d44/d6b/d6c/fe9 x:0 0 0 2024-04-26T04:53:02.819 INFO:tasks.workunit.client.1.smithi078.stdout:7/719: symlink d9/da/d33/d4b/dc0/led 0 2024-04-26T04:53:02.820 INFO:tasks.workunit.client.0.smithi077.stdout:4/360: dwrite d4/d4e/f77 [0,4194304] 0 2024-04-26T04:53:02.825 INFO:tasks.workunit.client.1.smithi078.stdout:7/720: dwrite d9/da/d33/fdb [0,4194304] 0 2024-04-26T04:53:02.831 INFO:tasks.workunit.client.1.smithi078.stdout:3/767: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f82 [1857878,65991] 0 2024-04-26T04:53:02.833 INFO:tasks.workunit.client.0.smithi077.stdout:4/361: creat d4/d17/f7a x:0 0 0 2024-04-26T04:53:02.834 INFO:tasks.workunit.client.1.smithi078.stdout:0/768: dread d2/d9/d31/fe0 [0,4194304] 0 2024-04-26T04:53:02.835 INFO:tasks.workunit.client.1.smithi078.stdout:2/751: link d8/d11/f3f d8/d11/d14/d69/d8f/d34/d70/def/f104 0 2024-04-26T04:53:02.835 INFO:tasks.workunit.client.1.smithi078.stdout:9/688: dread dc/df/d27/d81/f66 [0,4194304] 0 2024-04-26T04:53:02.837 INFO:tasks.workunit.client.0.smithi077.stdout:4/362: mknod d4/d17/d18/d43/d53/d64/c7b 0 2024-04-26T04:53:02.840 INFO:tasks.workunit.client.0.smithi077.stdout:4/363: rename d4/d4e/l6a to d4/de/d15/d1f/d35/l7c 0 2024-04-26T04:53:02.843 INFO:tasks.workunit.client.1.smithi078.stdout:6/815: link d2/de/d27/da9/da8/fb7 d2/de/d27/da9/d3a/d69/daa/dba/dc4/f102 0 2024-04-26T04:53:02.845 INFO:tasks.workunit.client.1.smithi078.stdout:8/721: symlink d0/d19/d52/d5c/lea 0 2024-04-26T04:53:02.847 INFO:tasks.workunit.client.1.smithi078.stdout:1/793: truncate d7/d1e/d9e/db/d49/fd8 144359 0 2024-04-26T04:53:02.848 INFO:tasks.workunit.client.1.smithi078.stdout:0/769: truncate d2/f6c 1890556 0 2024-04-26T04:53:02.849 INFO:tasks.workunit.client.1.smithi078.stdout:5/702: dwrite d7/d48/d57/d65/fb0 [0,4194304] 0 2024-04-26T04:53:02.850 INFO:tasks.workunit.client.1.smithi078.stdout:1/794: write d7/d1e/d9e/db/d49/fb1 [3472917,83670] 0 2024-04-26T04:53:02.851 INFO:tasks.workunit.client.1.smithi078.stdout:9/689: symlink dc/df/d4b/d22/d2a/d57/ld0 0 2024-04-26T04:53:02.852 INFO:tasks.workunit.client.0.smithi077.stdout:4/364: link d4/de/c30 d4/d17/c7d 0 2024-04-26T04:53:02.853 INFO:tasks.workunit.client.1.smithi078.stdout:3/768: write d16/d26/f37 [4922141,19778] 0 2024-04-26T04:53:02.853 INFO:tasks.workunit.client.1.smithi078.stdout:6/816: dread - d2/de/f82 zero size 2024-04-26T04:53:02.854 INFO:tasks.workunit.client.0.smithi077.stdout:4/365: chown d4/de/d15/c4f 1217646199 1 2024-04-26T04:53:02.856 INFO:tasks.workunit.client.1.smithi078.stdout:7/721: dwrite d9/da/d33/f70 [0,4194304] 0 2024-04-26T04:53:02.857 INFO:tasks.workunit.client.1.smithi078.stdout:2/752: creat d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/dec/f105 x:0 0 0 2024-04-26T04:53:02.857 INFO:tasks.workunit.client.1.smithi078.stdout:2/753: readlink d8/db7/lf8 0 2024-04-26T04:53:02.859 INFO:tasks.workunit.client.1.smithi078.stdout:0/770: unlink d2/f21 0 2024-04-26T04:53:02.868 INFO:tasks.workunit.client.1.smithi078.stdout:1/795: readlink d7/d1e/d9e/db/d37/d58/d5c/d68/lae 0 2024-04-26T04:53:02.871 INFO:tasks.workunit.client.0.smithi077.stdout:4/366: getdents d4/d17 0 2024-04-26T04:53:02.876 INFO:tasks.workunit.client.1.smithi078.stdout:8/722: creat d0/d19/d25/d36/d58/feb x:0 0 0 2024-04-26T04:53:02.880 INFO:tasks.workunit.client.1.smithi078.stdout:6/817: write d2/de/f67 [642668,111530] 0 2024-04-26T04:53:02.881 INFO:tasks.workunit.client.1.smithi078.stdout:5/703: dread d7/d21/d2f/db6/fd9 [0,4194304] 0 2024-04-26T04:53:02.884 INFO:tasks.workunit.client.1.smithi078.stdout:2/754: creat d8/d11/d14/d5b/f106 x:0 0 0 2024-04-26T04:53:02.884 INFO:tasks.workunit.client.1.smithi078.stdout:3/769: truncate d16/f30 571843 0 2024-04-26T04:53:02.888 INFO:tasks.workunit.client.1.smithi078.stdout:9/690: dread dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:53:02.889 INFO:tasks.workunit.client.1.smithi078.stdout:7/722: dwrite d9/da/d11/f45 [4194304,4194304] 0 2024-04-26T04:53:02.889 INFO:tasks.workunit.client.1.smithi078.stdout:0/771: dread d2/ff [0,4194304] 0 2024-04-26T04:53:02.890 INFO:tasks.workunit.client.1.smithi078.stdout:5/704: creat d7/da/fe8 x:0 0 0 2024-04-26T04:53:02.894 INFO:tasks.workunit.client.1.smithi078.stdout:2/755: write d8/d11/d14/d69/d8f/d34/d70/f75 [4978451,23167] 0 2024-04-26T04:53:02.895 INFO:tasks.workunit.client.1.smithi078.stdout:8/723: dread d0/d19/d52/f29 [0,4194304] 0 2024-04-26T04:53:02.898 INFO:tasks.workunit.client.0.smithi077.stdout:4/367: dread d4/d17/f3d [0,4194304] 0 2024-04-26T04:53:02.899 INFO:tasks.workunit.client.1.smithi078.stdout:9/691: write dc/df/d4b/d3e/d80/fce [210342,65440] 0 2024-04-26T04:53:02.900 INFO:tasks.workunit.client.1.smithi078.stdout:1/796: dread d7/d1e/d9e/f9f [0,4194304] 0 2024-04-26T04:53:02.900 INFO:tasks.workunit.client.1.smithi078.stdout:6/818: dwrite d2/d1a/d38/f68 [0,4194304] 0 2024-04-26T04:53:02.900 INFO:tasks.workunit.client.1.smithi078.stdout:7/723: mkdir d9/da/d33/d4b/d4c/d5b/d83/dee 0 2024-04-26T04:53:02.901 INFO:tasks.workunit.client.1.smithi078.stdout:4/777: sync 2024-04-26T04:53:02.902 INFO:tasks.workunit.client.1.smithi078.stdout:6/819: read - d2/de/d27/da9/d2a/f56 zero size 2024-04-26T04:53:02.903 INFO:tasks.workunit.client.1.smithi078.stdout:7/724: write d9/da/d11/d13/d1a/f42 [585888,58064] 0 2024-04-26T04:53:02.904 INFO:tasks.workunit.client.0.smithi077.stdout:4/368: rename d4/f39 to d4/d17/d18/d43/d53/d75/f7e 0 2024-04-26T04:53:02.905 INFO:tasks.workunit.client.1.smithi078.stdout:7/725: dread - d9/da/d33/d4b/d4c/d5b/f5c zero size 2024-04-26T04:53:02.906 INFO:tasks.workunit.client.1.smithi078.stdout:5/705: dwrite f3 [0,4194304] 0 2024-04-26T04:53:02.906 INFO:tasks.workunit.client.1.smithi078.stdout:2/756: creat d8/d11/d18/dbb/f107 x:0 0 0 2024-04-26T04:53:02.908 INFO:tasks.workunit.client.1.smithi078.stdout:0/772: mkdir d2/d9/d18/d7d/da9/dab/dfd 0 2024-04-26T04:53:02.908 INFO:tasks.workunit.client.1.smithi078.stdout:5/706: fdatasync d7/d18/d1c/d5a/fc3 0 2024-04-26T04:53:02.910 INFO:tasks.workunit.client.0.smithi077.stdout:4/369: mkdir d4/d17/d7f 0 2024-04-26T04:53:02.911 INFO:tasks.workunit.client.1.smithi078.stdout:8/724: chown d0/d19/d25/d32/d8a/db1/f88 697304 1 2024-04-26T04:53:02.913 INFO:tasks.workunit.client.1.smithi078.stdout:8/725: chown d0/d19/d25/d36/d58/d5b/d61 1443 1 2024-04-26T04:53:02.914 INFO:tasks.workunit.client.1.smithi078.stdout:6/820: creat d2/d52/f103 x:0 0 0 2024-04-26T04:53:02.915 INFO:tasks.workunit.client.0.smithi077.stdout:4/370: truncate d4/f27 1722159 0 2024-04-26T04:53:02.915 INFO:tasks.workunit.client.1.smithi078.stdout:6/821: chown d2/de/lad 884 1 2024-04-26T04:53:02.916 INFO:tasks.workunit.client.1.smithi078.stdout:8/726: truncate d0/d19/d25/d36/d58/d5b/d61/fd9 995516 0 2024-04-26T04:53:02.916 INFO:tasks.workunit.client.1.smithi078.stdout:2/757: creat d8/d11/d18/f108 x:0 0 0 2024-04-26T04:53:02.917 INFO:tasks.workunit.client.1.smithi078.stdout:7/726: readlink d9/da/d11/d13/d1a/d52/d5a/l39 0 2024-04-26T04:53:02.918 INFO:tasks.workunit.client.0.smithi077.stdout:4/371: mkdir d4/d17/d18/d43/d63/d80 0 2024-04-26T04:53:02.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/372: chown d4/d17/d18/d43/d53/d56/c61 136186632 1 2024-04-26T04:53:02.920 INFO:tasks.workunit.client.1.smithi078.stdout:2/758: read d8/d53/f71 [83856,29446] 0 2024-04-26T04:53:02.920 INFO:tasks.workunit.client.1.smithi078.stdout:3/770: link d16/d26/d2b/d31/d71/d8a/dbc/ce8 d16/d26/d2b/d31/c10f 0 2024-04-26T04:53:02.920 INFO:tasks.workunit.client.1.smithi078.stdout:5/707: creat d7/d48/d57/d65/d7a/d9c/fe9 x:0 0 0 2024-04-26T04:53:02.921 INFO:tasks.workunit.client.1.smithi078.stdout:5/708: dread - d7/d48/d57/d65/d46/dc4/fd5 zero size 2024-04-26T04:53:02.921 INFO:tasks.workunit.client.0.smithi077.stdout:4/373: dread - d4/d17/d18/d43/d53/d64/f65 zero size 2024-04-26T04:53:02.922 INFO:tasks.workunit.client.1.smithi078.stdout:6/822: creat d2/de/f104 x:0 0 0 2024-04-26T04:53:02.922 INFO:tasks.workunit.client.1.smithi078.stdout:5/709: write d7/d48/d57/d65/d46/f4f [1125231,79952] 0 2024-04-26T04:53:02.923 INFO:tasks.workunit.client.1.smithi078.stdout:6/823: stat d2/de/d27/da9/da/d35/f70 0 2024-04-26T04:53:02.923 INFO:tasks.workunit.client.0.smithi077.stdout:4/374: unlink d4/d17/l3a 0 2024-04-26T04:53:02.925 INFO:tasks.workunit.client.1.smithi078.stdout:2/759: creat d8/d11/d18/f109 x:0 0 0 2024-04-26T04:53:02.927 INFO:tasks.workunit.client.0.smithi077.stdout:4/375: creat d4/d17/d18/d43/d53/d64/f81 x:0 0 0 2024-04-26T04:53:02.928 INFO:tasks.workunit.client.1.smithi078.stdout:9/692: rmdir dc/df/d17/dc8 0 2024-04-26T04:53:02.929 INFO:tasks.workunit.client.1.smithi078.stdout:1/797: creat d7/d1e/d9e/db/f105 x:0 0 0 2024-04-26T04:53:02.929 INFO:tasks.workunit.client.0.smithi077.stdout:4/376: creat d4/de/d15/d1f/d76/f82 x:0 0 0 2024-04-26T04:53:02.930 INFO:tasks.workunit.client.0.smithi077.stdout:4/377: readlink d4/d17/d18/d43/d51/l69 0 2024-04-26T04:53:02.934 INFO:tasks.workunit.client.0.smithi077.stdout:4/378: mknod d4/de/d15/d1f/d35/c83 0 2024-04-26T04:53:02.935 INFO:tasks.workunit.client.1.smithi078.stdout:6/824: mkdir d2/de/d27/da9/da/d36/db9/d97/d105 0 2024-04-26T04:53:02.938 INFO:tasks.workunit.client.0.smithi077.stdout:4/379: creat d4/d17/d18/d43/d6e/f84 x:0 0 0 2024-04-26T04:53:02.940 INFO:tasks.workunit.client.1.smithi078.stdout:9/693: unlink f1 0 2024-04-26T04:53:02.940 INFO:tasks.workunit.client.1.smithi078.stdout:1/798: chown d7/l8 10184 1 2024-04-26T04:53:02.942 INFO:tasks.workunit.client.1.smithi078.stdout:5/710: truncate d7/d11/fc9 685165 0 2024-04-26T04:53:02.942 INFO:tasks.workunit.client.1.smithi078.stdout:0/773: write d2/ff [869375,67780] 0 2024-04-26T04:53:02.943 INFO:tasks.workunit.client.1.smithi078.stdout:6/825: creat d2/de/d27/da9/d11/f106 x:0 0 0 2024-04-26T04:53:02.944 INFO:tasks.workunit.client.0.smithi077.stdout:4/380: mknod d4/d17/d18/d43/d63/d80/c85 0 2024-04-26T04:53:02.944 INFO:tasks.workunit.client.1.smithi078.stdout:5/711: fsync d7/d48/d57/d65/d7a/d9c/fe9 0 2024-04-26T04:53:02.944 INFO:tasks.workunit.client.1.smithi078.stdout:4/778: write d0/d1d/de1/dd8/f10d [1856772,69840] 0 2024-04-26T04:53:02.945 INFO:tasks.workunit.client.1.smithi078.stdout:8/727: rename d0/d19/d25/d32/d8a/l7b to d0/d19/d52/d5c/lec 0 2024-04-26T04:53:02.946 INFO:tasks.workunit.client.1.smithi078.stdout:6/826: fsync d2/de/d27/da9/d2a/dbf/d5f/fef 0 2024-04-26T04:53:02.947 INFO:tasks.workunit.client.1.smithi078.stdout:8/728: write d0/fe4 [5177662,43178] 0 2024-04-26T04:53:02.947 INFO:tasks.workunit.client.0.smithi077.stdout:4/381: creat d4/d17/d18/d43/d6e/f86 x:0 0 0 2024-04-26T04:53:02.949 INFO:tasks.workunit.client.1.smithi078.stdout:6/827: chown d2/de/d27/da9/d11/la2 251413239 1 2024-04-26T04:53:02.949 INFO:tasks.workunit.client.1.smithi078.stdout:9/694: write dc/df/d4b/f79 [2450172,123337] 0 2024-04-26T04:53:02.950 INFO:tasks.workunit.client.1.smithi078.stdout:1/799: fdatasync d7/d1e/d9e/dd/fa8 0 2024-04-26T04:53:02.950 INFO:tasks.workunit.client.1.smithi078.stdout:3/771: write d16/d26/d2e/f4d [1062776,21937] 0 2024-04-26T04:53:02.951 INFO:tasks.workunit.client.0.smithi077.stdout:4/382: rmdir d4/d17/d18/d43/d63/d80 39 2024-04-26T04:53:02.951 INFO:tasks.workunit.client.0.smithi077.stdout:4/383: fsync d4/d17/f7a 0 2024-04-26T04:53:02.952 INFO:tasks.workunit.client.1.smithi078.stdout:0/774: write d2/d9/d18/db7/fd1 [8378835,8440] 0 2024-04-26T04:53:02.952 INFO:tasks.workunit.client.1.smithi078.stdout:1/800: fsync d7/d1e/d9e/f62 0 2024-04-26T04:53:02.956 INFO:tasks.workunit.client.1.smithi078.stdout:8/729: dwrite d0/d19/d25/d36/d58/fb7 [0,4194304] 0 2024-04-26T04:53:02.957 INFO:tasks.workunit.client.1.smithi078.stdout:8/730: write d0/d7a/fe6 [284952,99083] 0 2024-04-26T04:53:02.957 INFO:tasks.workunit.client.1.smithi078.stdout:4/779: unlink d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/l97 0 2024-04-26T04:53:02.958 INFO:tasks.workunit.client.0.smithi077.stdout:4/384: truncate d4/de/d15/f50 100048 0 2024-04-26T04:53:02.959 INFO:tasks.workunit.client.0.smithi077.stdout:4/385: stat d4/d17/d18/d43/d53/d64/c7b 0 2024-04-26T04:53:02.960 INFO:tasks.workunit.client.1.smithi078.stdout:6/828: mknod d2/de/d27/da9/d11/c107 0 2024-04-26T04:53:02.962 INFO:tasks.workunit.client.1.smithi078.stdout:1/801: dwrite d7/d1e/d9e/d4c/d3d/f101 [0,4194304] 0 2024-04-26T04:53:02.966 INFO:tasks.workunit.client.1.smithi078.stdout:1/802: dread d7/d1e/d9e/dd/f55 [4194304,4194304] 0 2024-04-26T04:53:02.966 INFO:tasks.workunit.client.1.smithi078.stdout:5/712: creat d7/d18/d32/d37/d79/da7/fea x:0 0 0 2024-04-26T04:53:02.967 INFO:tasks.workunit.client.1.smithi078.stdout:1/803: chown d7/d1e/d9e/db/d49/d93/fdb 1249273 1 2024-04-26T04:53:02.969 INFO:tasks.workunit.client.1.smithi078.stdout:7/727: rename d9/da/fa0 to d9/da/d11/d13/d1a/dd4/fef 0 2024-04-26T04:53:02.971 INFO:tasks.workunit.client.1.smithi078.stdout:3/772: creat d16/d7e/f110 x:0 0 0 2024-04-26T04:53:02.972 INFO:tasks.workunit.client.1.smithi078.stdout:0/775: creat d2/d9/d60/d6a/daa/ffe x:0 0 0 2024-04-26T04:53:02.973 INFO:tasks.workunit.client.1.smithi078.stdout:0/776: chown d2/d9/d78/d84/f8e 67232566 1 2024-04-26T04:53:02.976 INFO:tasks.workunit.client.1.smithi078.stdout:6/829: creat d2/de/d27/da9/d3a/d42/f108 x:0 0 0 2024-04-26T04:53:02.979 INFO:tasks.workunit.client.1.smithi078.stdout:2/760: rename d8/d11/d14/d69/d8f/d34/dd3/de4/d52/led to d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/dec/l10a 0 2024-04-26T04:53:02.983 INFO:tasks.workunit.client.0.smithi077.stdout:4/386: creat d4/de/d15/d1f/d76/f87 x:0 0 0 2024-04-26T04:53:02.984 INFO:tasks.workunit.client.0.smithi077.stdout:4/387: write d4/f14 [2158010,39993] 0 2024-04-26T04:53:02.986 INFO:tasks.workunit.client.0.smithi077.stdout:4/388: write d4/d17/d18/d43/d6e/f86 [295296,26430] 0 2024-04-26T04:53:02.986 INFO:tasks.workunit.client.1.smithi078.stdout:2/761: mknod d8/d11/d18/d5d/d61/df5/c10b 0 2024-04-26T04:53:02.987 INFO:tasks.workunit.client.0.smithi077.stdout:4/389: write d4/d17/d18/d43/f74 [168875,52363] 0 2024-04-26T04:53:02.988 INFO:tasks.workunit.client.1.smithi078.stdout:9/695: getdents dc/df/d27 0 2024-04-26T04:53:02.989 INFO:tasks.workunit.client.0.smithi077.stdout:4/390: fdatasync d4/d17/d18/f5d 0 2024-04-26T04:53:02.991 INFO:tasks.workunit.client.1.smithi078.stdout:6/830: rename d2/d52/fbd to d2/de/d27/da9/f109 0 2024-04-26T04:53:02.992 INFO:tasks.workunit.client.1.smithi078.stdout:3/773: dread d16/d26/f2c [4194304,4194304] 0 2024-04-26T04:53:02.993 INFO:tasks.workunit.client.1.smithi078.stdout:6/831: rmdir d2/de/d27/da9 39 2024-04-26T04:53:02.994 INFO:tasks.workunit.client.1.smithi078.stdout:3/774: symlink d16/d26/d2e/d5f/d49/dcc/de2/l111 0 2024-04-26T04:53:02.995 INFO:tasks.workunit.client.1.smithi078.stdout:3/775: chown d16/d3c/d81/fde 1849 1 2024-04-26T04:53:02.995 INFO:tasks.workunit.client.1.smithi078.stdout:9/696: getdents dc/df/d4b/d22/dbe 0 2024-04-26T04:53:02.996 INFO:tasks.workunit.client.1.smithi078.stdout:6/832: write d2/de/d27/da9/f109 [2276427,19324] 0 2024-04-26T04:53:02.998 INFO:tasks.workunit.client.1.smithi078.stdout:3/776: chown d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56 507 1 2024-04-26T04:53:02.999 INFO:tasks.workunit.client.1.smithi078.stdout:6/833: mkdir d2/d1a/d38/d10a 0 2024-04-26T04:53:03.000 INFO:tasks.workunit.client.1.smithi078.stdout:3/777: readlink d16/d26/d2e/d5f/d49/dcc/de2/l109 0 2024-04-26T04:53:03.006 INFO:tasks.workunit.client.1.smithi078.stdout:3/778: dwrite d16/d26/d2e/d5f/d49/d77/fc6 [4194304,4194304] 0 2024-04-26T04:53:03.007 INFO:tasks.workunit.client.1.smithi078.stdout:3/779: write d16/d26/d2b/d31/fc5 [3606184,8087] 0 2024-04-26T04:53:03.007 INFO:tasks.workunit.client.1.smithi078.stdout:3/780: readlink d16/d3c/lc0 0 2024-04-26T04:53:03.032 INFO:tasks.workunit.client.1.smithi078.stdout:6/834: creat d2/de/d27/da9/d2a/dbf/f10b x:0 0 0 2024-04-26T04:53:03.033 INFO:tasks.workunit.client.1.smithi078.stdout:6/835: readlink d2/de/d27/ldb 0 2024-04-26T04:53:03.037 INFO:tasks.workunit.client.1.smithi078.stdout:6/836: fsync d2/de/d27/de8/fc0 0 2024-04-26T04:53:03.040 INFO:tasks.workunit.client.1.smithi078.stdout:6/837: mkdir d2/de/d27/d58/d10c 0 2024-04-26T04:53:03.051 INFO:tasks.workunit.client.1.smithi078.stdout:4/780: dwrite d0/d1d/de1/f3a [0,4194304] 0 2024-04-26T04:53:03.053 INFO:tasks.workunit.client.1.smithi078.stdout:5/713: truncate d7/da/d8a/fd0 85116 0 2024-04-26T04:53:03.056 INFO:tasks.workunit.client.1.smithi078.stdout:8/731: dwrite d0/d19/d25/d36/d58/fe3 [0,4194304] 0 2024-04-26T04:53:03.057 INFO:tasks.workunit.client.1.smithi078.stdout:2/762: dread d8/d11/d6e/f7b [0,4194304] 0 2024-04-26T04:53:03.058 INFO:tasks.workunit.client.1.smithi078.stdout:5/714: chown d7/d48/d57/d65/d46/c50 95 1 2024-04-26T04:53:03.060 INFO:tasks.workunit.client.1.smithi078.stdout:2/763: fsync d8/d53/f7a 0 2024-04-26T04:53:03.060 INFO:tasks.workunit.client.1.smithi078.stdout:1/804: sync 2024-04-26T04:53:03.061 INFO:tasks.workunit.client.0.smithi077.stdout:4/391: dwrite d4/de/d15/f62 [0,4194304] 0 2024-04-26T04:53:03.064 INFO:tasks.workunit.client.1.smithi078.stdout:5/715: mkdir d7/d48/da9/deb 0 2024-04-26T04:53:03.066 INFO:tasks.workunit.client.1.smithi078.stdout:1/805: write d7/d1e/fe2 [21711,93378] 0 2024-04-26T04:53:03.066 INFO:tasks.workunit.client.0.smithi077.stdout:4/392: rmdir d4/d17/d18/d43 39 2024-04-26T04:53:03.069 INFO:tasks.workunit.client.1.smithi078.stdout:0/777: dwrite d2/f6c [0,4194304] 0 2024-04-26T04:53:03.071 INFO:tasks.workunit.client.1.smithi078.stdout:4/781: rename d0/d1d/de1/d20/d31/d47/db2/d44/d69 to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118 0 2024-04-26T04:53:03.075 INFO:tasks.workunit.client.0.smithi077.stdout:4/393: creat d4/de/d15/f88 x:0 0 0 2024-04-26T04:53:03.076 INFO:tasks.workunit.client.1.smithi078.stdout:8/732: creat d0/d19/d25/d32/d8a/d92/da4/fed x:0 0 0 2024-04-26T04:53:03.078 INFO:tasks.workunit.client.1.smithi078.stdout:9/697: dwrite dc/df/f2f [0,4194304] 0 2024-04-26T04:53:03.078 INFO:tasks.workunit.client.1.smithi078.stdout:2/764: creat d8/d11/d18/f10c x:0 0 0 2024-04-26T04:53:03.080 INFO:tasks.workunit.client.1.smithi078.stdout:0/778: stat d2/d9/d18/db7/d92/fad 0 2024-04-26T04:53:03.081 INFO:tasks.workunit.client.1.smithi078.stdout:3/781: dwrite d16/f57 [0,4194304] 0 2024-04-26T04:53:03.081 INFO:tasks.workunit.client.1.smithi078.stdout:6/838: dwrite d2/de/d27/da9/d2a/f94 [4194304,4194304] 0 2024-04-26T04:53:03.082 INFO:tasks.workunit.client.1.smithi078.stdout:2/765: stat d8/db7/lf8 0 2024-04-26T04:53:03.083 INFO:tasks.workunit.client.0.smithi077.stdout:4/394: rename d4/d17/d18/d43/d53/d64/c7b to d4/de/d15/c89 0 2024-04-26T04:53:03.084 INFO:tasks.workunit.client.1.smithi078.stdout:1/806: chown d7/d1e/d9e/db/d49/dba/d87/fbb 7 1 2024-04-26T04:53:03.086 INFO:tasks.workunit.client.1.smithi078.stdout:8/733: mkdir d0/d19/d25/d36/d58/d5b/dee 0 2024-04-26T04:53:03.087 INFO:tasks.workunit.client.1.smithi078.stdout:2/766: readlink d8/d11/d14/d5b/lf9 0 2024-04-26T04:53:03.088 INFO:tasks.workunit.client.1.smithi078.stdout:8/734: fsync d0/d19/d52/d35/d44/f4d 0 2024-04-26T04:53:03.089 INFO:tasks.workunit.client.1.smithi078.stdout:6/839: rmdir d2/de/d27/da9/d2a/dbf/d5f 39 2024-04-26T04:53:03.093 INFO:tasks.workunit.client.1.smithi078.stdout:1/807: write d7/d1e/d9e/db/d37/fa9 [688883,72335] 0 2024-04-26T04:53:03.095 INFO:tasks.workunit.client.1.smithi078.stdout:9/698: creat dc/df/fd1 x:0 0 0 2024-04-26T04:53:03.096 INFO:tasks.workunit.client.1.smithi078.stdout:8/735: mknod d0/d7a/dbe/cef 0 2024-04-26T04:53:03.097 INFO:tasks.workunit.client.1.smithi078.stdout:2/767: mkdir d8/d11/d18/d5d/d8b/db0/d10d 0 2024-04-26T04:53:03.099 INFO:tasks.workunit.client.1.smithi078.stdout:6/840: mknod d2/de/d27/da9/de5/c10d 0 2024-04-26T04:53:03.101 INFO:tasks.workunit.client.1.smithi078.stdout:1/808: mknod d7/d1e/d9e/db/d49/dba/dc5/c106 0 2024-04-26T04:53:03.103 INFO:tasks.workunit.client.1.smithi078.stdout:8/736: chown d0/d19/d52/d35/d44/cb6 13318606 1 2024-04-26T04:53:03.104 INFO:tasks.workunit.client.1.smithi078.stdout:0/779: rename d2/d9/d31/f66 to d2/d9/d18/db7/fff 0 2024-04-26T04:53:03.106 INFO:tasks.workunit.client.1.smithi078.stdout:1/809: dread d7/d1e/d9e/db/d37/d58/d5c/dc0/fa1 [0,4194304] 0 2024-04-26T04:53:03.107 INFO:tasks.workunit.client.1.smithi078.stdout:9/699: symlink dc/df/d4b/d22/dbb/ld2 0 2024-04-26T04:53:03.107 INFO:tasks.workunit.client.1.smithi078.stdout:5/716: write d7/d21/d2f/db6/fd9 [2886342,94698] 0 2024-04-26T04:53:03.109 INFO:tasks.workunit.client.1.smithi078.stdout:8/737: stat d0/d19/d25/d32/d8a/d92/fcf 0 2024-04-26T04:53:03.109 INFO:tasks.workunit.client.1.smithi078.stdout:9/700: chown dc/df/d4b/d22/d2a/d57/d69 113530822 1 2024-04-26T04:53:03.110 INFO:tasks.workunit.client.1.smithi078.stdout:8/738: write d0/d19/fa1 [4445291,20544] 0 2024-04-26T04:53:03.110 INFO:tasks.workunit.client.1.smithi078.stdout:6/841: mknod d2/de/d27/da9/da/c10e 0 2024-04-26T04:53:03.112 INFO:tasks.workunit.client.1.smithi078.stdout:3/782: write d16/f69 [1093373,57654] 0 2024-04-26T04:53:03.112 INFO:tasks.workunit.client.1.smithi078.stdout:1/810: mknod d7/d2e/c107 0 2024-04-26T04:53:03.113 INFO:tasks.workunit.client.1.smithi078.stdout:4/782: truncate d0/de/d22/d27/d56/f60 3139377 0 2024-04-26T04:53:03.114 INFO:tasks.workunit.client.1.smithi078.stdout:1/811: dread - d7/d2e/fe0 zero size 2024-04-26T04:53:03.115 INFO:tasks.workunit.client.1.smithi078.stdout:2/768: creat d8/d11/d14/d5b/f10e x:0 0 0 2024-04-26T04:53:03.115 INFO:tasks.workunit.client.1.smithi078.stdout:7/728: sync 2024-04-26T04:53:03.119 INFO:tasks.workunit.client.1.smithi078.stdout:5/717: dwrite d7/d18/d1c/d5a/d96/fe3 [0,4194304] 0 2024-04-26T04:53:03.121 INFO:tasks.workunit.client.1.smithi078.stdout:0/780: link d2/d9/d78/deb/ced d2/d9/d18/db7/d92/c100 0 2024-04-26T04:53:03.122 INFO:tasks.workunit.client.1.smithi078.stdout:6/842: creat d2/de/d27/da9/d3a/d69/dee/f10f x:0 0 0 2024-04-26T04:53:03.124 INFO:tasks.workunit.client.1.smithi078.stdout:0/781: chown d2/d9/d18/f49 55 1 2024-04-26T04:53:03.125 INFO:tasks.workunit.client.1.smithi078.stdout:1/812: rename d7/d1e/d9e/db/d49/d93/fd9 to d7/d1e/d9e/dd/d20/f108 0 2024-04-26T04:53:03.127 INFO:tasks.workunit.client.1.smithi078.stdout:9/701: creat dc/df/d4b/d3e/fd3 x:0 0 0 2024-04-26T04:53:03.128 INFO:tasks.workunit.client.1.smithi078.stdout:1/813: fsync d7/fbf 0 2024-04-26T04:53:03.130 INFO:tasks.workunit.client.1.smithi078.stdout:8/739: dread d0/d19/d25/d32/d8a/d92/fcf [0,4194304] 0 2024-04-26T04:53:03.132 INFO:tasks.workunit.client.1.smithi078.stdout:3/783: dread d16/d26/d2e/d74/f4f [0,4194304] 0 2024-04-26T04:53:03.135 INFO:tasks.workunit.client.1.smithi078.stdout:5/718: creat d7/d48/d57/d65/d46/de7/fec x:0 0 0 2024-04-26T04:53:03.135 INFO:tasks.workunit.client.1.smithi078.stdout:9/702: dwrite dc/df/d4b/f79 [0,4194304] 0 2024-04-26T04:53:03.137 INFO:tasks.workunit.client.1.smithi078.stdout:1/814: creat d7/d1e/d9e/db/d49/d93/f109 x:0 0 0 2024-04-26T04:53:03.137 INFO:tasks.workunit.client.1.smithi078.stdout:1/815: readlink d7/d1e/d9e/db/d47/l69 0 2024-04-26T04:53:03.139 INFO:tasks.workunit.client.1.smithi078.stdout:2/769: rename d8/d11/d14/d5b/f81 to d8/d11/d14/d69/d8f/d34/dd6/f10f 0 2024-04-26T04:53:03.141 INFO:tasks.workunit.client.1.smithi078.stdout:8/740: unlink d0/d19/f2a 0 2024-04-26T04:53:03.143 INFO:tasks.workunit.client.1.smithi078.stdout:7/729: getdents d9/da/d33/d4b/d4c 0 2024-04-26T04:53:03.170 INFO:tasks.workunit.client.1.smithi078.stdout:6/843: truncate d2/de/d27/da9/d2a/dbf/d5f/fd4 644425 0 2024-04-26T04:53:03.170 INFO:tasks.workunit.client.1.smithi078.stdout:4/783: rename d0/d1d/de1/d20/d31/d47/db2/fc5 to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/f119 0 2024-04-26T04:53:03.171 INFO:tasks.workunit.client.1.smithi078.stdout:1/816: dread d7/d1e/d9e/dd/d20/f45 [0,4194304] 0 2024-04-26T04:53:03.172 INFO:tasks.workunit.client.1.smithi078.stdout:3/784: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/f7d [3030500,52365] 0 2024-04-26T04:53:03.172 INFO:tasks.workunit.client.1.smithi078.stdout:4/784: write d0/d1d/db4/fd3 [794228,129548] 0 2024-04-26T04:53:03.172 INFO:tasks.workunit.client.1.smithi078.stdout:7/730: write f6 [59645,46773] 0 2024-04-26T04:53:03.173 INFO:tasks.workunit.client.1.smithi078.stdout:2/770: fsync d8/d11/d14/f40 0 2024-04-26T04:53:03.173 INFO:tasks.workunit.client.1.smithi078.stdout:1/817: chown d7/d1e/d9e/db/ldf 4 1 2024-04-26T04:53:03.175 INFO:tasks.workunit.client.1.smithi078.stdout:8/741: mkdir d0/d19/d52/d5c/df0 0 2024-04-26T04:53:03.176 INFO:tasks.workunit.client.1.smithi078.stdout:5/719: rmdir d7/da/d1e/da4 0 2024-04-26T04:53:03.176 INFO:tasks.workunit.client.1.smithi078.stdout:4/785: chown d0/d1d/de1/d20/f1b 815 1 2024-04-26T04:53:03.177 INFO:tasks.workunit.client.1.smithi078.stdout:6/844: creat d2/de/d27/da9/d3a/d42/f110 x:0 0 0 2024-04-26T04:53:03.178 INFO:tasks.workunit.client.1.smithi078.stdout:0/782: dread d2/d9/d31/ffa [0,4194304] 0 2024-04-26T04:53:03.178 INFO:tasks.workunit.client.1.smithi078.stdout:5/720: chown d7/d21/d2f/l80 13490968 1 2024-04-26T04:53:03.179 INFO:tasks.workunit.client.1.smithi078.stdout:1/818: fsync d7/d1e/d9e/db/d37/d58/ff4 0 2024-04-26T04:53:03.180 INFO:tasks.workunit.client.1.smithi078.stdout:4/786: readlink d0/d1d/de1/d20/l18 0 2024-04-26T04:53:03.180 INFO:tasks.workunit.client.1.smithi078.stdout:3/785: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f112 x:0 0 0 2024-04-26T04:53:03.182 INFO:tasks.workunit.client.1.smithi078.stdout:3/786: write d16/f69 [216994,26201] 0 2024-04-26T04:53:03.183 INFO:tasks.workunit.client.1.smithi078.stdout:2/771: mkdir d8/d11/d18/d5d/d8b/db0/d10d/d110 0 2024-04-26T04:53:03.184 INFO:tasks.workunit.client.1.smithi078.stdout:8/742: creat d0/d19/d52/d35/ff1 x:0 0 0 2024-04-26T04:53:03.187 INFO:tasks.workunit.client.1.smithi078.stdout:3/787: chown d16/d26/d2e/d74/l100 157553557 1 2024-04-26T04:53:03.187 INFO:tasks.workunit.client.1.smithi078.stdout:7/731: getdents d9/da/d33/d4b/d4c 0 2024-04-26T04:53:03.189 INFO:tasks.workunit.client.1.smithi078.stdout:1/819: rename d7/d1e/d9e/db/d49/c100 to d7/d1e/d9e/d4c/c10a 0 2024-04-26T04:53:03.190 INFO:tasks.workunit.client.1.smithi078.stdout:8/743: unlink d0/d19/d25/d32/c34 0 2024-04-26T04:53:03.190 INFO:tasks.workunit.client.1.smithi078.stdout:5/721: dwrite d7/d48/d57/faa [0,4194304] 0 2024-04-26T04:53:03.193 INFO:tasks.workunit.client.1.smithi078.stdout:5/722: chown d7/d48/d57/d65/d7a 1 1 2024-04-26T04:53:03.193 INFO:tasks.workunit.client.1.smithi078.stdout:7/732: creat d9/da/d33/d4b/d4c/d5b/ff0 x:0 0 0 2024-04-26T04:53:03.194 INFO:tasks.workunit.client.1.smithi078.stdout:8/744: read - d0/d19/d52/d35/d44/fdb zero size 2024-04-26T04:53:03.195 INFO:tasks.workunit.client.1.smithi078.stdout:6/845: dread d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:53:03.196 INFO:tasks.workunit.client.1.smithi078.stdout:3/788: unlink d16/d26/d2e/d5f/d49/dad/db0/db7/f102 0 2024-04-26T04:53:03.197 INFO:tasks.workunit.client.1.smithi078.stdout:4/787: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/f119 [1531149,34917] 0 2024-04-26T04:53:03.201 INFO:tasks.workunit.client.1.smithi078.stdout:7/733: unlink d9/da/d11/d13/d50/f86 0 2024-04-26T04:53:03.202 INFO:tasks.workunit.client.1.smithi078.stdout:2/772: truncate d8/d11/f2a 3549410 0 2024-04-26T04:53:03.206 INFO:tasks.workunit.client.1.smithi078.stdout:4/788: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/f11a x:0 0 0 2024-04-26T04:53:03.208 INFO:tasks.workunit.client.1.smithi078.stdout:7/734: dread d9/fd5 [8388608,4194304] 0 2024-04-26T04:53:03.210 INFO:tasks.workunit.client.1.smithi078.stdout:6/846: rmdir d2/de/d27/da9/d2a/dbf/d5f/df2 0 2024-04-26T04:53:03.211 INFO:tasks.workunit.client.1.smithi078.stdout:4/789: symlink d0/de/d22/d27/db5/l11b 0 2024-04-26T04:53:03.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/820: dwrite d7/d1e/d9e/db/d49/f84 [0,4194304] 0 2024-04-26T04:53:03.215 INFO:tasks.workunit.client.1.smithi078.stdout:7/735: dread - d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 zero size 2024-04-26T04:53:03.219 INFO:tasks.workunit.client.1.smithi078.stdout:7/736: chown d9/da/d33/d4b/d4c/fe1 189414 1 2024-04-26T04:53:03.220 INFO:tasks.workunit.client.1.smithi078.stdout:2/773: truncate d8/d11/d14/d69/d8f/d34/dd3/de4/f6a 1081822 0 2024-04-26T04:53:03.220 INFO:tasks.workunit.client.1.smithi078.stdout:3/789: dwrite d16/d3c/f5b [0,4194304] 0 2024-04-26T04:53:03.221 INFO:tasks.workunit.client.1.smithi078.stdout:6/847: link d2/d1a/d5c/f98 d2/de/d27/d58/d5b/d101/f111 0 2024-04-26T04:53:03.221 INFO:tasks.workunit.client.1.smithi078.stdout:5/723: dread d7/d18/f34 [0,4194304] 0 2024-04-26T04:53:03.224 INFO:tasks.workunit.client.1.smithi078.stdout:4/790: getdents d0/d1d/de1/d20/d31/d47/db2/dc3 0 2024-04-26T04:53:03.224 INFO:tasks.workunit.client.1.smithi078.stdout:2/774: write d8/d11/d6e/f7b [3545983,50228] 0 2024-04-26T04:53:03.225 INFO:tasks.workunit.client.1.smithi078.stdout:3/790: fdatasync d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f58 0 2024-04-26T04:53:03.225 INFO:tasks.workunit.client.1.smithi078.stdout:2/775: chown d8/d11/d14/d5b/f10e 541 1 2024-04-26T04:53:03.226 INFO:tasks.workunit.client.1.smithi078.stdout:3/791: fdatasync d16/d26/d2b/fa9 0 2024-04-26T04:53:03.226 INFO:tasks.workunit.client.1.smithi078.stdout:3/792: chown d16/f106 3381566 1 2024-04-26T04:53:03.228 INFO:tasks.workunit.client.1.smithi078.stdout:3/793: dread d16/d26/d2b/f40 [0,4194304] 0 2024-04-26T04:53:03.262 INFO:tasks.workunit.client.1.smithi078.stdout:9/703: sync 2024-04-26T04:53:03.263 INFO:tasks.workunit.client.1.smithi078.stdout:9/704: write dc/df/d8a/fb9 [259164,97798] 0 2024-04-26T04:53:03.292 INFO:tasks.workunit.client.1.smithi078.stdout:0/783: sync 2024-04-26T04:53:03.316 INFO:tasks.workunit.client.1.smithi078.stdout:5/724: dread d7/d48/f6f [0,4194304] 0 2024-04-26T04:53:03.317 INFO:tasks.workunit.client.1.smithi078.stdout:5/725: chown d7/d48/d57/d65/d7a/f98 4845 1 2024-04-26T04:53:03.318 INFO:tasks.workunit.client.1.smithi078.stdout:7/737: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/ff1 x:0 0 0 2024-04-26T04:53:03.318 INFO:tasks.workunit.client.1.smithi078.stdout:1/821: link d7/d1e/d9e/dd/lad d7/d1e/d9e/db/d37/d58/d5c/d68/l10b 0 2024-04-26T04:53:03.321 INFO:tasks.workunit.client.1.smithi078.stdout:7/738: dread d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e [0,4194304] 0 2024-04-26T04:53:03.322 INFO:tasks.workunit.client.1.smithi078.stdout:6/848: rename d2/de/d27/da9/d2a/f94 to d2/d1a/d38/dd5/ddf/f112 0 2024-04-26T04:53:03.325 INFO:tasks.workunit.client.1.smithi078.stdout:3/794: creat d16/d26/d2e/d5f/d49/d77/dba/f113 x:0 0 0 2024-04-26T04:53:03.326 INFO:tasks.workunit.client.1.smithi078.stdout:9/705: creat dc/df/d4b/d22/d2d/d98/fd4 x:0 0 0 2024-04-26T04:53:03.327 INFO:tasks.workunit.client.1.smithi078.stdout:0/784: creat d2/d9/d78/deb/f101 x:0 0 0 2024-04-26T04:53:03.328 INFO:tasks.workunit.client.1.smithi078.stdout:5/726: rmdir d7/d21/d2f 39 2024-04-26T04:53:03.328 INFO:tasks.workunit.client.1.smithi078.stdout:9/706: write dc/df/d4b/dae/fc1 [264900,57639] 0 2024-04-26T04:53:03.331 INFO:tasks.workunit.client.1.smithi078.stdout:7/739: unlink d9/da/d11/d13/d1a/d52/d5a/c35 0 2024-04-26T04:53:03.331 INFO:tasks.workunit.client.1.smithi078.stdout:6/849: read - d2/de/d27/da9/da/dab/fb3 zero size 2024-04-26T04:53:03.332 INFO:tasks.workunit.client.1.smithi078.stdout:0/785: creat d2/d1e/d5a/da3/f102 x:0 0 0 2024-04-26T04:53:03.333 INFO:tasks.workunit.client.1.smithi078.stdout:1/822: creat d7/d1e/d9e/f10c x:0 0 0 2024-04-26T04:53:03.334 INFO:tasks.workunit.client.1.smithi078.stdout:2/776: link d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/d91/c94 d8/d11/d14/d69/d8f/d34/dd6/c111 0 2024-04-26T04:53:03.335 INFO:tasks.workunit.client.1.smithi078.stdout:0/786: write d2/d9/d18/f49 [5209043,123795] 0 2024-04-26T04:53:03.336 INFO:tasks.workunit.client.1.smithi078.stdout:7/740: symlink d9/lf2 0 2024-04-26T04:53:03.336 INFO:tasks.workunit.client.1.smithi078.stdout:3/795: dwrite d16/f106 [0,4194304] 0 2024-04-26T04:53:03.337 INFO:tasks.workunit.client.1.smithi078.stdout:6/850: chown d2/d1a/d38/dd5/ffa 15 1 2024-04-26T04:53:03.337 INFO:tasks.workunit.client.1.smithi078.stdout:5/727: creat d7/d48/d57/d65/d70/dca/fed x:0 0 0 2024-04-26T04:53:03.339 INFO:tasks.workunit.client.1.smithi078.stdout:2/777: write d8/d11/d14/d5b/f10e [591583,62855] 0 2024-04-26T04:53:03.341 INFO:tasks.workunit.client.1.smithi078.stdout:0/787: write d2/d9/d60/d6a/f87 [2960562,21002] 0 2024-04-26T04:53:03.346 INFO:tasks.workunit.client.1.smithi078.stdout:5/728: rename d7/d18/d32/d37/db1 to d7/d11/dee 0 2024-04-26T04:53:03.347 INFO:tasks.workunit.client.1.smithi078.stdout:0/788: mkdir d2/d9/d78/d84/d103 0 2024-04-26T04:53:03.350 INFO:tasks.workunit.client.1.smithi078.stdout:8/745: sync 2024-04-26T04:53:03.350 INFO:tasks.workunit.client.1.smithi078.stdout:1/823: dwrite d7/d1e/d9e/db/f5a [0,4194304] 0 2024-04-26T04:53:03.350 INFO:tasks.workunit.client.1.smithi078.stdout:2/778: creat d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/f112 x:0 0 0 2024-04-26T04:53:03.351 INFO:tasks.workunit.client.1.smithi078.stdout:3/796: dwrite d16/d26/f7f [4194304,4194304] 0 2024-04-26T04:53:03.352 INFO:tasks.workunit.client.1.smithi078.stdout:7/741: rename d9/da/d11/d13/d1a/c88 to d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/cf3 0 2024-04-26T04:53:03.356 INFO:tasks.workunit.client.1.smithi078.stdout:8/746: symlink d0/d19/d52/d95/d9d/lf2 0 2024-04-26T04:53:03.356 INFO:tasks.workunit.client.1.smithi078.stdout:9/707: dread dc/df/d37/fa9 [0,4194304] 0 2024-04-26T04:53:03.358 INFO:tasks.workunit.client.1.smithi078.stdout:7/742: mkdir d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/df4 0 2024-04-26T04:53:03.359 INFO:tasks.workunit.client.1.smithi078.stdout:0/789: dwrite d2/d9/d60/d6a/f87 [0,4194304] 0 2024-04-26T04:53:03.360 INFO:tasks.workunit.client.1.smithi078.stdout:8/747: mkdir d0/d7a/d83/df3 0 2024-04-26T04:53:03.361 INFO:tasks.workunit.client.1.smithi078.stdout:8/748: chown d0/d19/d25/d36/f37 188704826 1 2024-04-26T04:53:03.361 INFO:tasks.workunit.client.1.smithi078.stdout:8/749: fsync d0/d19/d52/d35/d44/fc8 0 2024-04-26T04:53:03.362 INFO:tasks.workunit.client.1.smithi078.stdout:9/708: rename dc/df/d27/d81/da8/fc0 to dc/df/d37/fd5 0 2024-04-26T04:53:03.363 INFO:tasks.workunit.client.1.smithi078.stdout:8/750: write d0/d19/d52/d35/d44/fc8 [1182286,23401] 0 2024-04-26T04:53:03.365 INFO:tasks.workunit.client.1.smithi078.stdout:7/743: symlink d9/da/d33/d4b/lf5 0 2024-04-26T04:53:03.365 INFO:tasks.workunit.client.1.smithi078.stdout:2/779: link d8/d11/d14/f40 d8/d11/d14/f113 0 2024-04-26T04:53:03.366 INFO:tasks.workunit.client.1.smithi078.stdout:7/744: stat d9/da/d33/d4b/d4c/d5b/d83/fac 0 2024-04-26T04:53:03.366 INFO:tasks.workunit.client.1.smithi078.stdout:9/709: write dc/df/d4b/d22/d2a/d57/d69/fa0 [267545,98423] 0 2024-04-26T04:53:03.367 INFO:tasks.workunit.client.1.smithi078.stdout:9/710: readlink dc/l42 0 2024-04-26T04:53:03.368 INFO:tasks.workunit.client.1.smithi078.stdout:8/751: creat d0/d19/d25/d36/d58/d5b/ff4 x:0 0 0 2024-04-26T04:53:03.369 INFO:tasks.workunit.client.1.smithi078.stdout:2/780: write d8/d11/d18/dbb/f107 [1011154,19725] 0 2024-04-26T04:53:03.373 INFO:tasks.workunit.client.1.smithi078.stdout:0/790: dwrite d2/d9/d31/fe0 [4194304,4194304] 0 2024-04-26T04:53:03.377 INFO:tasks.workunit.client.1.smithi078.stdout:2/781: getdents d8/d11/d18/dbb 0 2024-04-26T04:53:03.378 INFO:tasks.workunit.client.1.smithi078.stdout:8/752: dread d0/d19/d52/d35/f5f [0,4194304] 0 2024-04-26T04:53:03.378 INFO:tasks.workunit.client.1.smithi078.stdout:2/782: dread - d8/ff2 zero size 2024-04-26T04:53:03.378 INFO:tasks.workunit.client.1.smithi078.stdout:7/745: dwrite d9/da/d11/f3a [0,4194304] 0 2024-04-26T04:53:03.379 INFO:tasks.workunit.client.1.smithi078.stdout:9/711: link dc/df/d4b/d22/d2a/d92/fa2 dc/df/d4b/d22/d2a/d57/d69/db3/fd6 0 2024-04-26T04:53:03.380 INFO:tasks.workunit.client.1.smithi078.stdout:0/791: mknod d2/d1e/d5a/df9/c104 0 2024-04-26T04:53:03.382 INFO:tasks.workunit.client.1.smithi078.stdout:7/746: symlink d9/da/d20/lf6 0 2024-04-26T04:53:03.383 INFO:tasks.workunit.client.1.smithi078.stdout:9/712: truncate dc/df/d4b/d22/d2d/d98/fbd 87695 0 2024-04-26T04:53:03.384 INFO:tasks.workunit.client.1.smithi078.stdout:0/792: creat d2/d9/d78/deb/f105 x:0 0 0 2024-04-26T04:53:03.385 INFO:tasks.workunit.client.1.smithi078.stdout:7/747: readlink d9/da/d11/d13/d1a/d52/d5a/l39 0 2024-04-26T04:53:03.385 INFO:tasks.workunit.client.1.smithi078.stdout:8/753: dread d0/d19/d52/d35/d44/fc8 [0,4194304] 0 2024-04-26T04:53:03.385 INFO:tasks.workunit.client.1.smithi078.stdout:0/793: stat d2/d39/l43 0 2024-04-26T04:53:03.389 INFO:tasks.workunit.client.1.smithi078.stdout:9/713: link dc/df/d4b/dae/fc1 dc/fd7 0 2024-04-26T04:53:03.391 INFO:tasks.workunit.client.1.smithi078.stdout:7/748: mknod d9/da/d33/d4b/d4c/d5b/d83/de6/cf7 0 2024-04-26T04:53:03.394 INFO:tasks.workunit.client.1.smithi078.stdout:0/794: dwrite d2/d1e/f67 [0,4194304] 0 2024-04-26T04:53:03.394 INFO:tasks.workunit.client.1.smithi078.stdout:7/749: symlink d9/da/d11/d13/lf8 0 2024-04-26T04:53:03.396 INFO:tasks.workunit.client.1.smithi078.stdout:7/750: readlink d9/da/d11/d13/d1a/d52/d5a/lb9 0 2024-04-26T04:53:03.396 INFO:tasks.workunit.client.1.smithi078.stdout:0/795: mknod d2/c106 0 2024-04-26T04:53:03.397 INFO:tasks.workunit.client.1.smithi078.stdout:7/751: dread - d9/da/d11/fa9 zero size 2024-04-26T04:53:03.401 INFO:tasks.workunit.client.1.smithi078.stdout:0/796: link d2/d1e/f56 d2/d9/d18/d7d/da9/dab/dc6/f107 0 2024-04-26T04:53:03.404 INFO:tasks.workunit.client.1.smithi078.stdout:7/752: getdents d9/da/d11/d13/d1a/d52/d5a/d28 0 2024-04-26T04:53:03.406 INFO:tasks.workunit.client.1.smithi078.stdout:7/753: creat d9/da/d33/d4b/d4c/d5b/d83/ff9 x:0 0 0 2024-04-26T04:53:03.407 INFO:tasks.workunit.client.1.smithi078.stdout:0/797: dwrite d2/d9/d31/d73/f83 [0,4194304] 0 2024-04-26T04:53:03.408 INFO:tasks.workunit.client.1.smithi078.stdout:0/798: dread - d2/d9/f75 zero size 2024-04-26T04:53:03.412 INFO:tasks.workunit.client.1.smithi078.stdout:0/799: read d2/d9/d31/fce [3117422,116132] 0 2024-04-26T04:53:03.413 INFO:tasks.workunit.client.1.smithi078.stdout:0/800: chown d2/d9/d18/d7d/da9/dab/dfd 451121 1 2024-04-26T04:53:03.415 INFO:tasks.workunit.client.1.smithi078.stdout:0/801: creat d2/d9/d60/dd5/f108 x:0 0 0 2024-04-26T04:53:03.418 INFO:tasks.workunit.client.1.smithi078.stdout:1/824: sync 2024-04-26T04:53:03.421 INFO:tasks.workunit.client.1.smithi078.stdout:1/825: rename d7/d1e/d9e/db/d37/d58/d5c/df2 to d7/d2e/d10d 0 2024-04-26T04:53:03.424 INFO:tasks.workunit.client.1.smithi078.stdout:1/826: read f1 [915504,78117] 0 2024-04-26T04:53:03.424 INFO:tasks.workunit.client.1.smithi078.stdout:1/827: chown d7/d1e/d9e/db/d47/f90 230779767 1 2024-04-26T04:53:03.430 INFO:tasks.workunit.client.1.smithi078.stdout:1/828: dwrite d7/d1e/d9e/dd/df0/ff5 [0,4194304] 0 2024-04-26T04:53:03.431 INFO:tasks.workunit.client.1.smithi078.stdout:1/829: chown d7/d1e/d9e/dd/fd5 52889 1 2024-04-26T04:53:03.433 INFO:tasks.workunit.client.1.smithi078.stdout:7/754: dread d9/da/d11/d13/d50/f7c [0,4194304] 0 2024-04-26T04:53:03.434 INFO:tasks.workunit.client.1.smithi078.stdout:1/830: dread d7/d1e/d9e/db/f15 [0,4194304] 0 2024-04-26T04:53:03.436 INFO:tasks.workunit.client.1.smithi078.stdout:1/831: chown d7/d1e/d9e/db/d37/d58/d5c/d68/lae 61013 1 2024-04-26T04:53:03.437 INFO:tasks.workunit.client.1.smithi078.stdout:7/755: creat d9/da/d33/d4b/d4c/ffa x:0 0 0 2024-04-26T04:53:03.438 INFO:tasks.workunit.client.1.smithi078.stdout:7/756: write d9/f90 [1415823,29550] 0 2024-04-26T04:53:03.439 INFO:tasks.workunit.client.1.smithi078.stdout:1/832: mknod d7/d1e/d9e/db/d49/dba/dc5/dcf/c10e 0 2024-04-26T04:53:03.442 INFO:tasks.workunit.client.1.smithi078.stdout:1/833: read d7/d1e/d9e/db/d47/f90 [2709437,102152] 0 2024-04-26T04:53:03.443 INFO:tasks.workunit.client.1.smithi078.stdout:3/797: fsync d16/d3c/f5b 0 2024-04-26T04:53:03.464 INFO:tasks.workunit.client.1.smithi078.stdout:6/851: write d2/fa0 [545085,71679] 0 2024-04-26T04:53:03.465 INFO:tasks.workunit.client.1.smithi078.stdout:5/729: truncate d7/d18/d1c/f9e 2098951 0 2024-04-26T04:53:03.466 INFO:tasks.workunit.client.1.smithi078.stdout:4/791: dwrite d0/de/d22/d27/db5/fff [0,4194304] 0 2024-04-26T04:53:03.467 INFO:tasks.workunit.client.1.smithi078.stdout:5/730: chown d7/da/f73 8110 1 2024-04-26T04:53:03.467 INFO:tasks.workunit.client.1.smithi078.stdout:4/792: dread - d0/f112 zero size 2024-04-26T04:53:03.471 INFO:tasks.workunit.client.1.smithi078.stdout:6/852: creat d2/de/d27/da9/da8/dc8/f113 x:0 0 0 2024-04-26T04:53:03.472 INFO:tasks.workunit.client.1.smithi078.stdout:5/731: symlink d7/d48/d57/d65/d46/de7/lef 0 2024-04-26T04:53:03.473 INFO:tasks.workunit.client.1.smithi078.stdout:2/783: truncate d8/d11/d14/f67 2123298 0 2024-04-26T04:53:03.476 INFO:tasks.workunit.client.1.smithi078.stdout:8/754: truncate d0/fae 2553803 0 2024-04-26T04:53:03.477 INFO:tasks.workunit.client.1.smithi078.stdout:6/853: rename d2/de/d27/d58/lb4 to d2/de/d27/da9/d11/l114 0 2024-04-26T04:53:03.477 INFO:tasks.workunit.client.1.smithi078.stdout:9/714: truncate dc/df/f2f 733953 0 2024-04-26T04:53:03.479 INFO:tasks.workunit.client.1.smithi078.stdout:5/732: dwrite d7/d18/d1c/d5a/f7c [0,4194304] 0 2024-04-26T04:53:03.482 INFO:tasks.workunit.client.1.smithi078.stdout:9/715: dread dc/f4e [0,4194304] 0 2024-04-26T04:53:03.482 INFO:tasks.workunit.client.1.smithi078.stdout:8/755: link d0/d19/d25/ld2 d0/d7a/dbe/lf5 0 2024-04-26T04:53:03.483 INFO:tasks.workunit.client.1.smithi078.stdout:8/756: fdatasync d0/d19/f1c 0 2024-04-26T04:53:03.484 INFO:tasks.workunit.client.1.smithi078.stdout:5/733: truncate d7/d48/d57/f89 787375 0 2024-04-26T04:53:03.484 INFO:tasks.workunit.client.1.smithi078.stdout:0/802: write d2/d39/f7c [72878,49957] 0 2024-04-26T04:53:03.484 INFO:tasks.workunit.client.1.smithi078.stdout:2/784: getdents d8/d11/d18/d5d/d8b/db0 0 2024-04-26T04:53:03.487 INFO:tasks.workunit.client.1.smithi078.stdout:9/716: fdatasync dc/df/d27/f13 0 2024-04-26T04:53:03.489 INFO:tasks.workunit.client.1.smithi078.stdout:0/803: creat d2/d9/d60/dae/f109 x:0 0 0 2024-04-26T04:53:03.491 INFO:tasks.workunit.client.1.smithi078.stdout:5/734: mkdir d7/d18/d1c/d5a/d96/df0 0 2024-04-26T04:53:03.494 INFO:tasks.workunit.client.1.smithi078.stdout:0/804: unlink d2/l91 0 2024-04-26T04:53:03.494 INFO:tasks.workunit.client.1.smithi078.stdout:8/757: dwrite d0/d19/d25/d36/d58/d5b/d61/fde [0,4194304] 0 2024-04-26T04:53:03.500 INFO:tasks.workunit.client.1.smithi078.stdout:0/805: rename d2/d9/d60/cee to d2/d9/d18/d7d/da9/dab/c10a 0 2024-04-26T04:53:03.502 INFO:tasks.workunit.client.1.smithi078.stdout:8/758: link d0/d19/d52/d5c/fb5 d0/d7a/dbe/ff6 0 2024-04-26T04:53:03.503 INFO:tasks.workunit.client.1.smithi078.stdout:4/793: dread d0/d1d/db4/ff8 [0,4194304] 0 2024-04-26T04:53:03.505 INFO:tasks.workunit.client.1.smithi078.stdout:0/806: truncate d2/d9/d31/d4a/f7a 154189 0 2024-04-26T04:53:03.506 INFO:tasks.workunit.client.1.smithi078.stdout:1/834: write d7/d1e/d9e/db/d37/f65 [685505,128090] 0 2024-04-26T04:53:03.507 INFO:tasks.workunit.client.1.smithi078.stdout:5/735: getdents d7/d21/d2f/d90 0 2024-04-26T04:53:03.508 INFO:tasks.workunit.client.1.smithi078.stdout:3/798: write d16/d26/d2b/d31/f65 [4197423,82806] 0 2024-04-26T04:53:03.511 INFO:tasks.workunit.client.1.smithi078.stdout:5/736: stat d7/d48/d57/d65/d46/dce 0 2024-04-26T04:53:03.511 INFO:tasks.workunit.client.1.smithi078.stdout:0/807: write d2/d9/d31/d4a/fa2 [8197501,6093] 0 2024-04-26T04:53:03.514 INFO:tasks.workunit.client.1.smithi078.stdout:8/759: dwrite d0/d19/d25/fd6 [0,4194304] 0 2024-04-26T04:53:03.520 INFO:tasks.workunit.client.1.smithi078.stdout:3/799: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f114 x:0 0 0 2024-04-26T04:53:03.523 INFO:tasks.workunit.client.1.smithi078.stdout:1/835: link d7/d1e/d9e/db/d37/f65 d7/d1e/d9e/db/d49/df8/f10f 0 2024-04-26T04:53:03.524 INFO:tasks.workunit.client.1.smithi078.stdout:1/836: write d7/d1e/d9e/db/d37/f88 [2128961,81567] 0 2024-04-26T04:53:03.525 INFO:tasks.workunit.client.1.smithi078.stdout:3/800: symlink d16/d26/d2e/d5f/dbb/l115 0 2024-04-26T04:53:03.528 INFO:tasks.workunit.client.1.smithi078.stdout:2/785: write d8/d11/d14/d69/d8f/d34/f3d [150974,128921] 0 2024-04-26T04:53:03.531 INFO:tasks.workunit.client.1.smithi078.stdout:3/801: creat d16/d7e/f116 x:0 0 0 2024-04-26T04:53:03.531 INFO:tasks.workunit.client.1.smithi078.stdout:5/737: link d7/d18/d32/d84/ca5 d7/d18/d32/d37/d79/da7/cf1 0 2024-04-26T04:53:03.533 INFO:tasks.workunit.client.1.smithi078.stdout:8/760: getdents d0/d19/d52/d35/d73 0 2024-04-26T04:53:03.534 INFO:tasks.workunit.client.1.smithi078.stdout:2/786: dread - d8/d11/d14/d69/d8f/fba zero size 2024-04-26T04:53:03.536 INFO:tasks.workunit.client.1.smithi078.stdout:3/802: creat d16/d26/d2e/d5f/d49/dad/db0/f117 x:0 0 0 2024-04-26T04:53:03.537 INFO:tasks.workunit.client.1.smithi078.stdout:8/761: readlink d0/d19/d25/ld2 0 2024-04-26T04:53:03.538 INFO:tasks.workunit.client.1.smithi078.stdout:9/717: dwrite dc/df/d37/fa9 [0,4194304] 0 2024-04-26T04:53:03.539 INFO:tasks.workunit.client.1.smithi078.stdout:8/762: write d0/d7a/dbe/fe1 [369878,30020] 0 2024-04-26T04:53:03.540 INFO:tasks.workunit.client.1.smithi078.stdout:4/794: truncate d0/de/d22/d71/fc7 447826 0 2024-04-26T04:53:03.541 INFO:tasks.workunit.client.1.smithi078.stdout:8/763: stat d0/d19/d52/d35/d73 0 2024-04-26T04:53:03.544 INFO:tasks.workunit.client.1.smithi078.stdout:2/787: creat d8/d11/d14/d69/d8f/f114 x:0 0 0 2024-04-26T04:53:03.548 INFO:tasks.workunit.client.1.smithi078.stdout:9/718: mknod dc/df/cd8 0 2024-04-26T04:53:03.550 INFO:tasks.workunit.client.1.smithi078.stdout:8/764: mkdir d0/d19/d52/d35/d44/df7 0 2024-04-26T04:53:03.553 INFO:tasks.workunit.client.1.smithi078.stdout:8/765: read d0/d19/d25/d36/d58/d5b/d61/fd9 [811364,109259] 0 2024-04-26T04:53:03.553 INFO:tasks.workunit.client.1.smithi078.stdout:2/788: dwrite d8/d11/d14/d5b/f10e [0,4194304] 0 2024-04-26T04:53:03.555 INFO:tasks.workunit.client.1.smithi078.stdout:2/789: rename d8/d11/d14 to d8/d11/d14/d69/d8f/d34/d70/def/d115 22 2024-04-26T04:53:03.556 INFO:tasks.workunit.client.1.smithi078.stdout:1/837: dread d7/f11 [0,4194304] 0 2024-04-26T04:53:03.557 INFO:tasks.workunit.client.1.smithi078.stdout:9/719: mkdir dc/dd9 0 2024-04-26T04:53:03.557 INFO:tasks.workunit.client.1.smithi078.stdout:2/790: read d8/d11/d14/d69/d8f/f6b [2780818,32278] 0 2024-04-26T04:53:03.558 INFO:tasks.workunit.client.1.smithi078.stdout:8/766: write d0/d19/d25/d32/f76 [3947871,54612] 0 2024-04-26T04:53:03.560 INFO:tasks.workunit.client.1.smithi078.stdout:8/767: readlink d0/d19/d25/l2e 0 2024-04-26T04:53:03.560 INFO:tasks.workunit.client.1.smithi078.stdout:5/738: dread d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:53:03.563 INFO:tasks.workunit.client.1.smithi078.stdout:9/720: unlink dc/df/d37/fa9 0 2024-04-26T04:53:03.563 INFO:tasks.workunit.client.1.smithi078.stdout:3/803: dread d16/d26/d2b/d31/f65 [0,4194304] 0 2024-04-26T04:53:03.564 INFO:tasks.workunit.client.1.smithi078.stdout:2/791: truncate d8/d11/f3f 486594 0 2024-04-26T04:53:03.565 INFO:tasks.workunit.client.1.smithi078.stdout:2/792: readlink d8/d11/d14/d69/l82 0 2024-04-26T04:53:03.565 INFO:tasks.workunit.client.1.smithi078.stdout:1/838: rename d7/d1e/d9e/dd/d20/f7b to d7/d1e/d9e/db/d37/d58/d5c/dc0/f110 0 2024-04-26T04:53:03.566 INFO:tasks.workunit.client.1.smithi078.stdout:9/721: truncate dc/df/d4b/d22/d2a/d57/fa4 1556825 0 2024-04-26T04:53:03.569 INFO:tasks.workunit.client.1.smithi078.stdout:2/793: write d8/d11/d18/d5d/d61/f3c [3619214,67838] 0 2024-04-26T04:53:03.572 INFO:tasks.workunit.client.1.smithi078.stdout:5/739: creat d7/d11/ff2 x:0 0 0 2024-04-26T04:53:03.574 INFO:tasks.workunit.client.1.smithi078.stdout:9/722: dwrite dc/df/d17/fcf [0,4194304] 0 2024-04-26T04:53:03.575 INFO:tasks.workunit.client.1.smithi078.stdout:1/839: symlink d7/d1e/d9e/d4c/l111 0 2024-04-26T04:53:03.577 INFO:tasks.workunit.client.1.smithi078.stdout:5/740: mknod d7/d48/da9/cf3 0 2024-04-26T04:53:03.577 INFO:tasks.workunit.client.1.smithi078.stdout:3/804: link d16/d26/d2e/ced d16/d26/d2b/d31/d71/d8a/dbc/c118 0 2024-04-26T04:53:03.578 INFO:tasks.workunit.client.1.smithi078.stdout:8/768: getdents d0/d19/d52/d35/d44/d6b 0 2024-04-26T04:53:03.579 INFO:tasks.workunit.client.1.smithi078.stdout:2/794: symlink d8/d11/d18/d5d/l116 0 2024-04-26T04:53:03.582 INFO:tasks.workunit.client.1.smithi078.stdout:1/840: dwrite d7/d1e/d9e/db/d47/f102 [0,4194304] 0 2024-04-26T04:53:03.584 INFO:tasks.workunit.client.1.smithi078.stdout:7/757: sync 2024-04-26T04:53:03.585 INFO:tasks.workunit.client.1.smithi078.stdout:9/723: getdents dc/df/d4b/d22/d2a/d57/d99 0 2024-04-26T04:53:03.587 INFO:tasks.workunit.client.1.smithi078.stdout:5/741: symlink d7/d21/lf4 0 2024-04-26T04:53:03.590 INFO:tasks.workunit.client.1.smithi078.stdout:7/758: truncate d9/da/d33/d4b/d4c/d5b/dc3/fdc 987613 0 2024-04-26T04:53:03.592 INFO:tasks.workunit.client.1.smithi078.stdout:5/742: rmdir d7/d48/d57 39 2024-04-26T04:53:03.593 INFO:tasks.workunit.client.1.smithi078.stdout:5/743: chown d7/d18/d32/d37/d79/ca8 0 1 2024-04-26T04:53:03.597 INFO:tasks.workunit.client.1.smithi078.stdout:1/841: creat d7/d1e/d9e/db/d49/dba/d87/db5/f112 x:0 0 0 2024-04-26T04:53:03.597 INFO:tasks.workunit.client.1.smithi078.stdout:1/842: chown d7/d1e 5753 1 2024-04-26T04:53:03.598 INFO:tasks.workunit.client.1.smithi078.stdout:1/843: write d7/d1e/fe2 [817286,53791] 0 2024-04-26T04:53:03.598 INFO:tasks.workunit.client.1.smithi078.stdout:7/759: creat d9/da/d11/d13/d1a/d52/d5a/d28/dab/ffb x:0 0 0 2024-04-26T04:53:03.598 INFO:tasks.workunit.client.1.smithi078.stdout:3/805: write d16/d26/d2e/f43 [66536,16600] 0 2024-04-26T04:53:03.601 INFO:tasks.workunit.client.1.smithi078.stdout:8/769: write d0/fae [1183655,17229] 0 2024-04-26T04:53:03.602 INFO:tasks.workunit.client.1.smithi078.stdout:1/844: fsync d7/d1e/d9e/dd/d20/f108 0 2024-04-26T04:53:03.603 INFO:tasks.workunit.client.1.smithi078.stdout:8/770: chown d0/d19/d52/d35/ldd 25616868 1 2024-04-26T04:53:03.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/795: write d8/d11/d14/d69/d8f/d34/dd3/de4/f6a [1785457,113782] 0 2024-04-26T04:53:03.604 INFO:tasks.workunit.client.1.smithi078.stdout:3/806: write d16/d26/d2e/d5f/ff5 [748076,89508] 0 2024-04-26T04:53:03.610 INFO:tasks.workunit.client.1.smithi078.stdout:1/845: rmdir d7/d1e/d9e/dd/d20/d53 39 2024-04-26T04:53:03.611 INFO:tasks.workunit.client.1.smithi078.stdout:7/760: getdents d9/da/d11/d13 0 2024-04-26T04:53:03.611 INFO:tasks.workunit.client.1.smithi078.stdout:8/771: dread - d0/d19/d52/d35/ff1 zero size 2024-04-26T04:53:03.612 INFO:tasks.workunit.client.1.smithi078.stdout:2/796: chown d8/d11/d18/c2c 332416 1 2024-04-26T04:53:03.612 INFO:tasks.workunit.client.1.smithi078.stdout:1/846: write d7/ff7 [56026,55273] 0 2024-04-26T04:53:03.614 INFO:tasks.workunit.client.1.smithi078.stdout:8/772: chown d0/d19/d52/d95/fc7 62819 1 2024-04-26T04:53:03.615 INFO:tasks.workunit.client.1.smithi078.stdout:1/847: write d7/d1e/d9e/dd/df0/ff5 [1019963,67873] 0 2024-04-26T04:53:03.617 INFO:tasks.workunit.client.1.smithi078.stdout:5/744: dread d7/d21/d2f/db6/fd9 [0,4194304] 0 2024-04-26T04:53:03.618 INFO:tasks.workunit.client.1.smithi078.stdout:9/724: dread dc/df/d4b/d22/d2a/d92/fad [0,4194304] 0 2024-04-26T04:53:03.619 INFO:tasks.workunit.client.1.smithi078.stdout:5/745: read d7/d18/d32/f4a [6196954,31456] 0 2024-04-26T04:53:03.622 INFO:tasks.workunit.client.1.smithi078.stdout:3/807: mknod d16/d26/d2e/d5f/d49/d77/dba/c119 0 2024-04-26T04:53:03.623 INFO:tasks.workunit.client.1.smithi078.stdout:7/761: unlink d9/da/d33/fdb 0 2024-04-26T04:53:03.626 INFO:tasks.workunit.client.1.smithi078.stdout:8/773: write d0/d19/d52/d35/f5f [658001,109750] 0 2024-04-26T04:53:03.627 INFO:tasks.workunit.client.1.smithi078.stdout:9/725: dread dc/df/f1d [0,4194304] 0 2024-04-26T04:53:03.628 INFO:tasks.workunit.client.1.smithi078.stdout:2/797: symlink d8/d11/d14/d69/d8f/d34/d70/def/l117 0 2024-04-26T04:53:03.631 INFO:tasks.workunit.client.1.smithi078.stdout:1/848: getdents d7/d1e/d9e/db/d49/dba/dc5/dcf 0 2024-04-26T04:53:03.633 INFO:tasks.workunit.client.1.smithi078.stdout:3/808: dwrite d16/f69 [0,4194304] 0 2024-04-26T04:53:03.635 INFO:tasks.workunit.client.1.smithi078.stdout:7/762: dwrite d9/da/d11/f45 [0,4194304] 0 2024-04-26T04:53:03.635 INFO:tasks.workunit.client.1.smithi078.stdout:6/854: sync 2024-04-26T04:53:03.637 INFO:tasks.workunit.client.1.smithi078.stdout:2/798: creat d8/d11/d14/d5b/f118 x:0 0 0 2024-04-26T04:53:03.641 INFO:tasks.workunit.client.1.smithi078.stdout:1/849: symlink d7/d1e/d9e/l113 0 2024-04-26T04:53:03.642 INFO:tasks.workunit.client.1.smithi078.stdout:9/726: dwrite dc/df/d37/fd5 [0,4194304] 0 2024-04-26T04:53:03.644 INFO:tasks.workunit.client.1.smithi078.stdout:3/809: mkdir d16/d3c/d88/d11a 0 2024-04-26T04:53:03.649 INFO:tasks.workunit.client.1.smithi078.stdout:8/774: getdents d0/d19/d25/d32/d8a 0 2024-04-26T04:53:03.651 INFO:tasks.workunit.client.1.smithi078.stdout:7/763: write d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc2 [706168,113005] 0 2024-04-26T04:53:03.656 INFO:tasks.workunit.client.1.smithi078.stdout:2/799: rename d8/d11/d14/d69/d8f/d34/fe0 to d8/d11/d18/d5d/f119 0 2024-04-26T04:53:03.656 INFO:tasks.workunit.client.1.smithi078.stdout:3/810: dwrite d16/d7e/fea [0,4194304] 0 2024-04-26T04:53:03.657 INFO:tasks.workunit.client.1.smithi078.stdout:7/764: write d9/da/d33/d4b/d4c/d5b/d83/fac [725714,106195] 0 2024-04-26T04:53:03.658 INFO:tasks.workunit.client.1.smithi078.stdout:9/727: link dc/df/d4b/d22/d2a/f4a dc/df/d4b/d22/dbe/dcd/fda 0 2024-04-26T04:53:03.658 INFO:tasks.workunit.client.0.smithi077.stdout:0/306: sync 2024-04-26T04:53:03.661 INFO:tasks.workunit.client.1.smithi078.stdout:1/850: truncate d7/d1e/d9e/db/d49/fd8 76042 0 2024-04-26T04:53:03.661 INFO:tasks.workunit.client.1.smithi078.stdout:9/728: chown dc/df/d4b/d22/d2a/d57/d69/db3 8971 1 2024-04-26T04:53:03.665 INFO:tasks.workunit.client.1.smithi078.stdout:7/765: creat d9/da/d11/d13/d1a/d52/d5a/d28/d6d/ffc x:0 0 0 2024-04-26T04:53:03.666 INFO:tasks.workunit.client.1.smithi078.stdout:8/775: dwrite d0/d19/d52/d35/f8f [0,4194304] 0 2024-04-26T04:53:03.667 INFO:tasks.workunit.client.1.smithi078.stdout:9/729: mkdir dc/df/d4b/d22/d2a/d92/ddb 0 2024-04-26T04:53:03.667 INFO:tasks.workunit.client.1.smithi078.stdout:8/776: read - d0/d19/d25/d32/d8a/d92/da4/da7/fda zero size 2024-04-26T04:53:03.668 INFO:tasks.workunit.client.1.smithi078.stdout:5/746: truncate d7/d48/d57/d65/fb0 2996687 0 2024-04-26T04:53:03.671 INFO:tasks.workunit.client.1.smithi078.stdout:6/855: write d2/d1a/fa6 [1053209,18236] 0 2024-04-26T04:53:03.673 INFO:tasks.workunit.client.1.smithi078.stdout:1/851: rename d7/d1e/d9e/db/d49/d93/faf to d7/d1e/d9e/d92/f114 0 2024-04-26T04:53:03.674 INFO:tasks.workunit.client.1.smithi078.stdout:2/800: dread d8/d11/d14/f90 [0,4194304] 0 2024-04-26T04:53:03.676 INFO:tasks.workunit.client.1.smithi078.stdout:3/811: creat d16/d3c/f11b x:0 0 0 2024-04-26T04:53:03.676 INFO:tasks.workunit.client.1.smithi078.stdout:7/766: dwrite d9/da/d11/d13/d1a/d52/fca [0,4194304] 0 2024-04-26T04:53:03.676 INFO:tasks.workunit.client.1.smithi078.stdout:6/856: chown d2/de/d27/da9/d3a/d69/dee/f10f 29310453 1 2024-04-26T04:53:03.678 INFO:tasks.workunit.client.1.smithi078.stdout:2/801: write d8/d11/d6e/fdd [1446923,107608] 0 2024-04-26T04:53:03.680 INFO:tasks.workunit.client.1.smithi078.stdout:8/777: mkdir d0/df8 0 2024-04-26T04:53:03.681 INFO:tasks.workunit.client.1.smithi078.stdout:9/730: chown dc/df/d4b/d22/d2a/f4a 11288 1 2024-04-26T04:53:03.681 INFO:tasks.workunit.client.1.smithi078.stdout:2/802: write d8/f84 [353563,101077] 0 2024-04-26T04:53:03.681 INFO:tasks.workunit.client.1.smithi078.stdout:7/767: chown d9/da/d11/d13/d1a/d52/d5a/f44 27303 1 2024-04-26T04:53:03.682 INFO:tasks.workunit.client.1.smithi078.stdout:5/747: mknod d7/cf5 0 2024-04-26T04:53:03.685 INFO:tasks.workunit.client.1.smithi078.stdout:6/857: truncate d2/de/d27/da9/d2a/dbf/f10b 519980 0 2024-04-26T04:53:03.687 INFO:tasks.workunit.client.1.smithi078.stdout:2/803: rmdir d8/d11/d18/dbb 39 2024-04-26T04:53:03.687 INFO:tasks.workunit.client.0.smithi077.stdout:0/307: write d0/d47/f22 [450321,16960] 0 2024-04-26T04:53:03.688 INFO:tasks.workunit.client.1.smithi078.stdout:1/852: dwrite d7/d1e/d9e/db/d37/f88 [0,4194304] 0 2024-04-26T04:53:03.688 INFO:tasks.workunit.client.1.smithi078.stdout:9/731: write dc/df/d4b/d22/d2d/d98/fb4 [1018380,40820] 0 2024-04-26T04:53:03.690 INFO:tasks.workunit.client.1.smithi078.stdout:7/768: mkdir d9/dfd 0 2024-04-26T04:53:03.693 INFO:tasks.workunit.client.1.smithi078.stdout:5/748: read d7/d18/f44 [869383,74894] 0 2024-04-26T04:53:03.694 INFO:tasks.workunit.client.0.smithi077.stdout:0/308: symlink d0/d47/l6b 0 2024-04-26T04:53:03.695 INFO:tasks.workunit.client.1.smithi078.stdout:8/778: dwrite d0/d7a/dbe/fd8 [0,4194304] 0 2024-04-26T04:53:03.695 INFO:tasks.workunit.client.0.smithi077.stdout:0/309: write d0/d2c/d4e/f4c [7700,33108] 0 2024-04-26T04:53:03.697 INFO:tasks.workunit.client.0.smithi077.stdout:0/310: truncate d0/d2c/f2e 4949386 0 2024-04-26T04:53:03.698 INFO:tasks.workunit.client.1.smithi078.stdout:8/779: chown d0/d19/d25/d36/d58/fb7 3 1 2024-04-26T04:53:03.702 INFO:tasks.workunit.client.1.smithi078.stdout:6/858: creat d2/de/d27/da9/d3a/d69/daa/dba/dc4/f115 x:0 0 0 2024-04-26T04:53:03.703 INFO:tasks.workunit.client.1.smithi078.stdout:9/732: dwrite dc/df/d4b/d22/d2a/d57/fbf [0,4194304] 0 2024-04-26T04:53:03.711 INFO:tasks.workunit.client.0.smithi077.stdout:0/311: rename d0/d2c/d4e/c5f to d0/c6c 0 2024-04-26T04:53:03.714 INFO:tasks.workunit.client.1.smithi078.stdout:6/859: mkdir d2/d1a/d38/dd5/d116 0 2024-04-26T04:53:03.714 INFO:tasks.workunit.client.1.smithi078.stdout:6/860: chown d2/de/d27/da9/da/d35/fae 271963111 1 2024-04-26T04:53:03.715 INFO:tasks.workunit.client.0.smithi077.stdout:0/312: creat d0/d47/d64/f6d x:0 0 0 2024-04-26T04:53:03.715 INFO:tasks.workunit.client.1.smithi078.stdout:9/733: creat dc/df/d4b/dae/fdc x:0 0 0 2024-04-26T04:53:03.716 INFO:tasks.workunit.client.1.smithi078.stdout:7/769: rename d9/da/f23 to d9/da/d11/d13/d1a/d52/d5a/d28/ffe 0 2024-04-26T04:53:03.717 INFO:tasks.workunit.client.0.smithi077.stdout:0/313: creat d0/d19/d1e/d59/f6e x:0 0 0 2024-04-26T04:53:03.718 INFO:tasks.workunit.client.1.smithi078.stdout:7/770: chown d9/da/d11/d13/d1a/d52/d5a/l39 132218 1 2024-04-26T04:53:03.720 INFO:tasks.workunit.client.1.smithi078.stdout:9/734: symlink dc/df/d4b/d3e/d80/ldd 0 2024-04-26T04:53:03.721 INFO:tasks.workunit.client.1.smithi078.stdout:6/861: dwrite d2/de/d27/da9/d3a/d42/f7d [4194304,4194304] 0 2024-04-26T04:53:03.722 INFO:tasks.workunit.client.1.smithi078.stdout:9/735: truncate dc/df/fd1 999693 0 2024-04-26T04:53:03.723 INFO:tasks.workunit.client.1.smithi078.stdout:2/804: write d8/d11/d14/d69/d8f/f6b [4511354,35582] 0 2024-04-26T04:53:03.723 INFO:tasks.workunit.client.0.smithi077.stdout:0/314: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:53:03.726 INFO:tasks.workunit.client.1.smithi078.stdout:9/736: write dc/fd7 [785119,43019] 0 2024-04-26T04:53:03.728 INFO:tasks.workunit.client.1.smithi078.stdout:1/853: write d7/d1e/d9e/d4c/d3d/f95 [279884,110178] 0 2024-04-26T04:53:03.728 INFO:tasks.workunit.client.0.smithi077.stdout:0/315: mknod d0/d47/c6f 0 2024-04-26T04:53:03.731 INFO:tasks.workunit.client.0.smithi077.stdout:0/316: symlink d0/d47/d64/l70 0 2024-04-26T04:53:03.732 INFO:tasks.workunit.client.1.smithi078.stdout:5/749: dwrite d7/d18/f44 [0,4194304] 0 2024-04-26T04:53:03.732 INFO:tasks.workunit.client.0.smithi077.stdout:0/317: chown d0/d2c/d4e/f4c 9 1 2024-04-26T04:53:03.736 INFO:tasks.workunit.client.0.smithi077.stdout:0/318: rename d0/c42 to d0/d19/d1e/d59/d5a/c71 0 2024-04-26T04:53:03.738 INFO:tasks.workunit.client.1.smithi078.stdout:5/750: creat d7/da/d1e/d91/ff6 x:0 0 0 2024-04-26T04:53:03.738 INFO:tasks.workunit.client.1.smithi078.stdout:8/780: dwrite d0/d19/d25/d32/d8a/d92/fcf [0,4194304] 0 2024-04-26T04:53:03.738 INFO:tasks.workunit.client.0.smithi077.stdout:0/319: truncate d0/d2c/d4e/f3e 566597 0 2024-04-26T04:53:03.739 INFO:tasks.workunit.client.0.smithi077.stdout:0/320: read - d0/d47/f55 zero size 2024-04-26T04:53:03.742 INFO:tasks.workunit.client.1.smithi078.stdout:8/781: rmdir d0/d7a 39 2024-04-26T04:53:03.744 INFO:tasks.workunit.client.1.smithi078.stdout:5/751: dwrite d7/da/d1e/d91/ff6 [0,4194304] 0 2024-04-26T04:53:03.744 INFO:tasks.workunit.client.1.smithi078.stdout:7/771: write d9/da/d33/d4b/d4c/d5b/f5c [448348,59262] 0 2024-04-26T04:53:03.745 INFO:tasks.workunit.client.1.smithi078.stdout:2/805: write d8/d11/d14/fab [136223,37121] 0 2024-04-26T04:53:03.747 INFO:tasks.workunit.client.1.smithi078.stdout:8/782: unlink d0/d7a/fe6 0 2024-04-26T04:53:03.749 INFO:tasks.workunit.client.0.smithi077.stdout:0/321: getdents d0/d19/d1e/d2f/d52 0 2024-04-26T04:53:03.752 INFO:tasks.workunit.client.1.smithi078.stdout:5/752: rename d7/d18/d1c/d5a/d96/fe3 to d7/d48/d57/d65/d46/ff7 0 2024-04-26T04:53:03.752 INFO:tasks.workunit.client.1.smithi078.stdout:4/795: sync 2024-04-26T04:53:03.752 INFO:tasks.workunit.client.1.smithi078.stdout:0/808: sync 2024-04-26T04:53:03.755 INFO:tasks.workunit.client.1.smithi078.stdout:0/809: fsync d2/d9/d78/deb/f105 0 2024-04-26T04:53:03.755 INFO:tasks.workunit.client.1.smithi078.stdout:4/796: chown d0/de/d22/d27/de7 364 1 2024-04-26T04:53:03.755 INFO:tasks.workunit.client.1.smithi078.stdout:2/806: dread d8/d11/f3f [0,4194304] 0 2024-04-26T04:53:03.757 INFO:tasks.workunit.client.0.smithi077.stdout:0/322: rename d0/d47/l45 to d0/d28/l72 0 2024-04-26T04:53:03.757 INFO:tasks.workunit.client.1.smithi078.stdout:8/783: mknod d0/d19/d25/de0/cf9 0 2024-04-26T04:53:03.758 INFO:tasks.workunit.client.1.smithi078.stdout:5/753: dread d7/da/f75 [0,4194304] 0 2024-04-26T04:53:03.758 INFO:tasks.workunit.client.1.smithi078.stdout:8/784: readlink d0/lca 0 2024-04-26T04:53:03.760 INFO:tasks.workunit.client.0.smithi077.stdout:0/323: creat d0/d19/d1e/d59/d5a/f73 x:0 0 0 2024-04-26T04:53:03.761 INFO:tasks.workunit.client.1.smithi078.stdout:5/754: write d7/d18/f27 [5112653,10808] 0 2024-04-26T04:53:03.762 INFO:tasks.workunit.client.1.smithi078.stdout:2/807: rename d8/d11/d18/f109 to d8/d53/f11a 0 2024-04-26T04:53:03.763 INFO:tasks.workunit.client.1.smithi078.stdout:8/785: mknod d0/d19/d52/d95/cfa 0 2024-04-26T04:53:03.764 INFO:tasks.workunit.client.1.smithi078.stdout:8/786: fsync d0/d19/d52/d35/d44/f26 0 2024-04-26T04:53:03.764 INFO:tasks.workunit.client.1.smithi078.stdout:0/810: link d2/d1e/d5a/da3/f102 d2/d9/d60/d6a/daa/f10b 0 2024-04-26T04:53:03.765 INFO:tasks.workunit.client.1.smithi078.stdout:8/787: write d0/d19/d25/d36/d58/feb [124747,72956] 0 2024-04-26T04:53:03.766 INFO:tasks.workunit.client.1.smithi078.stdout:0/811: symlink d2/d9/d18/db7/d92/l10c 0 2024-04-26T04:53:03.770 INFO:tasks.workunit.client.0.smithi077.stdout:1/405: sync 2024-04-26T04:53:03.773 INFO:tasks.workunit.client.0.smithi077.stdout:1/406: read d9/db/dd/d34/d4a/f59 [379225,12321] 0 2024-04-26T04:53:03.774 INFO:tasks.workunit.client.0.smithi077.stdout:0/324: chown d0/d19/d1e/d59/d5a/c71 4910433 1 2024-04-26T04:53:03.776 INFO:tasks.workunit.client.0.smithi077.stdout:0/325: symlink d0/d19/d1e/d59/l74 0 2024-04-26T04:53:03.777 INFO:tasks.workunit.client.0.smithi077.stdout:0/326: fsync d0/d19/d1e/f34 0 2024-04-26T04:53:03.793 INFO:tasks.workunit.client.0.smithi077.stdout:1/407: dwrite d9/f73 [0,4194304] 0 2024-04-26T04:53:03.794 INFO:tasks.workunit.client.0.smithi077.stdout:0/327: dwrite d0/d47/ff [0,4194304] 0 2024-04-26T04:53:03.795 INFO:tasks.workunit.client.0.smithi077.stdout:1/408: chown d9/db/d2b/d5c/l67 12631 1 2024-04-26T04:53:03.797 INFO:tasks.workunit.client.0.smithi077.stdout:0/328: fsync d0/f4 0 2024-04-26T04:53:03.800 INFO:tasks.workunit.client.0.smithi077.stdout:1/409: creat d9/db/dd/d34/d63/f76 x:0 0 0 2024-04-26T04:53:03.816 INFO:tasks.workunit.client.0.smithi077.stdout:0/329: write d0/d2c/f43 [1107749,91063] 0 2024-04-26T04:53:03.818 INFO:tasks.workunit.client.0.smithi077.stdout:0/330: mknod d0/d28/c75 0 2024-04-26T04:53:03.818 INFO:tasks.workunit.client.0.smithi077.stdout:0/331: fdatasync d0/d2c/d4e/f66 0 2024-04-26T04:53:03.820 INFO:tasks.workunit.client.0.smithi077.stdout:0/332: dread d0/d2c/f2e [4194304,4194304] 0 2024-04-26T04:53:03.821 INFO:tasks.workunit.client.0.smithi077.stdout:1/410: dread d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:53:03.823 INFO:tasks.workunit.client.0.smithi077.stdout:0/333: unlink d0/d19/d1e/d2f/f41 0 2024-04-26T04:53:03.823 INFO:tasks.workunit.client.0.smithi077.stdout:1/411: creat d9/db/dd/d34/d63/f77 x:0 0 0 2024-04-26T04:53:03.824 INFO:tasks.workunit.client.0.smithi077.stdout:3/313: sync 2024-04-26T04:53:03.824 INFO:tasks.workunit.client.0.smithi077.stdout:6/463: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:7/444: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:9/379: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:8/427: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:4/395: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:5/418: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.0.smithi077.stdout:2/411: sync 2024-04-26T04:53:03.825 INFO:tasks.workunit.client.1.smithi078.stdout:2/808: truncate d8/d11/f54 2339371 0 2024-04-26T04:53:03.826 INFO:tasks.workunit.client.1.smithi078.stdout:8/788: symlink d0/d19/d25/d32/d8a/d92/da4/lfb 0 2024-04-26T04:53:03.826 INFO:tasks.workunit.client.1.smithi078.stdout:0/812: rmdir d2/d9/d18/d7d/da9 39 2024-04-26T04:53:03.826 INFO:tasks.workunit.client.1.smithi078.stdout:1/854: dread d7/d1e/d9e/db/d47/f90 [0,4194304] 0 2024-04-26T04:53:03.826 INFO:tasks.workunit.client.0.smithi077.stdout:0/334: creat d0/d19/d1e/d2f/d52/f76 x:0 0 0 2024-04-26T04:53:03.827 INFO:tasks.workunit.client.0.smithi077.stdout:1/412: truncate d9/f42 754395 0 2024-04-26T04:53:03.828 INFO:tasks.workunit.client.0.smithi077.stdout:6/464: chown d8/d52/d53/f65 1199925 1 2024-04-26T04:53:03.828 INFO:tasks.workunit.client.0.smithi077.stdout:9/380: read d9/dd/d3d/f5a [1967343,13617] 0 2024-04-26T04:53:03.828 INFO:tasks.workunit.client.0.smithi077.stdout:5/419: chown d1/d12/d21/l29 14240039 1 2024-04-26T04:53:03.828 INFO:tasks.workunit.client.1.smithi078.stdout:5/755: dwrite d7/d18/f34 [4194304,4194304] 0 2024-04-26T04:53:03.829 INFO:tasks.workunit.client.0.smithi077.stdout:7/445: unlink da/d13/f98 0 2024-04-26T04:53:03.830 INFO:tasks.workunit.client.0.smithi077.stdout:9/381: write d9/d45/f7e [147987,52506] 0 2024-04-26T04:53:03.830 INFO:tasks.workunit.client.0.smithi077.stdout:2/412: truncate d9/dc/d1d/d34/f50 165604 0 2024-04-26T04:53:03.830 INFO:tasks.workunit.client.0.smithi077.stdout:4/396: unlink d4/d17/c7d 0 2024-04-26T04:53:03.831 INFO:tasks.workunit.client.0.smithi077.stdout:6/465: truncate d8/d55/f9b 222318 0 2024-04-26T04:53:03.832 INFO:tasks.workunit.client.0.smithi077.stdout:3/314: read - d0/d7/d1f/f43 zero size 2024-04-26T04:53:03.832 INFO:tasks.workunit.client.0.smithi077.stdout:4/397: write d4/de/f2b [815697,55534] 0 2024-04-26T04:53:03.833 INFO:tasks.workunit.client.0.smithi077.stdout:0/335: unlink d0/d2c/f43 0 2024-04-26T04:53:03.833 INFO:tasks.workunit.client.0.smithi077.stdout:6/466: write d8/f37 [4558126,60218] 0 2024-04-26T04:53:03.835 INFO:tasks.workunit.client.0.smithi077.stdout:1/413: fsync d9/db/d2b/d3f/f50 0 2024-04-26T04:53:03.837 INFO:tasks.workunit.client.0.smithi077.stdout:6/467: write d8/d2a/f2e [204250,69961] 0 2024-04-26T04:53:03.839 INFO:tasks.workunit.client.0.smithi077.stdout:2/413: mknod d9/dc/c8a 0 2024-04-26T04:53:03.840 INFO:tasks.workunit.client.0.smithi077.stdout:7/446: dread - da/d13/d22/d5d/f66 zero size 2024-04-26T04:53:03.841 INFO:tasks.workunit.client.0.smithi077.stdout:3/315: dwrite d0/d9/d32/f36 [4194304,4194304] 0 2024-04-26T04:53:03.842 INFO:tasks.workunit.client.0.smithi077.stdout:3/316: dread - d0/d7/d15/f3f zero size 2024-04-26T04:53:03.843 INFO:tasks.workunit.client.0.smithi077.stdout:9/382: creat d9/dd/d3d/f88 x:0 0 0 2024-04-26T04:53:03.849 INFO:tasks.workunit.client.0.smithi077.stdout:0/336: mknod d0/d28/d40/c77 0 2024-04-26T04:53:03.850 INFO:tasks.workunit.client.0.smithi077.stdout:0/337: chown d0/d19/d1e/f20 5938818 1 2024-04-26T04:53:03.851 INFO:tasks.workunit.client.1.smithi078.stdout:3/812: sync 2024-04-26T04:53:03.853 INFO:tasks.workunit.client.0.smithi077.stdout:4/398: creat d4/d17/d18/d43/d53/f8a x:0 0 0 2024-04-26T04:53:03.854 INFO:tasks.workunit.client.0.smithi077.stdout:1/414: symlink d9/d53/d6e/l78 0 2024-04-26T04:53:03.855 INFO:tasks.workunit.client.0.smithi077.stdout:1/415: chown d9/db/d2b/f36 1392 1 2024-04-26T04:53:03.858 INFO:tasks.workunit.client.0.smithi077.stdout:5/420: rename d1/d8/l3f to d1/d12/d28/d38/l9b 0 2024-04-26T04:53:03.858 INFO:tasks.workunit.client.1.smithi078.stdout:3/813: readlink d16/d26/d2b/lae 0 2024-04-26T04:53:03.858 INFO:tasks.workunit.client.1.smithi078.stdout:5/756: write d7/d21/f4e [1892999,90112] 0 2024-04-26T04:53:03.860 INFO:tasks.workunit.client.1.smithi078.stdout:1/855: mkdir d7/d1e/d9e/dd/d20/dca/d115 0 2024-04-26T04:53:03.862 INFO:tasks.workunit.client.1.smithi078.stdout:4/797: dwrite d0/de/d22/d27/d7b/f82 [0,4194304] 0 2024-04-26T04:53:03.863 INFO:tasks.workunit.client.1.smithi078.stdout:8/789: dwrite d0/d19/d25/d32/d8a/fc4 [0,4194304] 0 2024-04-26T04:53:03.864 INFO:tasks.workunit.client.1.smithi078.stdout:2/809: rename d8/d11/d14/d5b/l63 to d8/d11/d14/d69/d8f/d34/dd3/dd9/l11b 0 2024-04-26T04:53:03.866 INFO:tasks.workunit.client.0.smithi077.stdout:7/447: creat da/d13/d22/d5d/f9a x:0 0 0 2024-04-26T04:53:03.867 INFO:tasks.workunit.client.1.smithi078.stdout:0/813: truncate d2/d9/d18/db7/d92/fad 627260 0 2024-04-26T04:53:03.871 INFO:tasks.workunit.client.0.smithi077.stdout:9/383: chown d9/dd/le 8504596 1 2024-04-26T04:53:03.872 INFO:tasks.workunit.client.0.smithi077.stdout:9/384: write d9/d43/f61 [748024,82018] 0 2024-04-26T04:53:03.873 INFO:tasks.workunit.client.1.smithi078.stdout:8/790: dwrite d0/d19/d25/d32/d8a/d92/da4/fed [0,4194304] 0 2024-04-26T04:53:03.874 INFO:tasks.workunit.client.0.smithi077.stdout:8/428: truncate d4/df/d18/f4c 795470 0 2024-04-26T04:53:03.880 INFO:tasks.workunit.client.1.smithi078.stdout:8/791: dread d0/d19/d52/d35/d44/d6b/f9b [0,4194304] 0 2024-04-26T04:53:03.882 INFO:tasks.workunit.client.1.smithi078.stdout:0/814: creat d2/d9/d78/deb/f10d x:0 0 0 2024-04-26T04:53:03.885 INFO:tasks.workunit.client.0.smithi077.stdout:7/448: rmdir da/d13/d22 39 2024-04-26T04:53:03.887 INFO:tasks.workunit.client.0.smithi077.stdout:3/317: mknod d0/d9/d32/c62 0 2024-04-26T04:53:03.891 INFO:tasks.workunit.client.0.smithi077.stdout:0/338: mkdir d0/d19/d1e/d78 0 2024-04-26T04:53:03.892 INFO:tasks.workunit.client.1.smithi078.stdout:1/856: truncate d7/d1e/d9e/dd/d20/f50 2496950 0 2024-04-26T04:53:03.893 INFO:tasks.workunit.client.0.smithi077.stdout:8/429: rmdir d4/df/d22/d2d 39 2024-04-26T04:53:03.893 INFO:tasks.workunit.client.1.smithi078.stdout:2/810: getdents d8/d11/d18/d5d/d61/df5/d102 0 2024-04-26T04:53:03.894 INFO:tasks.workunit.client.0.smithi077.stdout:4/399: mkdir d4/de/d15/d1f/d35/d5c/d8b 0 2024-04-26T04:53:03.897 INFO:tasks.workunit.client.1.smithi078.stdout:8/792: creat d0/d19/d25/d32/d8a/d92/da4/da7/ffc x:0 0 0 2024-04-26T04:53:03.898 INFO:tasks.workunit.client.1.smithi078.stdout:5/757: dread d7/d48/d57/d65/d46/f68 [0,4194304] 0 2024-04-26T04:53:03.898 INFO:tasks.workunit.client.0.smithi077.stdout:2/414: fdatasync d9/dc/d1d/d34/d39/d63/d79/d2b/f46 0 2024-04-26T04:53:03.899 INFO:tasks.workunit.client.1.smithi078.stdout:3/814: creat d16/d26/d2e/d5f/f11c x:0 0 0 2024-04-26T04:53:03.900 INFO:tasks.workunit.client.0.smithi077.stdout:2/415: readlink d9/dc/d1d/d34/d39/d6b/l7c 0 2024-04-26T04:53:03.902 INFO:tasks.workunit.client.0.smithi077.stdout:6/468: write d8/f27 [1066059,10304] 0 2024-04-26T04:53:03.903 INFO:tasks.workunit.client.0.smithi077.stdout:4/400: dwrite d4/d17/d18/d43/d53/d56/f2d [4194304,4194304] 0 2024-04-26T04:53:03.903 INFO:tasks.workunit.client.0.smithi077.stdout:3/318: creat d0/d7/d1f/d4d/f63 x:0 0 0 2024-04-26T04:53:03.906 INFO:tasks.workunit.client.1.smithi078.stdout:8/793: dwrite d0/d19/d52/d35/d44/f4d [4194304,4194304] 0 2024-04-26T04:53:03.907 INFO:tasks.workunit.client.0.smithi077.stdout:9/385: write d9/f26 [2204273,1562] 0 2024-04-26T04:53:03.907 INFO:tasks.workunit.client.1.smithi078.stdout:2/811: creat d8/db7/f11c x:0 0 0 2024-04-26T04:53:03.907 INFO:tasks.workunit.client.0.smithi077.stdout:0/339: stat d0/l7 0 2024-04-26T04:53:03.907 INFO:tasks.workunit.client.0.smithi077.stdout:9/386: stat d9/d7d 0 2024-04-26T04:53:03.908 INFO:tasks.workunit.client.0.smithi077.stdout:9/387: write d9/f26 [4886186,104755] 0 2024-04-26T04:53:03.909 INFO:tasks.workunit.client.1.smithi078.stdout:4/798: link d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/fde d0/d1d/db4/f11c 0 2024-04-26T04:53:03.909 INFO:tasks.workunit.client.1.smithi078.stdout:5/758: dwrite d7/d18/d32/d37/d79/fad [4194304,4194304] 0 2024-04-26T04:53:03.910 INFO:tasks.workunit.client.0.smithi077.stdout:9/388: stat d9/d45/l4d 0 2024-04-26T04:53:03.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/430: rmdir d4/df/d18 39 2024-04-26T04:53:03.911 INFO:tasks.workunit.client.1.smithi078.stdout:8/794: fsync d0/d19/d25/d32/d8a/d92/fcf 0 2024-04-26T04:53:03.911 INFO:tasks.workunit.client.1.smithi078.stdout:6/862: sync 2024-04-26T04:53:03.918 INFO:tasks.workunit.client.0.smithi077.stdout:0/340: dwrite d0/d19/d1e/f20 [0,4194304] 0 2024-04-26T04:53:03.920 INFO:tasks.workunit.client.0.smithi077.stdout:9/389: dwrite d9/d45/f7e [0,4194304] 0 2024-04-26T04:53:03.924 INFO:tasks.workunit.client.0.smithi077.stdout:9/390: fsync d9/f2d 0 2024-04-26T04:53:03.924 INFO:tasks.workunit.client.0.smithi077.stdout:9/391: fsync d9/d3a/d79/f68 0 2024-04-26T04:53:03.924 INFO:tasks.workunit.client.1.smithi078.stdout:8/795: dwrite d0/d19/d52/d35/d44/d6b/d6c/fe9 [0,4194304] 0 2024-04-26T04:53:03.926 INFO:tasks.workunit.client.1.smithi078.stdout:6/863: dread d2/de/f67 [0,4194304] 0 2024-04-26T04:53:03.928 INFO:tasks.workunit.client.0.smithi077.stdout:3/319: unlink d0/c5e 0 2024-04-26T04:53:03.931 INFO:tasks.workunit.client.0.smithi077.stdout:2/416: truncate d9/dc/d1d/d34/d39/d63/d79/f3f 703724 0 2024-04-26T04:53:03.931 INFO:tasks.workunit.client.1.smithi078.stdout:8/796: dread d0/d19/d25/d36/d58/feb [0,4194304] 0 2024-04-26T04:53:03.936 INFO:tasks.workunit.client.0.smithi077.stdout:8/431: readlink d4/lc 0 2024-04-26T04:53:03.937 INFO:tasks.workunit.client.1.smithi078.stdout:5/759: creat d7/da/d1e/ff8 x:0 0 0 2024-04-26T04:53:03.938 INFO:tasks.workunit.client.1.smithi078.stdout:1/857: dread d7/d1e/d9e/db/d37/d58/f5e [0,4194304] 0 2024-04-26T04:53:03.940 INFO:tasks.workunit.client.0.smithi077.stdout:1/416: getdents d9/db/d2b 0 2024-04-26T04:53:03.944 INFO:tasks.workunit.client.1.smithi078.stdout:2/812: mknod d8/d11/d18/d5d/d8b/c11d 0 2024-04-26T04:53:03.945 INFO:tasks.workunit.client.1.smithi078.stdout:1/858: dread d7/d1e/d9e/db/d37/fa9 [0,4194304] 0 2024-04-26T04:53:03.945 INFO:tasks.workunit.client.0.smithi077.stdout:9/392: creat d9/d3a/f89 x:0 0 0 2024-04-26T04:53:03.946 INFO:tasks.workunit.client.0.smithi077.stdout:9/393: truncate d9/f46 265577 0 2024-04-26T04:53:03.947 INFO:tasks.workunit.client.0.smithi077.stdout:9/394: chown d9/d43/c70 16064 1 2024-04-26T04:53:03.948 INFO:tasks.workunit.client.1.smithi078.stdout:0/815: getdents d2/d9/d31 0 2024-04-26T04:53:03.948 INFO:tasks.workunit.client.1.smithi078.stdout:2/813: truncate d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/dec/f105 909516 0 2024-04-26T04:53:03.949 INFO:tasks.workunit.client.1.smithi078.stdout:2/814: chown d8/ff2 1673626566 1 2024-04-26T04:53:03.949 INFO:tasks.workunit.client.1.smithi078.stdout:7/772: sync 2024-04-26T04:53:03.949 INFO:tasks.workunit.client.1.smithi078.stdout:9/737: sync 2024-04-26T04:53:03.949 INFO:tasks.workunit.client.1.smithi078.stdout:8/797: symlink d0/d19/d25/d32/d8a/d92/da4/da7/lfd 0 2024-04-26T04:53:03.949 INFO:tasks.workunit.client.1.smithi078.stdout:0/816: write d2/d1e/f67 [3979831,128624] 0 2024-04-26T04:53:03.951 INFO:tasks.workunit.client.1.smithi078.stdout:4/799: dread d0/d1d/de1/f19 [0,4194304] 0 2024-04-26T04:53:03.953 INFO:tasks.workunit.client.0.smithi077.stdout:7/449: truncate da/d15/f2f 1065756 0 2024-04-26T04:53:03.954 INFO:tasks.workunit.client.1.smithi078.stdout:4/800: dread - d0/d1d/de1/d20/d31/d47/db2/ff9 zero size 2024-04-26T04:53:03.956 INFO:tasks.workunit.client.1.smithi078.stdout:5/760: mkdir d7/d18/d1c/d5a/df9 0 2024-04-26T04:53:03.957 INFO:tasks.workunit.client.1.smithi078.stdout:3/815: dwrite d16/d26/d2e/d5f/d49/f8f [0,4194304] 0 2024-04-26T04:53:03.961 INFO:tasks.workunit.client.0.smithi077.stdout:7/450: dread da/d13/f4b [0,4194304] 0 2024-04-26T04:53:03.963 INFO:tasks.workunit.client.0.smithi077.stdout:3/320: mknod d0/d7/d1f/c64 0 2024-04-26T04:53:03.965 INFO:tasks.workunit.client.0.smithi077.stdout:2/417: chown d9/dc/d1d/d34/d39/d63/d79/f3f 343 1 2024-04-26T04:53:03.966 INFO:tasks.workunit.client.0.smithi077.stdout:2/418: fsync d9/dc/d1d/d34/d39/d63/d79/f22 0 2024-04-26T04:53:03.967 INFO:tasks.workunit.client.1.smithi078.stdout:1/859: symlink d7/d1e/d9e/db/d49/dba/d87/l116 0 2024-04-26T04:53:03.968 INFO:tasks.workunit.client.0.smithi077.stdout:8/432: read - d4/df/d22/d2d/d2b/d33/f39 zero size 2024-04-26T04:53:03.969 INFO:tasks.workunit.client.0.smithi077.stdout:8/433: chown d4/df/d22/d2d/d2b/f5d 2559 1 2024-04-26T04:53:03.971 INFO:tasks.workunit.client.1.smithi078.stdout:3/816: dread d16/d26/d2b/dd6/fe3 [0,4194304] 0 2024-04-26T04:53:03.973 INFO:tasks.workunit.client.1.smithi078.stdout:9/738: mknod dc/df/d27/d81/da8/cde 0 2024-04-26T04:53:03.973 INFO:tasks.workunit.client.1.smithi078.stdout:3/817: fsync d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/fa2 0 2024-04-26T04:53:03.977 INFO:tasks.workunit.client.0.smithi077.stdout:0/341: write d0/f1 [4107861,27413] 0 2024-04-26T04:53:03.981 INFO:tasks.workunit.client.0.smithi077.stdout:0/342: dread - d0/d19/d1e/d2f/d52/f62 zero size 2024-04-26T04:53:03.981 INFO:tasks.workunit.client.0.smithi077.stdout:1/417: dwrite d9/db/d2b/f61 [0,4194304] 0 2024-04-26T04:53:03.983 INFO:tasks.workunit.client.1.smithi078.stdout:4/801: unlink d0/d1d/de1/d20/l8c 0 2024-04-26T04:53:03.985 INFO:tasks.workunit.client.1.smithi078.stdout:5/761: creat d7/da/d1e/d91/ffa x:0 0 0 2024-04-26T04:53:03.985 INFO:tasks.workunit.client.1.smithi078.stdout:8/798: write d0/d7a/dbe/ff6 [313099,21324] 0 2024-04-26T04:53:03.986 INFO:tasks.workunit.client.0.smithi077.stdout:0/343: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:53:03.987 INFO:tasks.workunit.client.1.smithi078.stdout:8/799: stat d0/d19/d52/d5c/fb3 0 2024-04-26T04:53:03.988 INFO:tasks.workunit.client.0.smithi077.stdout:0/344: dread d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:03.991 INFO:tasks.workunit.client.1.smithi078.stdout:6/864: rmdir d2/d1a/d38/dd5/ddf/df5 0 2024-04-26T04:53:03.994 INFO:tasks.workunit.client.1.smithi078.stdout:1/860: creat d7/d1e/d9e/db/d49/df8/f117 x:0 0 0 2024-04-26T04:53:03.996 INFO:tasks.workunit.client.1.smithi078.stdout:2/815: unlink d8/d11/d14/l56 0 2024-04-26T04:53:03.998 INFO:tasks.workunit.client.1.smithi078.stdout:6/865: dwrite d2/de/d27/da9/d3a/d42/f9c [4194304,4194304] 0 2024-04-26T04:53:04.001 INFO:tasks.workunit.client.1.smithi078.stdout:9/739: symlink dc/df/d4b/d22/d2a/d57/d93/ldf 0 2024-04-26T04:53:04.005 INFO:tasks.workunit.client.1.smithi078.stdout:5/762: creat d7/d21/d2f/db6/ffb x:0 0 0 2024-04-26T04:53:04.010 INFO:tasks.workunit.client.1.smithi078.stdout:4/802: dwrite d0/de/d22/d27/db5/fd4 [0,4194304] 0 2024-04-26T04:53:04.011 INFO:tasks.workunit.client.1.smithi078.stdout:4/803: chown d0/de/d22/d27/d56/f6a 283141784 1 2024-04-26T04:53:04.012 INFO:tasks.workunit.client.0.smithi077.stdout:4/401: getdents d4/de/d15/d1f 0 2024-04-26T04:53:04.018 INFO:tasks.workunit.client.1.smithi078.stdout:6/866: rename d2/de/c2d to d2/de/d27/da9/d2a/dbf/df6/c117 0 2024-04-26T04:53:04.020 INFO:tasks.workunit.client.1.smithi078.stdout:0/817: creat d2/d9/d18/d7d/f10e x:0 0 0 2024-04-26T04:53:04.021 INFO:tasks.workunit.client.1.smithi078.stdout:7/773: link d9/da/d11/fa9 d9/da/d33/fff 0 2024-04-26T04:53:04.023 INFO:tasks.workunit.client.1.smithi078.stdout:8/800: creat d0/d19/d52/d5c/df0/ffe x:0 0 0 2024-04-26T04:53:04.024 INFO:tasks.workunit.client.1.smithi078.stdout:8/801: fsync d0/fe4 0 2024-04-26T04:53:04.026 INFO:tasks.workunit.client.1.smithi078.stdout:3/818: dwrite d16/d26/d2e/d5f/d49/d77/f7b [0,4194304] 0 2024-04-26T04:53:04.027 INFO:tasks.workunit.client.1.smithi078.stdout:2/816: mknod d8/d11/d14/c11e 0 2024-04-26T04:53:04.028 INFO:tasks.workunit.client.0.smithi077.stdout:0/345: mknod d0/d2c/c79 0 2024-04-26T04:53:04.032 INFO:tasks.workunit.client.0.smithi077.stdout:3/321: write d0/d7/f34 [3553165,90687] 0 2024-04-26T04:53:04.032 INFO:tasks.workunit.client.0.smithi077.stdout:7/451: stat da/d13/d22/d5d/c91 0 2024-04-26T04:53:04.036 INFO:tasks.workunit.client.1.smithi078.stdout:0/818: creat d2/d9/d31/f10f x:0 0 0 2024-04-26T04:53:04.038 INFO:tasks.workunit.client.0.smithi077.stdout:4/402: rename d4/d17/f3b to d4/de/d15/d1f/d35/d5c/f8c 0 2024-04-26T04:53:04.039 INFO:tasks.workunit.client.0.smithi077.stdout:4/403: chown d4/d17/d18/d43/d63/l67 131 1 2024-04-26T04:53:04.041 INFO:tasks.workunit.client.0.smithi077.stdout:2/419: mknod d9/d15/d56/c8b 0 2024-04-26T04:53:04.041 INFO:tasks.workunit.client.1.smithi078.stdout:5/763: write d7/d48/d57/d65/f36 [3420200,117143] 0 2024-04-26T04:53:04.045 INFO:tasks.workunit.client.0.smithi077.stdout:1/418: mknod d9/db/d2b/d75/c79 0 2024-04-26T04:53:04.045 INFO:tasks.workunit.client.0.smithi077.stdout:1/419: chown d9/db/dd/d34/d4a 9845 1 2024-04-26T04:53:04.048 INFO:tasks.workunit.client.0.smithi077.stdout:9/395: truncate d9/dd/f5d 2918351 0 2024-04-26T04:53:04.049 INFO:tasks.workunit.client.0.smithi077.stdout:9/396: stat d9/dd/d1b/d2b/d32/c6f 0 2024-04-26T04:53:04.049 INFO:tasks.workunit.client.0.smithi077.stdout:0/346: mknod d0/d19/d1e/d59/d5a/c7a 0 2024-04-26T04:53:04.051 INFO:tasks.workunit.client.1.smithi078.stdout:3/819: symlink d16/d26/d2b/d31/d78/dd1/l11d 0 2024-04-26T04:53:04.051 INFO:tasks.workunit.client.1.smithi078.stdout:6/867: dread d2/de/d27/d58/d5b/d101/f111 [0,4194304] 0 2024-04-26T04:53:04.053 INFO:tasks.workunit.client.1.smithi078.stdout:1/861: link d7/d1e/d9e/db/ldf d7/d1e/d9e/db/d37/d58/d5c/l118 0 2024-04-26T04:53:04.053 INFO:tasks.workunit.client.1.smithi078.stdout:6/868: stat d2/de/d27/da9/da/d35/f3e 0 2024-04-26T04:53:04.055 INFO:tasks.workunit.client.1.smithi078.stdout:9/740: rename dc/df/d4b/d22/l31 to dc/df/d4b/d3e/d80/le0 0 2024-04-26T04:53:04.056 INFO:tasks.workunit.client.1.smithi078.stdout:8/802: dread d0/d19/d25/f2b [0,4194304] 0 2024-04-26T04:53:04.056 INFO:tasks.workunit.client.0.smithi077.stdout:4/404: rename d4/d17/d18/d43/d53/f8a to d4/d4e/f8d 0 2024-04-26T04:53:04.058 INFO:tasks.workunit.client.0.smithi077.stdout:8/434: dwrite d4/df/d22/d2d/d2b/f4d [0,4194304] 0 2024-04-26T04:53:04.059 INFO:tasks.workunit.client.0.smithi077.stdout:4/405: truncate d4/d17/d18/d43/d53/d64/f65 236773 0 2024-04-26T04:53:04.060 INFO:tasks.workunit.client.0.smithi077.stdout:4/406: write d4/de/f2b [1567617,120379] 0 2024-04-26T04:53:04.095 INFO:tasks.workunit.client.1.smithi078.stdout:4/804: sync 2024-04-26T04:53:04.123 INFO:tasks.workunit.client.1.smithi078.stdout:5/764: mkdir d7/d18/d1c/d5a/d96/dfc 0 2024-04-26T04:53:04.129 INFO:tasks.workunit.client.1.smithi078.stdout:3/820: unlink d16/d26/d2e/d5f/d49/d77/dba/f113 0 2024-04-26T04:53:04.132 INFO:tasks.workunit.client.1.smithi078.stdout:1/862: mkdir d7/d1e/d9e/db/d49/dba/d87/d119 0 2024-04-26T04:53:04.134 INFO:tasks.workunit.client.1.smithi078.stdout:1/863: write d7/d1e/d9e/db/d37/d58/fef [949708,14361] 0 2024-04-26T04:53:04.134 INFO:tasks.workunit.client.1.smithi078.stdout:6/869: fdatasync d2/de/d27/da9/d2a/dbf/d5f/f9f 0 2024-04-26T04:53:04.136 INFO:tasks.workunit.client.0.smithi077.stdout:3/322: link d0/d7/d15/f50 d0/d7/d15/f65 0 2024-04-26T04:53:04.142 INFO:tasks.workunit.client.1.smithi078.stdout:6/870: dwrite d2/de/d27/da9/d2a/dbf/fce [0,4194304] 0 2024-04-26T04:53:04.144 INFO:tasks.workunit.client.1.smithi078.stdout:9/741: dread dc/df/f2f [0,4194304] 0 2024-04-26T04:53:04.145 INFO:tasks.workunit.client.0.smithi077.stdout:2/420: rename d9/dc/d1d/d34/d39/d63/d79/d2b/f7b to d9/d15/d56/f8c 0 2024-04-26T04:53:04.145 INFO:tasks.workunit.client.1.smithi078.stdout:9/742: readlink dc/d55/l9c 0 2024-04-26T04:53:04.149 INFO:tasks.workunit.client.1.smithi078.stdout:7/774: link d9/f6a d9/da/d11/d13/d1a/d52/d5a/d28/f100 0 2024-04-26T04:53:04.152 INFO:tasks.workunit.client.1.smithi078.stdout:5/765: stat d7/d11/fb5 0 2024-04-26T04:53:04.154 INFO:tasks.workunit.client.0.smithi077.stdout:4/407: unlink d4/d17/d18/d43/d53/d64/f65 0 2024-04-26T04:53:04.156 INFO:tasks.workunit.client.1.smithi078.stdout:2/817: creat d8/d11/d18/f11f x:0 0 0 2024-04-26T04:53:04.158 INFO:tasks.workunit.client.1.smithi078.stdout:5/766: dwrite d7/d18/d32/f60 [0,4194304] 0 2024-04-26T04:53:04.158 INFO:tasks.workunit.client.0.smithi077.stdout:9/397: mknod d9/d7d/c8a 0 2024-04-26T04:53:04.161 INFO:tasks.workunit.client.0.smithi077.stdout:0/347: symlink d0/d19/d69/l7b 0 2024-04-26T04:53:04.166 INFO:tasks.workunit.client.1.smithi078.stdout:1/864: creat d7/ddd/f11a x:0 0 0 2024-04-26T04:53:04.168 INFO:tasks.workunit.client.0.smithi077.stdout:3/323: unlink d0/d7/d1f/l59 0 2024-04-26T04:53:04.172 INFO:tasks.workunit.client.1.smithi078.stdout:0/819: mkdir d2/d9/d18/d7d/da9/dab/dfd/d110 0 2024-04-26T04:53:04.173 INFO:tasks.workunit.client.0.smithi077.stdout:8/435: symlink d4/l95 0 2024-04-26T04:53:04.181 INFO:tasks.workunit.client.0.smithi077.stdout:7/452: getdents da/d13/d22/d7d 0 2024-04-26T04:53:04.184 INFO:tasks.workunit.client.0.smithi077.stdout:3/324: symlink d0/d9/d32/d29/l66 0 2024-04-26T04:53:04.187 INFO:tasks.workunit.client.0.smithi077.stdout:2/421: mkdir d9/dc/d1d/d34/d8d 0 2024-04-26T04:53:04.187 INFO:tasks.workunit.client.1.smithi078.stdout:5/767: dread - d7/d18/d32/d37/d79/fa1 zero size 2024-04-26T04:53:04.190 INFO:tasks.workunit.client.1.smithi078.stdout:1/865: creat d7/d1e/d9e/db/d49/dba/d87/f11b x:0 0 0 2024-04-26T04:53:04.192 INFO:tasks.workunit.client.1.smithi078.stdout:0/820: mknod d2/d9/d60/d7f/c111 0 2024-04-26T04:53:04.194 INFO:tasks.workunit.client.0.smithi077.stdout:9/398: rmdir d9/dd/d1b/d2b/d32/d36 39 2024-04-26T04:53:04.195 INFO:tasks.workunit.client.0.smithi077.stdout:9/399: readlink d9/dd/le 0 2024-04-26T04:53:04.195 INFO:tasks.workunit.client.0.smithi077.stdout:7/453: truncate da/d13/f19 1265296 0 2024-04-26T04:53:04.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/743: mkdir dc/dd9/de1 0 2024-04-26T04:53:04.199 INFO:tasks.workunit.client.0.smithi077.stdout:0/348: dread d0/fe [0,4194304] 0 2024-04-26T04:53:04.200 INFO:tasks.workunit.client.0.smithi077.stdout:8/436: symlink d4/df/d22/d80/d86/l96 0 2024-04-26T04:53:04.202 INFO:tasks.workunit.client.1.smithi078.stdout:5/768: write d7/d48/d57/d65/d46/ff7 [2532064,68598] 0 2024-04-26T04:53:04.204 INFO:tasks.workunit.client.1.smithi078.stdout:1/866: mknod d7/d1e/d9e/dfc/c11c 0 2024-04-26T04:53:04.205 INFO:tasks.workunit.client.1.smithi078.stdout:1/867: write d7/d1e/d9e/d4c/d3d/f95 [199436,30798] 0 2024-04-26T04:53:04.206 INFO:tasks.workunit.client.1.smithi078.stdout:1/868: chown d7/d1e/d9e/c4b 8 1 2024-04-26T04:53:04.208 INFO:tasks.workunit.client.1.smithi078.stdout:0/821: write d2/d9/d18/fd [5301648,60103] 0 2024-04-26T04:53:04.208 INFO:tasks.workunit.client.0.smithi077.stdout:5/421: sync 2024-04-26T04:53:04.208 INFO:tasks.workunit.client.0.smithi077.stdout:6/469: sync 2024-04-26T04:53:04.209 INFO:tasks.workunit.client.0.smithi077.stdout:5/422: chown d1/d12/d21/f64 1218410 1 2024-04-26T04:53:04.210 INFO:tasks.workunit.client.0.smithi077.stdout:5/423: chown d1/d12/d21/d84/c86 1919229307 1 2024-04-26T04:53:04.212 INFO:tasks.workunit.client.1.smithi078.stdout:6/871: creat d2/d1a/f118 x:0 0 0 2024-04-26T04:53:04.213 INFO:tasks.workunit.client.0.smithi077.stdout:6/470: dread d8/d1c/f3e [0,4194304] 0 2024-04-26T04:53:04.214 INFO:tasks.workunit.client.1.smithi078.stdout:9/744: read - dc/df/d17/f88 zero size 2024-04-26T04:53:04.217 INFO:tasks.workunit.client.0.smithi077.stdout:0/349: mknod d0/d19/d1e/d2f/c7c 0 2024-04-26T04:53:04.219 INFO:tasks.workunit.client.0.smithi077.stdout:8/437: symlink d4/df/d22/d2d/l97 0 2024-04-26T04:53:04.222 INFO:tasks.workunit.client.0.smithi077.stdout:0/350: dread d0/d47/d64/f5c [0,4194304] 0 2024-04-26T04:53:04.223 INFO:tasks.workunit.client.0.smithi077.stdout:7/454: link da/d6b/f93 da/d15/d42/d8c/f9b 0 2024-04-26T04:53:04.227 INFO:tasks.workunit.client.1.smithi078.stdout:6/872: creat d2/de/d27/de8/f119 x:0 0 0 2024-04-26T04:53:04.228 INFO:tasks.workunit.client.1.smithi078.stdout:6/873: dread - d2/de/d27/da9/da/dab/fb3 zero size 2024-04-26T04:53:04.228 INFO:tasks.workunit.client.1.smithi078.stdout:9/745: mkdir dc/d55/de2 0 2024-04-26T04:53:04.230 INFO:tasks.workunit.client.0.smithi077.stdout:5/424: creat d1/d12/d21/f9c x:0 0 0 2024-04-26T04:53:04.230 INFO:tasks.workunit.client.0.smithi077.stdout:5/425: readlink d1/d30/l37 0 2024-04-26T04:53:04.234 INFO:tasks.workunit.client.0.smithi077.stdout:2/422: rmdir d9/dc/d1d/d34/d8d 0 2024-04-26T04:53:04.235 INFO:tasks.workunit.client.0.smithi077.stdout:2/423: chown d9/dc/d1d/d34/d39/d63/d79/d2b/f46 66552 1 2024-04-26T04:53:04.236 INFO:tasks.workunit.client.1.smithi078.stdout:6/874: creat d2/d1a/d5c/f11a x:0 0 0 2024-04-26T04:53:04.236 INFO:tasks.workunit.client.1.smithi078.stdout:6/875: fsync d2/fa0 0 2024-04-26T04:53:04.237 INFO:tasks.workunit.client.1.smithi078.stdout:7/775: mkdir d9/da/d33/d101 0 2024-04-26T04:53:04.240 INFO:tasks.workunit.client.0.smithi077.stdout:8/438: write d4/d17/f6a [3211309,70780] 0 2024-04-26T04:53:04.242 INFO:tasks.workunit.client.1.smithi078.stdout:6/876: dwrite d2/de/d27/da9/f109 [0,4194304] 0 2024-04-26T04:53:04.242 INFO:tasks.workunit.client.0.smithi077.stdout:9/400: creat d9/f8b x:0 0 0 2024-04-26T04:53:04.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:03 smithi077 ceph-mon[26311]: pgmap v21: 65 pgs: 65 active+clean; 2.2 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 74 MiB/s rd, 267 MiB/s wr, 519 op/s 2024-04-26T04:53:04.244 INFO:tasks.workunit.client.0.smithi077.stdout:0/351: dread - d0/d19/d1e/d2f/f3b zero size 2024-04-26T04:53:04.244 INFO:tasks.workunit.client.1.smithi078.stdout:3/821: creat d16/d26/d2e/d5f/d49/dad/db0/db7/f11e x:0 0 0 2024-04-26T04:53:04.245 INFO:tasks.workunit.client.0.smithi077.stdout:0/352: stat d0/d2c/f2e 0 2024-04-26T04:53:04.246 INFO:tasks.workunit.client.1.smithi078.stdout:3/822: chown d16/d26/d2e/d5f/d49/d77/f8b 1231203 1 2024-04-26T04:53:04.247 INFO:tasks.workunit.client.1.smithi078.stdout:3/823: readlink d16/d26/d2e/d5f/dbb/l115 0 2024-04-26T04:53:04.249 INFO:tasks.workunit.client.0.smithi077.stdout:5/426: mknod d1/d12/d21/d84/c9d 0 2024-04-26T04:53:04.249 INFO:tasks.workunit.client.0.smithi077.stdout:5/427: dread - d1/d12/d21/f9c zero size 2024-04-26T04:53:04.251 INFO:tasks.workunit.client.0.smithi077.stdout:7/455: mkdir da/d6b/d63/d9c 0 2024-04-26T04:53:04.252 INFO:tasks.workunit.client.0.smithi077.stdout:7/456: dread - da/d13/d22/d5d/f9a zero size 2024-04-26T04:53:04.256 INFO:tasks.workunit.client.0.smithi077.stdout:5/428: creat d1/d42/d62/f9e x:0 0 0 2024-04-26T04:53:04.256 INFO:tasks.workunit.client.0.smithi077.stdout:5/429: readlink d1/d12/d21/l29 0 2024-04-26T04:53:04.258 INFO:tasks.workunit.client.1.smithi078.stdout:6/877: dwrite d2/de/d27/da9/d11/fc2 [0,4194304] 0 2024-04-26T04:53:04.260 INFO:tasks.workunit.client.1.smithi078.stdout:6/878: write d2/de/d27/da9/d3a/d42/f5a [1369637,42708] 0 2024-04-26T04:53:04.261 INFO:tasks.workunit.client.1.smithi078.stdout:9/746: getdents dc/df/d4b/d22 0 2024-04-26T04:53:04.262 INFO:tasks.workunit.client.1.smithi078.stdout:9/747: dread - dc/df/d27/fb1 zero size 2024-04-26T04:53:04.262 INFO:tasks.workunit.client.1.smithi078.stdout:6/879: chown d2/de/d27/da9/d3a/d42/lc5 13 1 2024-04-26T04:53:04.264 INFO:tasks.workunit.client.0.smithi077.stdout:1/420: dwrite d9/d53/f6f [0,4194304] 0 2024-04-26T04:53:04.265 INFO:tasks.workunit.client.0.smithi077.stdout:1/421: stat d9/db/dd/f5b 0 2024-04-26T04:53:04.267 INFO:tasks.workunit.client.1.smithi078.stdout:9/748: dread dc/df/d4b/d22/d2d/d98/fb4 [0,4194304] 0 2024-04-26T04:53:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:03 smithi078 ceph-mon[33746]: pgmap v21: 65 pgs: 65 active+clean; 2.2 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 74 MiB/s rd, 267 MiB/s wr, 519 op/s 2024-04-26T04:53:04.268 INFO:tasks.workunit.client.0.smithi077.stdout:0/353: dread d0/d19/d1e/f34 [0,4194304] 0 2024-04-26T04:53:04.272 INFO:tasks.workunit.client.0.smithi077.stdout:9/401: creat d9/f8c x:0 0 0 2024-04-26T04:53:04.273 INFO:tasks.workunit.client.0.smithi077.stdout:9/402: read - d9/d3a/d79/f68 zero size 2024-04-26T04:53:04.274 INFO:tasks.workunit.client.1.smithi078.stdout:9/749: truncate dc/df/d27/d81/f87 1271423 0 2024-04-26T04:53:04.279 INFO:tasks.workunit.client.0.smithi077.stdout:7/457: rmdir da/d15/d1a/d6f 0 2024-04-26T04:53:04.283 INFO:tasks.workunit.client.0.smithi077.stdout:9/403: readlink d9/dd/d1b/d2b/d32/l63 0 2024-04-26T04:53:04.284 INFO:tasks.workunit.client.0.smithi077.stdout:9/404: fsync d9/dd/d3d/f59 0 2024-04-26T04:53:04.284 INFO:tasks.workunit.client.0.smithi077.stdout:9/405: fsync d9/f26 0 2024-04-26T04:53:04.288 INFO:tasks.workunit.client.0.smithi077.stdout:1/422: creat d9/f7a x:0 0 0 2024-04-26T04:53:04.290 INFO:tasks.workunit.client.0.smithi077.stdout:9/406: fsync d9/dd/d3d/f3e 0 2024-04-26T04:53:04.291 INFO:tasks.workunit.client.0.smithi077.stdout:7/458: mknod da/d13/c9d 0 2024-04-26T04:53:04.295 INFO:tasks.workunit.client.0.smithi077.stdout:7/459: creat da/d6b/d63/f9e x:0 0 0 2024-04-26T04:53:04.295 INFO:tasks.workunit.client.0.smithi077.stdout:0/354: link d0/c6c d0/d47/d68/c7d 0 2024-04-26T04:53:04.296 INFO:tasks.workunit.client.0.smithi077.stdout:7/460: truncate da/d15/d1a/d39/f7a 4440370 0 2024-04-26T04:53:04.297 INFO:tasks.workunit.client.0.smithi077.stdout:1/423: creat d9/db/dd/d68/f7b x:0 0 0 2024-04-26T04:53:04.298 INFO:tasks.workunit.client.0.smithi077.stdout:0/355: chown d0/d19/d1e/l3d 212 1 2024-04-26T04:53:04.303 INFO:tasks.workunit.client.0.smithi077.stdout:7/461: dwrite da/f89 [0,4194304] 0 2024-04-26T04:53:04.304 INFO:tasks.workunit.client.0.smithi077.stdout:1/424: getdents d9/db/dd 0 2024-04-26T04:53:04.304 INFO:tasks.workunit.client.0.smithi077.stdout:0/356: dread d0/fe [0,4194304] 0 2024-04-26T04:53:04.307 INFO:tasks.workunit.client.0.smithi077.stdout:1/425: rmdir d9/d53 39 2024-04-26T04:53:04.310 INFO:tasks.workunit.client.0.smithi077.stdout:0/357: write d0/d19/d1e/f26 [1003662,85136] 0 2024-04-26T04:53:04.312 INFO:tasks.workunit.client.0.smithi077.stdout:0/358: fsync d0/d47/f31 0 2024-04-26T04:53:04.315 INFO:tasks.workunit.client.0.smithi077.stdout:7/462: dwrite da/d6b/f93 [4194304,4194304] 0 2024-04-26T04:53:04.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/805: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/d55/f7f [0,4194304] 0 2024-04-26T04:53:04.321 INFO:tasks.workunit.client.0.smithi077.stdout:0/359: dwrite d0/d19/d1e/d2f/d52/f76 [0,4194304] 0 2024-04-26T04:53:04.326 INFO:tasks.workunit.client.1.smithi078.stdout:4/806: mknod d0/de/d22/deb/d104/d105/c11d 0 2024-04-26T04:53:04.330 INFO:tasks.workunit.client.0.smithi077.stdout:0/360: dwrite d0/d47/f22 [0,4194304] 0 2024-04-26T04:53:04.332 INFO:tasks.workunit.client.0.smithi077.stdout:0/361: write d0/d19/f49 [3230321,69623] 0 2024-04-26T04:53:04.332 INFO:tasks.workunit.client.1.smithi078.stdout:4/807: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/d62/l11e 0 2024-04-26T04:53:04.333 INFO:tasks.workunit.client.0.smithi077.stdout:4/408: truncate d4/d17/f57 306475 0 2024-04-26T04:53:04.336 INFO:tasks.workunit.client.0.smithi077.stdout:0/362: mknod d0/d19/d69/c7e 0 2024-04-26T04:53:04.336 INFO:tasks.workunit.client.0.smithi077.stdout:0/363: write d0/d47/f16 [225106,8354] 0 2024-04-26T04:53:04.343 INFO:tasks.workunit.client.0.smithi077.stdout:4/409: link d4/d4e/f77 d4/d17/d18/d43/d53/d75/f8e 0 2024-04-26T04:53:04.345 INFO:tasks.workunit.client.0.smithi077.stdout:4/410: write d4/d17/d18/d43/f74 [944657,19624] 0 2024-04-26T04:53:04.347 INFO:tasks.workunit.client.1.smithi078.stdout:2/818: mknod d8/d11/c120 0 2024-04-26T04:53:04.348 INFO:tasks.workunit.client.1.smithi078.stdout:1/869: getdents d7/d1e/d9e/dfc 0 2024-04-26T04:53:04.350 INFO:tasks.workunit.client.1.smithi078.stdout:2/819: creat d8/d11/d14/d69/d8f/d34/d70/def/f121 x:0 0 0 2024-04-26T04:53:04.351 INFO:tasks.workunit.client.1.smithi078.stdout:1/870: truncate d7/d1e/f36 2408921 0 2024-04-26T04:53:04.351 INFO:tasks.workunit.client.0.smithi077.stdout:3/325: write d0/d7/d15/f22 [1378417,123108] 0 2024-04-26T04:53:04.352 INFO:tasks.workunit.client.1.smithi078.stdout:2/820: fdatasync d8/d11/d14/f3b 0 2024-04-26T04:53:04.355 INFO:tasks.workunit.client.1.smithi078.stdout:1/871: fdatasync d7/d1e/d9e/db/d37/d58/d5c/fdc 0 2024-04-26T04:53:04.356 INFO:tasks.workunit.client.1.smithi078.stdout:1/872: write d7/d1e/fe2 [75054,94134] 0 2024-04-26T04:53:04.357 INFO:tasks.workunit.client.1.smithi078.stdout:8/803: rename d0/d19/d52/c12 to d0/d19/d25/d32/d8a/d92/da4/cff 0 2024-04-26T04:53:04.359 INFO:tasks.workunit.client.1.smithi078.stdout:3/824: rename d16/d26 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d11f 22 2024-04-26T04:53:04.360 INFO:tasks.workunit.client.1.smithi078.stdout:4/808: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/f7a [0,4194304] 0 2024-04-26T04:53:04.361 INFO:tasks.workunit.client.1.smithi078.stdout:1/873: dread d7/d1e/d9e/db/d47/f90 [0,4194304] 0 2024-04-26T04:53:04.363 INFO:tasks.workunit.client.1.smithi078.stdout:1/874: chown c6 4 1 2024-04-26T04:53:04.363 INFO:tasks.workunit.client.1.smithi078.stdout:4/809: write d0/f8 [3124304,127193] 0 2024-04-26T04:53:04.363 INFO:tasks.workunit.client.1.smithi078.stdout:6/880: rename d2/de/d27/da9/da/fc7 to d2/de/d27/da9/da/d36/db9/d97/d105/f11b 0 2024-04-26T04:53:04.366 INFO:tasks.workunit.client.1.smithi078.stdout:5/769: dwrite d7/d18/d32/d84/fb2 [0,4194304] 0 2024-04-26T04:53:04.367 INFO:tasks.workunit.client.1.smithi078.stdout:3/825: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/f120 x:0 0 0 2024-04-26T04:53:04.368 INFO:tasks.workunit.client.1.smithi078.stdout:5/770: chown d7/d48/d57/d65/f3a 42027303 1 2024-04-26T04:53:04.370 INFO:tasks.workunit.client.1.smithi078.stdout:1/875: read d7/d1e/d9e/db/d49/fd8 [9504,50280] 0 2024-04-26T04:53:04.374 INFO:tasks.workunit.client.1.smithi078.stdout:0/822: write d2/d9/d31/d73/f97 [870177,70606] 0 2024-04-26T04:53:04.375 INFO:tasks.workunit.client.1.smithi078.stdout:4/810: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/f8d [0,4194304] 0 2024-04-26T04:53:04.377 INFO:tasks.workunit.client.1.smithi078.stdout:5/771: creat d7/d48/d57/d65/d46/dce/ffd x:0 0 0 2024-04-26T04:53:04.378 INFO:tasks.workunit.client.1.smithi078.stdout:1/876: dwrite d7/d1e/d9e/db/d49/fb1 [0,4194304] 0 2024-04-26T04:53:04.378 INFO:tasks.workunit.client.1.smithi078.stdout:2/821: rename d8/d11/d14/d69/l82 to d8/d11/d18/d5d/l122 0 2024-04-26T04:53:04.378 INFO:tasks.workunit.client.1.smithi078.stdout:8/804: dread d0/d19/fa1 [0,4194304] 0 2024-04-26T04:53:04.380 INFO:tasks.workunit.client.1.smithi078.stdout:1/877: chown d7/d1e/d9e/db/f5a 227525114 1 2024-04-26T04:53:04.380 INFO:tasks.workunit.client.1.smithi078.stdout:5/772: stat d7/da/d1e/d91/ffa 0 2024-04-26T04:53:04.380 INFO:tasks.workunit.client.1.smithi078.stdout:0/823: mkdir d2/d9/d60/d6a/daa/d112 0 2024-04-26T04:53:04.390 INFO:tasks.workunit.client.1.smithi078.stdout:5/773: mkdir d7/d48/da9/deb/dfe 0 2024-04-26T04:53:04.393 INFO:tasks.workunit.client.1.smithi078.stdout:8/805: getdents d0/d19/d52/d5c/df0 0 2024-04-26T04:53:04.395 INFO:tasks.workunit.client.1.smithi078.stdout:1/878: link d7/lf9 d7/d1e/d9e/dd/df0/l11d 0 2024-04-26T04:53:04.396 INFO:tasks.workunit.client.0.smithi077.stdout:6/471: rename d8/d2a/d57 to d8/d52/da0 0 2024-04-26T04:53:04.399 INFO:tasks.workunit.client.1.smithi078.stdout:8/806: chown d0/l28 128967418 1 2024-04-26T04:53:04.402 INFO:tasks.workunit.client.0.smithi077.stdout:6/472: dwrite d8/d52/da0/f75 [0,4194304] 0 2024-04-26T04:53:04.404 INFO:tasks.workunit.client.1.smithi078.stdout:0/824: link d2/d9/d18/db7/fff d2/d1e/d5a/da3/f113 0 2024-04-26T04:53:04.406 INFO:tasks.workunit.client.0.smithi077.stdout:8/439: rename d4/c14 to d4/df/d22/d2d/c98 0 2024-04-26T04:53:04.409 INFO:tasks.workunit.client.1.smithi078.stdout:5/774: rename d7/d18/d32/d37/f6d to d7/d48/d57/d65/fff 0 2024-04-26T04:53:04.410 INFO:tasks.workunit.client.1.smithi078.stdout:1/879: fsync d7/d1e/d9e/db/d37/d58/d5c/dc0/f110 0 2024-04-26T04:53:04.414 INFO:tasks.workunit.client.0.smithi077.stdout:5/430: dwrite d1/d30/d35/f6c [0,4194304] 0 2024-04-26T04:53:04.415 INFO:tasks.workunit.client.0.smithi077.stdout:8/440: unlink d4/df/c47 0 2024-04-26T04:53:04.419 INFO:tasks.workunit.client.1.smithi078.stdout:9/750: write dc/df/d4b/d22/d2a/f4a [768533,20652] 0 2024-04-26T04:53:04.420 INFO:tasks.workunit.client.1.smithi078.stdout:5/775: truncate d7/d11/fc1 730404 0 2024-04-26T04:53:04.425 INFO:tasks.workunit.client.1.smithi078.stdout:7/776: mkdir d9/da/d11/d13/d102 0 2024-04-26T04:53:04.425 INFO:tasks.workunit.client.1.smithi078.stdout:0/825: creat d2/d9/d60/d6a/daa/d112/f114 x:0 0 0 2024-04-26T04:53:04.428 INFO:tasks.workunit.client.0.smithi077.stdout:6/473: creat d8/fa1 x:0 0 0 2024-04-26T04:53:04.431 INFO:tasks.workunit.client.0.smithi077.stdout:9/407: rename d9/d43/f61 to d9/f8d 0 2024-04-26T04:53:04.432 INFO:tasks.workunit.client.0.smithi077.stdout:5/431: dread d1/d12/d21/d25/f4b [0,4194304] 0 2024-04-26T04:53:04.432 INFO:tasks.workunit.client.1.smithi078.stdout:8/807: creat d0/f100 x:0 0 0 2024-04-26T04:53:04.433 INFO:tasks.workunit.client.0.smithi077.stdout:8/441: symlink d4/df/d22/l99 0 2024-04-26T04:53:04.433 INFO:tasks.workunit.client.0.smithi077.stdout:6/474: dread d8/f11 [0,4194304] 0 2024-04-26T04:53:04.434 INFO:tasks.workunit.client.0.smithi077.stdout:5/432: write d1/d42/d62/d6b/f77 [575331,13309] 0 2024-04-26T04:53:04.435 INFO:tasks.workunit.client.0.smithi077.stdout:5/433: stat d1/d12/d45 0 2024-04-26T04:53:04.440 INFO:tasks.workunit.client.0.smithi077.stdout:7/463: rename da/d6b/d63/d9c to da/d13/d22/d5d/d9f 0 2024-04-26T04:53:04.441 INFO:tasks.workunit.client.0.smithi077.stdout:0/364: rename d0/d47/d64 to d0/d47/d64/d7f 22 2024-04-26T04:53:04.445 INFO:tasks.workunit.client.0.smithi077.stdout:6/475: rmdir d8/d2a/d86 39 2024-04-26T04:53:04.446 INFO:tasks.workunit.client.1.smithi078.stdout:0/826: chown d2/d1e/d5a/da3/f113 0 1 2024-04-26T04:53:04.450 INFO:tasks.workunit.client.0.smithi077.stdout:5/434: creat d1/d12/d28/f9f x:0 0 0 2024-04-26T04:53:04.450 INFO:tasks.workunit.client.1.smithi078.stdout:0/827: truncate d2/f2e 1038245 0 2024-04-26T04:53:04.451 INFO:tasks.workunit.client.1.smithi078.stdout:5/776: getdents d7/d18/d1c/d8e 0 2024-04-26T04:53:04.453 INFO:tasks.workunit.client.1.smithi078.stdout:0/828: symlink d2/d1e/d5a/l115 0 2024-04-26T04:53:04.454 INFO:tasks.workunit.client.0.smithi077.stdout:0/365: creat d0/d47/d64/f80 x:0 0 0 2024-04-26T04:53:04.455 INFO:tasks.workunit.client.1.smithi078.stdout:8/808: dread d0/fe4 [0,4194304] 0 2024-04-26T04:53:04.455 INFO:tasks.workunit.client.1.smithi078.stdout:5/777: mkdir d7/d48/d57/d65/d46/dce/d100 0 2024-04-26T04:53:04.458 INFO:tasks.workunit.client.0.smithi077.stdout:6/476: symlink d8/d52/la2 0 2024-04-26T04:53:04.459 INFO:tasks.workunit.client.1.smithi078.stdout:0/829: dwrite d2/d9/d18/f49 [4194304,4194304] 0 2024-04-26T04:53:04.459 INFO:tasks.workunit.client.0.smithi077.stdout:5/435: creat d1/d12/d21/fa0 x:0 0 0 2024-04-26T04:53:04.466 INFO:tasks.workunit.client.0.smithi077.stdout:0/366: truncate d0/f4 1014018 0 2024-04-26T04:53:04.470 INFO:tasks.workunit.client.0.smithi077.stdout:7/464: link da/d13/d22/f27 da/d13/d22/d34/fa0 0 2024-04-26T04:53:04.471 INFO:tasks.workunit.client.0.smithi077.stdout:7/465: fsync da/f87 0 2024-04-26T04:53:04.472 INFO:tasks.workunit.client.0.smithi077.stdout:9/408: creat d9/dd/d1b/d2b/d32/f8e x:0 0 0 2024-04-26T04:53:04.476 INFO:tasks.workunit.client.0.smithi077.stdout:4/411: rename d4/d17/d18/d43/d53/d56/l4d to d4/l8f 0 2024-04-26T04:53:04.479 INFO:tasks.workunit.client.1.smithi078.stdout:8/809: dread d0/d19/d25/d32/d8a/f5e [0,4194304] 0 2024-04-26T04:53:04.481 INFO:tasks.workunit.client.0.smithi077.stdout:5/436: symlink d1/d8f/la1 0 2024-04-26T04:53:04.483 INFO:tasks.workunit.client.0.smithi077.stdout:0/367: dwrite d0/d19/d1e/d2f/d52/f62 [0,4194304] 0 2024-04-26T04:53:04.484 INFO:tasks.workunit.client.1.smithi078.stdout:4/811: write d0/de/d22/fb7 [1537917,56039] 0 2024-04-26T04:53:04.484 INFO:tasks.workunit.client.0.smithi077.stdout:0/368: chown d0/d47/f22 49314488 1 2024-04-26T04:53:04.486 INFO:tasks.workunit.client.1.smithi078.stdout:8/810: creat d0/df8/f101 x:0 0 0 2024-04-26T04:53:04.488 INFO:tasks.workunit.client.1.smithi078.stdout:4/812: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/l11f 0 2024-04-26T04:53:04.489 INFO:tasks.workunit.client.1.smithi078.stdout:4/813: readlink d0/de/d22/d71/dae/lda 0 2024-04-26T04:53:04.495 INFO:tasks.workunit.client.0.smithi077.stdout:7/466: dread da/d6b/d63/f81 [0,4194304] 0 2024-04-26T04:53:04.509 INFO:tasks.workunit.client.1.smithi078.stdout:8/811: rmdir d0/d19/d25/d32/d8a/d92/da4 39 2024-04-26T04:53:04.529 INFO:tasks.workunit.client.1.smithi078.stdout:6/881: sync 2024-04-26T04:53:04.535 INFO:tasks.workunit.client.0.smithi077.stdout:5/437: creat d1/d8/d72/fa2 x:0 0 0 2024-04-26T04:53:04.536 INFO:tasks.workunit.client.1.smithi078.stdout:4/814: symlink d0/l120 0 2024-04-26T04:53:04.537 INFO:tasks.workunit.client.1.smithi078.stdout:8/812: mkdir d0/d19/d52/d35/d44/d6b/dc3/d102 0 2024-04-26T04:53:04.538 INFO:tasks.workunit.client.0.smithi077.stdout:9/409: link d9/d3a/l44 d9/d43/l8f 0 2024-04-26T04:53:04.539 INFO:tasks.workunit.client.1.smithi078.stdout:1/880: write f1 [1823694,68997] 0 2024-04-26T04:53:04.540 INFO:tasks.workunit.client.1.smithi078.stdout:8/813: mkdir d0/d19/d52/d5c/da2/d103 0 2024-04-26T04:53:04.542 INFO:tasks.workunit.client.1.smithi078.stdout:4/815: dread d0/f37 [0,4194304] 0 2024-04-26T04:53:04.543 INFO:tasks.workunit.client.1.smithi078.stdout:4/816: fsync d0/d1d/de1/f107 0 2024-04-26T04:53:04.544 INFO:tasks.workunit.client.0.smithi077.stdout:3/326: rename d0/d7/d1f/f43 to d0/f67 0 2024-04-26T04:53:04.544 INFO:tasks.workunit.client.1.smithi078.stdout:4/817: stat d0/de/d22/d27/d7b/f82 0 2024-04-26T04:53:04.545 INFO:tasks.workunit.client.0.smithi077.stdout:3/327: read - d0/d7/d15/f3f zero size 2024-04-26T04:53:04.545 INFO:tasks.workunit.client.1.smithi078.stdout:8/814: dwrite d0/fe4 [0,4194304] 0 2024-04-26T04:53:04.546 INFO:tasks.workunit.client.1.smithi078.stdout:4/818: chown d0/d1d/de1/d20/d31/d47/db2/d44/d55/f95 1814 1 2024-04-26T04:53:04.548 INFO:tasks.workunit.client.1.smithi078.stdout:9/751: write dc/df/d27/f53 [1153985,82106] 0 2024-04-26T04:53:04.548 INFO:tasks.workunit.client.0.smithi077.stdout:5/438: dread d1/d12/f2a [0,4194304] 0 2024-04-26T04:53:04.554 INFO:tasks.workunit.client.1.smithi078.stdout:9/752: dread dc/df/d4b/d22/d2a/d92/fa2 [0,4194304] 0 2024-04-26T04:53:04.555 INFO:tasks.workunit.client.1.smithi078.stdout:7/777: rename d9/f90 to d9/da/d11/d13/d1a/d2d/f103 0 2024-04-26T04:53:04.560 INFO:tasks.workunit.client.0.smithi077.stdout:0/369: rename d0/d19/d1e/d78 to d0/d19/d1e/d81 0 2024-04-26T04:53:04.563 INFO:tasks.workunit.client.0.smithi077.stdout:5/439: fdatasync d1/d12/d45/f63 0 2024-04-26T04:53:04.569 INFO:tasks.workunit.client.1.smithi078.stdout:8/815: rename d0/d19/d25/d36 to d0/d19/d52/d35/d44/d6b/d104 0 2024-04-26T04:53:04.570 INFO:tasks.workunit.client.1.smithi078.stdout:5/778: dwrite d7/d48/d57/d65/d7a/f98 [0,4194304] 0 2024-04-26T04:53:04.571 INFO:tasks.workunit.client.0.smithi077.stdout:3/328: fsync d0/d7/d1f/d4d/f61 0 2024-04-26T04:53:04.574 INFO:tasks.workunit.client.1.smithi078.stdout:5/779: dread - d7/d18/d32/d37/d79/fa1 zero size 2024-04-26T04:53:04.576 INFO:tasks.workunit.client.1.smithi078.stdout:9/753: link dc/df/f1d dc/df/d27/fe3 0 2024-04-26T04:53:04.576 INFO:tasks.workunit.client.1.smithi078.stdout:9/754: readlink dc/d55/lb0 0 2024-04-26T04:53:04.577 INFO:tasks.workunit.client.1.smithi078.stdout:0/830: dwrite d2/d9/d31/d4a/f7a [0,4194304] 0 2024-04-26T04:53:04.578 INFO:tasks.workunit.client.0.smithi077.stdout:6/477: dwrite d8/f5a [0,4194304] 0 2024-04-26T04:53:04.579 INFO:tasks.workunit.client.1.smithi078.stdout:2/822: sync 2024-04-26T04:53:04.579 INFO:tasks.workunit.client.1.smithi078.stdout:3/826: sync 2024-04-26T04:53:04.582 INFO:tasks.workunit.client.1.smithi078.stdout:9/755: dread dc/df/d4b/d22/d2a/d92/fa3 [0,4194304] 0 2024-04-26T04:53:04.585 INFO:tasks.workunit.client.1.smithi078.stdout:7/778: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d104 0 2024-04-26T04:53:04.587 INFO:tasks.workunit.client.1.smithi078.stdout:4/819: mkdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121 0 2024-04-26T04:53:04.589 INFO:tasks.workunit.client.1.smithi078.stdout:2/823: dwrite d8/d11/d6e/f7b [4194304,4194304] 0 2024-04-26T04:53:04.590 INFO:tasks.workunit.client.1.smithi078.stdout:7/779: stat d9/da/d33/d4b/d4c/d5b/d83/de6 0 2024-04-26T04:53:04.594 INFO:tasks.workunit.client.1.smithi078.stdout:9/756: dread dc/df/d4b/d22/dbe/dcd/fda [0,4194304] 0 2024-04-26T04:53:04.594 INFO:tasks.workunit.client.1.smithi078.stdout:5/780: mknod d7/da/d1e/c101 0 2024-04-26T04:53:04.594 INFO:tasks.workunit.client.1.smithi078.stdout:4/820: write d0/d1d/de1/d20/d31/d47/db2/d44/d55/f5d [2603276,128256] 0 2024-04-26T04:53:04.594 INFO:tasks.workunit.client.1.smithi078.stdout:9/757: chown dc/df/d8a/fb9 1 1 2024-04-26T04:53:04.594 INFO:tasks.workunit.client.1.smithi078.stdout:0/831: fsync f1 0 2024-04-26T04:53:04.595 INFO:tasks.workunit.client.1.smithi078.stdout:6/882: write d2/d1a/f9b [3055101,43303] 0 2024-04-26T04:53:04.596 INFO:tasks.workunit.client.1.smithi078.stdout:7/780: truncate d9/da/d11/d13/d1a/dd4/fdf 265853 0 2024-04-26T04:53:04.598 INFO:tasks.workunit.client.1.smithi078.stdout:1/881: write d7/d1e/d9e/db/f15 [1075968,127767] 0 2024-04-26T04:53:04.600 INFO:tasks.workunit.client.1.smithi078.stdout:3/827: symlink d16/d26/d2b/d31/d78/l121 0 2024-04-26T04:53:04.603 INFO:tasks.workunit.client.1.smithi078.stdout:2/824: fdatasync d8/d11/f48 0 2024-04-26T04:53:04.607 INFO:tasks.workunit.client.1.smithi078.stdout:4/821: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/dba/l122 0 2024-04-26T04:53:04.608 INFO:tasks.workunit.client.1.smithi078.stdout:7/781: stat d9/da/d11/d13/cbd 0 2024-04-26T04:53:04.609 INFO:tasks.workunit.client.1.smithi078.stdout:6/883: mkdir d2/de/d27/da9/d2a/dbf/d5f/d11c 0 2024-04-26T04:53:04.614 INFO:tasks.workunit.client.1.smithi078.stdout:0/832: symlink d2/d9/d18/d7d/da9/dab/l116 0 2024-04-26T04:53:04.614 INFO:tasks.workunit.client.1.smithi078.stdout:8/816: creat d0/d19/d52/d35/f105 x:0 0 0 2024-04-26T04:53:04.614 INFO:tasks.workunit.client.1.smithi078.stdout:3/828: chown d16/d26/d2b/d66/fc2 1431909 1 2024-04-26T04:53:04.615 INFO:tasks.workunit.client.1.smithi078.stdout:0/833: write d2/d9/d18/db7/fd1 [8304970,85382] 0 2024-04-26T04:53:04.617 INFO:tasks.workunit.client.1.smithi078.stdout:3/829: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/f7d [3384257,79719] 0 2024-04-26T04:53:04.617 INFO:tasks.workunit.client.1.smithi078.stdout:5/781: link d7/d48/d57/d65/d7a/d9c/fe9 d7/da/d1e/d91/f102 0 2024-04-26T04:53:04.620 INFO:tasks.workunit.client.1.smithi078.stdout:4/822: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/fde zero size 2024-04-26T04:53:04.637 INFO:tasks.workunit.client.1.smithi078.stdout:2/825: mkdir d8/d11/d18/dbb/d123 0 2024-04-26T04:53:04.638 INFO:tasks.workunit.client.1.smithi078.stdout:8/817: mknod d0/d19/d52/d35/d44/d6b/d6c/c106 0 2024-04-26T04:53:04.639 INFO:tasks.workunit.client.1.smithi078.stdout:5/782: creat d7/d18/d1c/d8e/f103 x:0 0 0 2024-04-26T04:53:04.641 INFO:tasks.workunit.client.1.smithi078.stdout:7/782: fsync d9/da/d20/f9b 0 2024-04-26T04:53:04.642 INFO:tasks.workunit.client.1.smithi078.stdout:4/823: truncate d0/de/f1e 5235048 0 2024-04-26T04:53:04.642 INFO:tasks.workunit.client.1.smithi078.stdout:7/783: readlink d9/da/d11/d13/d1a/d52/l5f 0 2024-04-26T04:53:04.643 INFO:tasks.workunit.client.1.smithi078.stdout:6/884: symlink d2/d1a/d38/d10a/l11d 0 2024-04-26T04:53:04.646 INFO:tasks.workunit.client.0.smithi077.stdout:0/370: mkdir d0/d19/d1e/d82 0 2024-04-26T04:53:04.647 INFO:tasks.workunit.client.0.smithi077.stdout:4/412: dwrite d4/d17/d18/f5d [0,4194304] 0 2024-04-26T04:53:04.647 INFO:tasks.workunit.client.0.smithi077.stdout:6/478: chown d8/d52/d53/c71 7080 1 2024-04-26T04:53:04.648 INFO:tasks.workunit.client.0.smithi077.stdout:7/467: write da/d15/f2f [533663,35085] 0 2024-04-26T04:53:04.648 INFO:tasks.workunit.client.1.smithi078.stdout:5/783: rename d7/d18/d1c/d5a to d7/d48/d57/d65/d7a/d9c/d104 0 2024-04-26T04:53:04.651 INFO:tasks.workunit.client.0.smithi077.stdout:5/440: creat d1/fa3 x:0 0 0 2024-04-26T04:53:04.652 INFO:tasks.workunit.client.0.smithi077.stdout:5/441: chown d1/d30/d35 71269 1 2024-04-26T04:53:04.653 INFO:tasks.workunit.client.0.smithi077.stdout:9/410: link d9/d3a/d79/c74 d9/dd/d1b/d2b/d32/d36/c90 0 2024-04-26T04:53:04.655 INFO:tasks.workunit.client.1.smithi078.stdout:4/824: creat d0/de/d22/deb/d104/d105/f123 x:0 0 0 2024-04-26T04:53:04.659 INFO:tasks.workunit.client.0.smithi077.stdout:7/468: fsync da/d15/f94 0 2024-04-26T04:53:04.659 INFO:tasks.workunit.client.0.smithi077.stdout:7/469: chown da 1757796 1 2024-04-26T04:53:04.660 INFO:tasks.workunit.client.1.smithi078.stdout:2/826: creat d8/d11/d18/d5d/d8b/db0/d10d/d110/f124 x:0 0 0 2024-04-26T04:53:04.663 INFO:tasks.workunit.client.1.smithi078.stdout:8/818: creat d0/d7a/d83/df3/f107 x:0 0 0 2024-04-26T04:53:04.665 INFO:tasks.workunit.client.0.smithi077.stdout:5/442: rename d1/d12/d21 to d1/d42/da4 0 2024-04-26T04:53:04.666 INFO:tasks.workunit.client.0.smithi077.stdout:5/443: dread - d1/fa3 zero size 2024-04-26T04:53:04.667 INFO:tasks.workunit.client.1.smithi078.stdout:5/784: symlink d7/d48/d57/d65/d46/dc4/l105 0 2024-04-26T04:53:04.668 INFO:tasks.workunit.client.0.smithi077.stdout:9/411: chown d9/dd/c84 0 1 2024-04-26T04:53:04.668 INFO:tasks.workunit.client.1.smithi078.stdout:0/834: getdents d2/d9/d60/dd5 0 2024-04-26T04:53:04.670 INFO:tasks.workunit.client.1.smithi078.stdout:4/825: mknod d0/d1d/db4/c124 0 2024-04-26T04:53:04.670 INFO:tasks.workunit.client.0.smithi077.stdout:5/444: dread d1/d42/f69 [0,4194304] 0 2024-04-26T04:53:04.671 INFO:tasks.workunit.client.1.smithi078.stdout:1/882: sync 2024-04-26T04:53:04.671 INFO:tasks.workunit.client.1.smithi078.stdout:4/826: fsync d0/d1d/de1/d20/d31/d47/db2/d4c/d62/ffe 0 2024-04-26T04:53:04.672 INFO:tasks.workunit.client.0.smithi077.stdout:0/371: link d0/d19/d1e/f34 d0/d19/d1e/d81/f83 0 2024-04-26T04:53:04.676 INFO:tasks.workunit.client.1.smithi078.stdout:1/883: dread d7/d1e/d9e/d4c/d3d/f95 [0,4194304] 0 2024-04-26T04:53:04.676 INFO:tasks.workunit.client.1.smithi078.stdout:6/885: creat d2/de/d27/da9/d2a/dbf/d5f/d11c/f11e x:0 0 0 2024-04-26T04:53:04.678 INFO:tasks.workunit.client.0.smithi077.stdout:9/412: creat d9/d35/f91 x:0 0 0 2024-04-26T04:53:04.680 INFO:tasks.workunit.client.0.smithi077.stdout:0/372: creat d0/d19/d69/f84 x:0 0 0 2024-04-26T04:53:04.680 INFO:tasks.workunit.client.1.smithi078.stdout:0/835: dread d2/d9/d18/d7d/da9/dab/dc6/fb2 [0,4194304] 0 2024-04-26T04:53:04.681 INFO:tasks.workunit.client.1.smithi078.stdout:5/785: creat d7/d48/d57/d65/d70/dca/f106 x:0 0 0 2024-04-26T04:53:04.681 INFO:tasks.workunit.client.0.smithi077.stdout:6/479: link d8/d55/c9d d8/d1c/ca3 0 2024-04-26T04:53:04.682 INFO:tasks.workunit.client.0.smithi077.stdout:5/445: dread d1/f5 [0,4194304] 0 2024-04-26T04:53:04.683 INFO:tasks.workunit.client.1.smithi078.stdout:6/886: dwrite d2/de/d27/da9/de5/df4/ff8 [0,4194304] 0 2024-04-26T04:53:04.684 INFO:tasks.workunit.client.1.smithi078.stdout:9/758: truncate dc/df/d4b/d22/d2a/d57/f90 3885409 0 2024-04-26T04:53:04.684 INFO:tasks.workunit.client.1.smithi078.stdout:6/887: chown d2/d52/f103 869534770 1 2024-04-26T04:53:04.685 INFO:tasks.workunit.client.1.smithi078.stdout:9/759: write dc/df/d4b/d22/d2a/d57/fa4 [1335647,76319] 0 2024-04-26T04:53:04.687 INFO:tasks.workunit.client.1.smithi078.stdout:9/760: dread - dc/df/d4b/d22/d2d/d98/fd4 zero size 2024-04-26T04:53:04.687 INFO:tasks.workunit.client.0.smithi077.stdout:9/413: symlink d9/dd/d1b/d2b/l92 0 2024-04-26T04:53:04.688 INFO:tasks.workunit.client.0.smithi077.stdout:9/414: chown d9/dd/d1b/d4e/c6c 31091589 1 2024-04-26T04:53:04.691 INFO:tasks.workunit.client.0.smithi077.stdout:0/373: rmdir d0 39 2024-04-26T04:53:04.692 INFO:tasks.workunit.client.1.smithi078.stdout:2/827: symlink d8/d11/d14/d5b/l125 0 2024-04-26T04:53:04.696 INFO:tasks.workunit.client.1.smithi078.stdout:8/819: mknod d0/d19/d52/d35/d44/d6b/d6c/db4/c108 0 2024-04-26T04:53:04.700 INFO:tasks.workunit.client.1.smithi078.stdout:3/830: dwrite d16/d26/d2e/d74/f4f [0,4194304] 0 2024-04-26T04:53:04.700 INFO:tasks.workunit.client.1.smithi078.stdout:8/820: dread - d0/d19/d52/d35/ff1 zero size 2024-04-26T04:53:04.702 INFO:tasks.workunit.client.1.smithi078.stdout:5/786: creat d7/d48/d57/d65/d7a/d9c/f107 x:0 0 0 2024-04-26T04:53:04.704 INFO:tasks.workunit.client.0.smithi077.stdout:9/415: link d9/dd/d1b/d2b/l92 d9/d3a/d79/d50/l93 0 2024-04-26T04:53:04.704 INFO:tasks.workunit.client.1.smithi078.stdout:8/821: chown d0/d19/d25/d32/d8a/ccb 350265 1 2024-04-26T04:53:04.705 INFO:tasks.workunit.client.1.smithi078.stdout:2/828: dwrite d8/d11/d18/d5d/d61/f1b [4194304,4194304] 0 2024-04-26T04:53:04.705 INFO:tasks.workunit.client.1.smithi078.stdout:3/831: write d16/d26/d2b/d31/d71/d8a/f107 [672214,20594] 0 2024-04-26T04:53:04.706 INFO:tasks.workunit.client.0.smithi077.stdout:4/413: dwrite d4/d17/f3d [0,4194304] 0 2024-04-26T04:53:04.708 INFO:tasks.workunit.client.0.smithi077.stdout:4/414: write d4/de/d15/f62 [1886660,130313] 0 2024-04-26T04:53:04.712 INFO:tasks.workunit.client.1.smithi078.stdout:5/787: truncate d7/d48/d57/d65/d70/dca/fed 686033 0 2024-04-26T04:53:04.712 INFO:tasks.workunit.client.1.smithi078.stdout:7/784: dwrite d9/da/d11/d13/d1a/d52/d5a/f44 [4194304,4194304] 0 2024-04-26T04:53:04.713 INFO:tasks.workunit.client.0.smithi077.stdout:9/416: unlink d9/dd/d3d/l5b 0 2024-04-26T04:53:04.714 INFO:tasks.workunit.client.1.smithi078.stdout:8/822: write d0/d19/d52/d35/d44/d6b/d104/d58/d5b/ff4 [700319,111618] 0 2024-04-26T04:53:04.715 INFO:tasks.workunit.client.1.smithi078.stdout:8/823: stat d0/d7a/dbe/lf5 0 2024-04-26T04:53:04.719 INFO:tasks.workunit.client.1.smithi078.stdout:9/761: rename dc/df/d4b/d22/dbe/dcd/fda to dc/df/d4b/d22/dbb/fe4 0 2024-04-26T04:53:04.719 INFO:tasks.workunit.client.1.smithi078.stdout:9/762: readlink dc/df/d4b/d22/d2a/d57/l8e 0 2024-04-26T04:53:04.723 INFO:tasks.workunit.client.0.smithi077.stdout:9/417: mkdir d9/d7d/d94 0 2024-04-26T04:53:04.725 INFO:tasks.workunit.client.0.smithi077.stdout:9/418: chown d9/dd/d1b/c21 275504010 1 2024-04-26T04:53:04.726 INFO:tasks.workunit.client.1.smithi078.stdout:1/884: fsync d7/d1e/d9e/db/d49/dba/fc8 0 2024-04-26T04:53:04.726 INFO:tasks.workunit.client.0.smithi077.stdout:4/415: dwrite d4/de/d15/d1f/d35/f52 [0,4194304] 0 2024-04-26T04:53:04.727 INFO:tasks.workunit.client.0.smithi077.stdout:9/419: chown d9/dd/d1b/d2b/f2f 305353 1 2024-04-26T04:53:04.732 INFO:tasks.workunit.client.0.smithi077.stdout:0/374: dread d0/f15 [0,4194304] 0 2024-04-26T04:53:04.732 INFO:tasks.workunit.client.0.smithi077.stdout:0/375: fsync d0/d47/f22 0 2024-04-26T04:53:04.740 INFO:tasks.workunit.client.1.smithi078.stdout:4/827: dwrite d0/d1d/de1/d20/d31/d47/db2/fa3 [0,4194304] 0 2024-04-26T04:53:04.741 INFO:tasks.workunit.client.0.smithi077.stdout:6/480: dwrite d8/fe [0,4194304] 0 2024-04-26T04:53:04.742 INFO:tasks.workunit.client.0.smithi077.stdout:0/376: dread d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:04.744 INFO:tasks.workunit.client.1.smithi078.stdout:7/785: unlink d9/da/d33/d4b/d4c/cd1 0 2024-04-26T04:53:04.745 INFO:tasks.workunit.client.0.smithi077.stdout:9/420: mkdir d9/d3a/d79/d95 0 2024-04-26T04:53:04.745 INFO:tasks.workunit.client.1.smithi078.stdout:3/832: read d16/d26/d2b/d31/d71/d8a/fca [2324563,122214] 0 2024-04-26T04:53:04.746 INFO:tasks.workunit.client.1.smithi078.stdout:0/836: dwrite d2/d9/d31/d73/fa7 [4194304,4194304] 0 2024-04-26T04:53:04.747 INFO:tasks.workunit.client.0.smithi077.stdout:6/481: chown d8/c1f 3 1 2024-04-26T04:53:04.748 INFO:tasks.workunit.client.0.smithi077.stdout:0/377: creat d0/d28/d40/f85 x:0 0 0 2024-04-26T04:53:04.749 INFO:tasks.workunit.client.1.smithi078.stdout:0/837: write d2/d9/d31/d73/f97 [1142359,126175] 0 2024-04-26T04:53:04.751 INFO:tasks.workunit.client.1.smithi078.stdout:6/888: write d2/f14 [875443,126063] 0 2024-04-26T04:53:04.756 INFO:tasks.workunit.client.1.smithi078.stdout:9/763: dwrite dc/df/d27/d81/f66 [0,4194304] 0 2024-04-26T04:53:04.756 INFO:tasks.workunit.client.0.smithi077.stdout:6/482: dwrite d8/d1c/f5c [0,4194304] 0 2024-04-26T04:53:04.767 INFO:tasks.workunit.client.0.smithi077.stdout:0/378: creat d0/d47/d68/f86 x:0 0 0 2024-04-26T04:53:04.769 INFO:tasks.workunit.client.1.smithi078.stdout:4/828: creat d0/d1d/de1/d20/d31/d47/db2/f125 x:0 0 0 2024-04-26T04:53:04.770 INFO:tasks.workunit.client.0.smithi077.stdout:6/483: mkdir d8/d1c/da4 0 2024-04-26T04:53:04.772 INFO:tasks.workunit.client.0.smithi077.stdout:0/379: dread d0/d47/f22 [0,4194304] 0 2024-04-26T04:53:04.772 INFO:tasks.workunit.client.0.smithi077.stdout:6/484: write d8/f6d [213829,113491] 0 2024-04-26T04:53:04.774 INFO:tasks.workunit.client.1.smithi078.stdout:7/786: rmdir d9/da/d11 39 2024-04-26T04:53:04.775 INFO:tasks.workunit.client.0.smithi077.stdout:4/416: dread d4/de/d15/f62 [4194304,4194304] 0 2024-04-26T04:53:04.776 INFO:tasks.workunit.client.0.smithi077.stdout:6/485: read d8/d2a/f2e [1794106,87838] 0 2024-04-26T04:53:04.777 INFO:tasks.workunit.client.0.smithi077.stdout:0/380: dread d0/d47/ff [0,4194304] 0 2024-04-26T04:53:04.778 INFO:tasks.workunit.client.1.smithi078.stdout:8/824: unlink d0/d19/d52/l1b 0 2024-04-26T04:53:04.779 INFO:tasks.workunit.client.1.smithi078.stdout:3/833: mknod d16/d26/d2e/d5f/dbb/c122 0 2024-04-26T04:53:04.782 INFO:tasks.workunit.client.1.smithi078.stdout:6/889: mkdir d2/de/d27/da9/de5/df4/d11f 0 2024-04-26T04:53:04.783 INFO:tasks.workunit.client.0.smithi077.stdout:6/486: creat d8/d52/d53/d5e/fa5 x:0 0 0 2024-04-26T04:53:04.784 INFO:tasks.workunit.client.0.smithi077.stdout:0/381: creat d0/d19/f87 x:0 0 0 2024-04-26T04:53:04.784 INFO:tasks.workunit.client.1.smithi078.stdout:0/838: dread d2/d39/d96/fc8 [0,4194304] 0 2024-04-26T04:53:04.787 INFO:tasks.workunit.client.0.smithi077.stdout:6/487: mkdir d8/d52/d53/d5e/da6 0 2024-04-26T04:53:04.787 INFO:tasks.workunit.client.1.smithi078.stdout:5/788: link d7/d48/d57/d65/d46/de7/lef d7/d48/d57/d65/d46/l108 0 2024-04-26T04:53:04.788 INFO:tasks.workunit.client.1.smithi078.stdout:6/890: dwrite d2/d1a/f118 [0,4194304] 0 2024-04-26T04:53:04.791 INFO:tasks.workunit.client.0.smithi077.stdout:0/382: dwrite d0/d19/d1e/f26 [0,4194304] 0 2024-04-26T04:53:04.791 INFO:tasks.workunit.client.1.smithi078.stdout:8/825: symlink d0/d19/d52/d35/d44/l109 0 2024-04-26T04:53:04.793 INFO:tasks.workunit.client.1.smithi078.stdout:3/834: fsync d16/d26/f2a 0 2024-04-26T04:53:04.794 INFO:tasks.workunit.client.0.smithi077.stdout:0/383: read - d0/d19/d1e/d59/f6e zero size 2024-04-26T04:53:04.794 INFO:tasks.workunit.client.1.smithi078.stdout:3/835: dread - d16/d26/d2e/d5f/f11c zero size 2024-04-26T04:53:04.795 INFO:tasks.workunit.client.1.smithi078.stdout:9/764: unlink dc/df/d17/f2e 0 2024-04-26T04:53:04.795 INFO:tasks.workunit.client.1.smithi078.stdout:3/836: write f6 [2218738,44553] 0 2024-04-26T04:53:04.796 INFO:tasks.workunit.client.0.smithi077.stdout:6/488: dwrite d8/d52/da0/f9a [0,4194304] 0 2024-04-26T04:53:04.796 INFO:tasks.workunit.client.1.smithi078.stdout:0/839: write d2/d39/d96/fe9 [2141149,92545] 0 2024-04-26T04:53:04.803 INFO:tasks.workunit.client.0.smithi077.stdout:6/489: mkdir d8/d52/d53/da7 0 2024-04-26T04:53:04.804 INFO:tasks.workunit.client.0.smithi077.stdout:6/490: dread - d8/d52/d53/d5e/fa5 zero size 2024-04-26T04:53:04.805 INFO:tasks.workunit.client.1.smithi078.stdout:5/789: mkdir d7/d48/d57/d65/d46/dce/d109 0 2024-04-26T04:53:04.807 INFO:tasks.workunit.client.1.smithi078.stdout:6/891: dread - d2/de/d27/da9/d2a/d4d/fa7 zero size 2024-04-26T04:53:04.807 INFO:tasks.workunit.client.1.smithi078.stdout:1/885: write d7/d1e/d9e/dd/fd5 [249997,119613] 0 2024-04-26T04:53:04.807 INFO:tasks.workunit.client.1.smithi078.stdout:7/787: symlink d9/da/d11/d13/d1a/d2d/l105 0 2024-04-26T04:53:04.808 INFO:tasks.workunit.client.1.smithi078.stdout:2/829: write d8/d11/f54 [2545774,124596] 0 2024-04-26T04:53:04.809 INFO:tasks.workunit.client.0.smithi077.stdout:9/421: truncate d9/d45/f7e 3326494 0 2024-04-26T04:53:04.809 INFO:tasks.workunit.client.1.smithi078.stdout:7/788: chown d9/da/d33/fb0 3 1 2024-04-26T04:53:04.809 INFO:tasks.workunit.client.1.smithi078.stdout:2/830: stat d8/d11/d18/dfe 0 2024-04-26T04:53:04.810 INFO:tasks.workunit.client.0.smithi077.stdout:9/422: truncate d9/dd/d1b/d2b/f72 407745 0 2024-04-26T04:53:04.811 INFO:tasks.workunit.client.0.smithi077.stdout:9/423: dread d9/d43/f56 [0,4194304] 0 2024-04-26T04:53:04.812 INFO:tasks.workunit.client.0.smithi077.stdout:6/491: link d8/d52/d53/f80 d8/d1c/da4/fa8 0 2024-04-26T04:53:04.816 INFO:tasks.workunit.client.0.smithi077.stdout:2/424: sync 2024-04-26T04:53:04.816 INFO:tasks.workunit.client.0.smithi077.stdout:1/426: sync 2024-04-26T04:53:04.816 INFO:tasks.workunit.client.0.smithi077.stdout:8/442: sync 2024-04-26T04:53:04.818 INFO:tasks.workunit.client.1.smithi078.stdout:9/765: rmdir dc/dd9 39 2024-04-26T04:53:04.822 INFO:tasks.workunit.client.0.smithi077.stdout:1/427: chown d9/d53/d6e/c3c 18022 1 2024-04-26T04:53:04.823 INFO:tasks.workunit.client.1.smithi078.stdout:6/892: dread d2/de/d27/da9/da/f4a [0,4194304] 0 2024-04-26T04:53:04.823 INFO:tasks.workunit.client.0.smithi077.stdout:1/428: readlink d9/l46 0 2024-04-26T04:53:04.823 INFO:tasks.workunit.client.0.smithi077.stdout:2/425: read - d9/dc/f7a zero size 2024-04-26T04:53:04.824 INFO:tasks.workunit.client.1.smithi078.stdout:6/893: chown d2/de/d27/d58/d5b/fd6 1805800 1 2024-04-26T04:53:04.827 INFO:tasks.workunit.client.0.smithi077.stdout:9/424: symlink d9/d7d/d94/l96 0 2024-04-26T04:53:04.828 INFO:tasks.workunit.client.0.smithi077.stdout:4/417: dwrite d4/de/f44 [0,4194304] 0 2024-04-26T04:53:04.828 INFO:tasks.workunit.client.1.smithi078.stdout:8/826: creat d0/d19/d52/d35/d44/d6b/d104/d58/d5b/dee/f10a x:0 0 0 2024-04-26T04:53:04.828 INFO:tasks.workunit.client.1.smithi078.stdout:6/894: dwrite d2/de/d27/da9/d11/fc2 [0,4194304] 0 2024-04-26T04:53:04.830 INFO:tasks.workunit.client.0.smithi077.stdout:8/443: truncate d4/df/d22/d2d/d2b/d33/f39 118909 0 2024-04-26T04:53:04.833 INFO:tasks.workunit.client.0.smithi077.stdout:4/418: write d4/de/d15/d1f/d35/f52 [656471,101231] 0 2024-04-26T04:53:04.834 INFO:tasks.workunit.client.0.smithi077.stdout:6/492: mknod d8/d2a/ca9 0 2024-04-26T04:53:04.836 INFO:tasks.workunit.client.0.smithi077.stdout:1/429: dwrite d9/db/dd/d34/d63/f77 [0,4194304] 0 2024-04-26T04:53:04.836 INFO:tasks.workunit.client.0.smithi077.stdout:4/419: fdatasync d4/de/d15/f5b 0 2024-04-26T04:53:04.839 INFO:tasks.workunit.client.1.smithi078.stdout:9/766: truncate dc/d55/f52 1135379 0 2024-04-26T04:53:04.845 INFO:tasks.workunit.client.0.smithi077.stdout:9/425: mknod d9/dd/d3d/d7c/c97 0 2024-04-26T04:53:04.849 INFO:tasks.workunit.client.1.smithi078.stdout:4/829: rename d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/ff4 to d0/de/f126 0 2024-04-26T04:53:04.849 INFO:tasks.workunit.client.0.smithi077.stdout:8/444: rmdir d4/d69/d73 39 2024-04-26T04:53:04.850 INFO:tasks.workunit.client.1.smithi078.stdout:4/830: chown d0/de/fad 35825768 1 2024-04-26T04:53:04.854 INFO:tasks.workunit.client.1.smithi078.stdout:3/837: write d16/d26/d2e/d5f/d49/f4b [4341670,27760] 0 2024-04-26T04:53:04.856 INFO:tasks.workunit.client.1.smithi078.stdout:1/886: unlink d7/d1e/d9e/dd/d20/f50 0 2024-04-26T04:53:04.858 INFO:tasks.workunit.client.1.smithi078.stdout:6/895: unlink d2/d1a/d38/dd5/ddf/f112 0 2024-04-26T04:53:04.859 INFO:tasks.workunit.client.0.smithi077.stdout:2/426: mkdir d9/dc/d5f/d8e 0 2024-04-26T04:53:04.859 INFO:tasks.workunit.client.0.smithi077.stdout:2/427: chown d9/dc/d5f/d8e 1654 1 2024-04-26T04:53:04.860 INFO:tasks.workunit.client.0.smithi077.stdout:4/420: creat d4/de/d15/d1f/d35/f90 x:0 0 0 2024-04-26T04:53:04.860 INFO:tasks.workunit.client.1.smithi078.stdout:3/838: dwrite d16/d26/f32 [0,4194304] 0 2024-04-26T04:53:04.861 INFO:tasks.workunit.client.0.smithi077.stdout:9/426: chown d9/dd/c2a 684 1 2024-04-26T04:53:04.861 INFO:tasks.workunit.client.0.smithi077.stdout:4/421: dread - d4/de/d15/d1f/d35/f90 zero size 2024-04-26T04:53:04.863 INFO:tasks.workunit.client.1.smithi078.stdout:1/887: dwrite d7/d1e/d9e/db/d49/f61 [4194304,4194304] 0 2024-04-26T04:53:04.865 INFO:tasks.workunit.client.0.smithi077.stdout:6/493: creat d8/d52/d53/d5e/da6/faa x:0 0 0 2024-04-26T04:53:04.866 INFO:tasks.workunit.client.1.smithi078.stdout:1/888: chown d7/d1e/d9e/db/d49/dba/d87/cd2 0 1 2024-04-26T04:53:04.868 INFO:tasks.workunit.client.1.smithi078.stdout:8/827: rename d0/d19/d52/d5c/da2/dd3/cdc to d0/d7a/d83/c10b 0 2024-04-26T04:53:04.872 INFO:tasks.workunit.client.1.smithi078.stdout:1/889: dwrite d7/d1e/fe2 [0,4194304] 0 2024-04-26T04:53:04.875 INFO:tasks.workunit.client.1.smithi078.stdout:5/790: link d7/d48/ca6 d7/d18/d1c/d8e/c10a 0 2024-04-26T04:53:04.876 INFO:tasks.workunit.client.0.smithi077.stdout:6/494: dwrite d8/d2a/f32 [0,4194304] 0 2024-04-26T04:53:04.876 INFO:tasks.workunit.client.1.smithi078.stdout:5/791: stat d7/d18/d1c/f95 0 2024-04-26T04:53:04.885 INFO:tasks.workunit.client.1.smithi078.stdout:4/831: dread d0/d1d/de1/d20/d31/f73 [0,4194304] 0 2024-04-26T04:53:04.886 INFO:tasks.workunit.client.0.smithi077.stdout:8/445: dread d4/df/d22/d2d/f1c [0,4194304] 0 2024-04-26T04:53:04.891 INFO:tasks.workunit.client.1.smithi078.stdout:3/839: stat d16/d26/d2e/d5f/d49/dad/c103 0 2024-04-26T04:53:04.892 INFO:tasks.workunit.client.0.smithi077.stdout:8/446: read d4/df/d22/d2d/f4a [164932,85157] 0 2024-04-26T04:53:04.893 INFO:tasks.workunit.client.1.smithi078.stdout:3/840: dread - d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/f120 zero size 2024-04-26T04:53:04.894 INFO:tasks.workunit.client.1.smithi078.stdout:9/767: symlink dc/df/d4b/d22/d2a/d92/ddb/le5 0 2024-04-26T04:53:04.896 INFO:tasks.workunit.client.0.smithi077.stdout:8/447: read d4/d17/d35/d5b/f63 [2360872,35310] 0 2024-04-26T04:53:04.897 INFO:tasks.workunit.client.0.smithi077.stdout:8/448: readlink d4/d17/l28 0 2024-04-26T04:53:04.897 INFO:tasks.workunit.client.1.smithi078.stdout:0/840: getdents d2/d1e/d5a 0 2024-04-26T04:53:04.902 INFO:tasks.workunit.client.0.smithi077.stdout:4/422: dread d4/de/d15/d1f/f32 [0,4194304] 0 2024-04-26T04:53:04.905 INFO:tasks.workunit.client.0.smithi077.stdout:9/427: link d9/dd/d1b/d2b/d32/l86 d9/dd/d1b/d2b/d32/l98 0 2024-04-26T04:53:04.906 INFO:tasks.workunit.client.0.smithi077.stdout:9/428: dread - d9/dd/d1b/d2b/f83 zero size 2024-04-26T04:53:04.907 INFO:tasks.workunit.client.1.smithi078.stdout:5/792: symlink d7/da/d1e/d91/l10b 0 2024-04-26T04:53:04.908 INFO:tasks.workunit.client.0.smithi077.stdout:4/423: unlink d4/d17/d18/l1a 0 2024-04-26T04:53:04.908 INFO:tasks.workunit.client.1.smithi078.stdout:2/831: getdents d8/d11/d14/d69/d8f/d34/dd6 0 2024-04-26T04:53:04.909 INFO:tasks.workunit.client.1.smithi078.stdout:2/832: dread - d8/d53/f11a zero size 2024-04-26T04:53:04.909 INFO:tasks.workunit.client.0.smithi077.stdout:4/424: stat d4/d17/d18/d43/d53/d64/l70 0 2024-04-26T04:53:04.910 INFO:tasks.workunit.client.1.smithi078.stdout:2/833: fdatasync d8/d11/d14/d69/d8f/d34/d70/f103 0 2024-04-26T04:53:04.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/449: unlink d4/df/d22/d2d/d2b/l31 0 2024-04-26T04:53:04.911 INFO:tasks.workunit.client.1.smithi078.stdout:2/834: write d8/d11/d18/d5d/d61/f1b [3462234,3906] 0 2024-04-26T04:53:04.911 INFO:tasks.workunit.client.1.smithi078.stdout:2/835: fsync d8/f1f 0 2024-04-26T04:53:04.913 INFO:tasks.workunit.client.0.smithi077.stdout:9/429: dwrite d9/f8b [0,4194304] 0 2024-04-26T04:53:04.914 INFO:tasks.workunit.client.1.smithi078.stdout:7/789: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d68 0 2024-04-26T04:53:04.915 INFO:tasks.workunit.client.0.smithi077.stdout:6/495: write d8/d1c/f3e [292086,97198] 0 2024-04-26T04:53:04.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/425: dwrite d4/d17/d18/d43/d53/d56/f2d [4194304,4194304] 0 2024-04-26T04:53:04.921 INFO:tasks.workunit.client.1.smithi078.stdout:3/841: mkdir d16/d26/d2b/d31/d78/dd1/d123 0 2024-04-26T04:53:04.925 INFO:tasks.workunit.client.0.smithi077.stdout:4/426: dread d4/de/f44 [0,4194304] 0 2024-04-26T04:53:04.928 INFO:tasks.workunit.client.0.smithi077.stdout:9/430: mkdir d9/d35/d99 0 2024-04-26T04:53:04.929 INFO:tasks.workunit.client.0.smithi077.stdout:9/431: truncate d9/f46 989323 0 2024-04-26T04:53:04.929 INFO:tasks.workunit.client.1.smithi078.stdout:8/828: mknod d0/d19/d52/d35/d44/df7/c10c 0 2024-04-26T04:53:04.930 INFO:tasks.workunit.client.1.smithi078.stdout:8/829: write d0/d19/d52/d35/f105 [996970,124468] 0 2024-04-26T04:53:04.931 INFO:tasks.workunit.client.0.smithi077.stdout:6/496: creat d8/d52/d53/d5e/fab x:0 0 0 2024-04-26T04:53:04.935 INFO:tasks.workunit.client.0.smithi077.stdout:8/450: write d4/d17/d35/f52 [2352488,118718] 0 2024-04-26T04:53:04.936 INFO:tasks.workunit.client.1.smithi078.stdout:1/890: getdents d7/d1e/d9e/db/d49/dba/d87/d119 0 2024-04-26T04:53:04.937 INFO:tasks.workunit.client.1.smithi078.stdout:2/836: dread d8/d11/d14/d69/d8f/d34/dd3/de4/f93 [0,4194304] 0 2024-04-26T04:53:04.940 INFO:tasks.workunit.client.0.smithi077.stdout:4/427: creat d4/d4e/f91 x:0 0 0 2024-04-26T04:53:04.942 INFO:tasks.workunit.client.0.smithi077.stdout:9/432: rename d9/dd/d1b/d2b/d32/f8e to d9/d3a/d79/d50/f9a 0 2024-04-26T04:53:04.944 INFO:tasks.workunit.client.0.smithi077.stdout:6/497: mknod d8/d55/cac 0 2024-04-26T04:53:04.946 INFO:tasks.workunit.client.1.smithi078.stdout:6/896: creat d2/de/d27/da9/d2a/d4d/f120 x:0 0 0 2024-04-26T04:53:04.950 INFO:tasks.workunit.client.1.smithi078.stdout:7/790: creat d9/da/d20/f106 x:0 0 0 2024-04-26T04:53:04.952 INFO:tasks.workunit.client.0.smithi077.stdout:8/451: fdatasync d4/f5 0 2024-04-26T04:53:04.952 INFO:tasks.workunit.client.1.smithi078.stdout:7/791: fsync d9/da/d33/d4b/d4c/d5b/d83/fac 0 2024-04-26T04:53:04.953 INFO:tasks.workunit.client.0.smithi077.stdout:8/452: chown d4/df/f15 0 1 2024-04-26T04:53:04.953 INFO:tasks.workunit.client.1.smithi078.stdout:5/793: dwrite d7/da/d8a/fb7 [0,4194304] 0 2024-04-26T04:53:04.954 INFO:tasks.workunit.client.1.smithi078.stdout:7/792: stat d9/da/d33/d4b/d4c/d5b/f5c 0 2024-04-26T04:53:04.957 INFO:tasks.workunit.client.0.smithi077.stdout:9/433: symlink d9/d3a/l9b 0 2024-04-26T04:53:04.958 INFO:tasks.workunit.client.0.smithi077.stdout:6/498: mkdir d8/d52/da0/dad 0 2024-04-26T04:53:04.964 INFO:tasks.workunit.client.1.smithi078.stdout:4/832: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab/l127 0 2024-04-26T04:53:04.964 INFO:tasks.workunit.client.0.smithi077.stdout:8/453: rename d4/f76 to d4/d17/f9a 0 2024-04-26T04:53:04.965 INFO:tasks.workunit.client.0.smithi077.stdout:6/499: unlink d8/f37 0 2024-04-26T04:53:04.970 INFO:tasks.workunit.client.1.smithi078.stdout:9/768: creat dc/fe6 x:0 0 0 2024-04-26T04:53:04.971 INFO:tasks.workunit.client.1.smithi078.stdout:9/769: chown dc/df/d17/c7a 390095 1 2024-04-26T04:53:04.972 INFO:tasks.workunit.client.1.smithi078.stdout:5/794: symlink d7/d18/d1c/d8e/l10c 0 2024-04-26T04:53:04.974 INFO:tasks.workunit.client.1.smithi078.stdout:7/793: mknod d9/da/d11/d13/d1a/d65/c107 0 2024-04-26T04:53:04.975 INFO:tasks.workunit.client.1.smithi078.stdout:7/794: fdatasync d9/da/d33/d4b/d4c/d6c/f81 0 2024-04-26T04:53:04.979 INFO:tasks.workunit.client.0.smithi077.stdout:9/434: dread d9/dd/d3d/f47 [0,4194304] 0 2024-04-26T04:53:04.980 INFO:tasks.workunit.client.1.smithi078.stdout:0/841: rename d2/d9/d18/d7d/fcf to d2/d9/d18/f117 0 2024-04-26T04:53:04.983 INFO:tasks.workunit.client.1.smithi078.stdout:0/842: dread d2/d9/f3e [4194304,4194304] 0 2024-04-26T04:53:04.985 INFO:tasks.workunit.client.1.smithi078.stdout:6/897: mknod d2/de/d27/da9/da/c121 0 2024-04-26T04:53:04.987 INFO:tasks.workunit.client.1.smithi078.stdout:9/770: mknod dc/df/d4b/dae/ce7 0 2024-04-26T04:53:04.988 INFO:tasks.workunit.client.1.smithi078.stdout:1/891: dwrite d7/d1e/d9e/db/d37/f88 [0,4194304] 0 2024-04-26T04:53:04.992 INFO:tasks.workunit.client.1.smithi078.stdout:8/830: truncate d0/d19/d52/d35/d44/d6b/d104/f7f 3486600 0 2024-04-26T04:53:04.997 INFO:tasks.workunit.client.1.smithi078.stdout:4/833: rename d0/de/d22/d27/d7b/l106 to d0/d1d/de1/d20/d31/d47/db2/d44/l128 0 2024-04-26T04:53:04.998 INFO:tasks.workunit.client.1.smithi078.stdout:0/843: dread d2/d9/f54 [0,4194304] 0 2024-04-26T04:53:05.002 INFO:tasks.workunit.client.1.smithi078.stdout:2/837: creat d8/d11/d14/d69/d8f/d34/f126 x:0 0 0 2024-04-26T04:53:05.006 INFO:tasks.workunit.client.1.smithi078.stdout:6/898: creat d2/d1a/d38/dd5/ddf/f122 x:0 0 0 2024-04-26T04:53:05.006 INFO:tasks.workunit.client.1.smithi078.stdout:5/795: fsync d7/da/d8a/fd0 0 2024-04-26T04:53:05.007 INFO:tasks.workunit.client.1.smithi078.stdout:3/842: link d16/d26/d2e/d74/l7a d16/d26/d2b/d31/l124 0 2024-04-26T04:53:05.010 INFO:tasks.workunit.client.1.smithi078.stdout:8/831: creat d0/d19/d52/d35/d44/df7/f10d x:0 0 0 2024-04-26T04:53:05.015 INFO:tasks.workunit.client.0.smithi077.stdout:8/454: creat d4/d87/f9b x:0 0 0 2024-04-26T04:53:05.018 INFO:tasks.workunit.client.1.smithi078.stdout:2/838: write d8/f4e [5917562,63088] 0 2024-04-26T04:53:05.019 INFO:tasks.workunit.client.1.smithi078.stdout:2/839: dread d8/d11/d14/f86 [0,4194304] 0 2024-04-26T04:53:05.021 INFO:tasks.workunit.client.0.smithi077.stdout:4/428: rename d4/d17/d18/d43/d53/d56/l21 to d4/de/d15/d1f/l92 0 2024-04-26T04:53:05.022 INFO:tasks.workunit.client.1.smithi078.stdout:6/899: dread - d2/de/d27/da9/da/d36/db9/d97/d105/f11b zero size 2024-04-26T04:53:05.022 INFO:tasks.workunit.client.0.smithi077.stdout:8/455: truncate d4/df/f1f 4778084 0 2024-04-26T04:53:05.023 INFO:tasks.workunit.client.0.smithi077.stdout:8/456: write d4/fe [2685673,51825] 0 2024-04-26T04:53:05.024 INFO:tasks.workunit.client.0.smithi077.stdout:9/435: creat d9/dd/f9c x:0 0 0 2024-04-26T04:53:05.025 INFO:tasks.workunit.client.0.smithi077.stdout:9/436: write d9/d45/f71 [556566,130940] 0 2024-04-26T04:53:05.027 INFO:tasks.workunit.client.1.smithi078.stdout:5/796: fsync d7/da/d1e/d91/f102 0 2024-04-26T04:53:05.033 INFO:tasks.workunit.client.0.smithi077.stdout:7/470: sync 2024-04-26T04:53:05.033 INFO:tasks.workunit.client.0.smithi077.stdout:3/329: sync 2024-04-26T04:53:05.035 INFO:tasks.workunit.client.0.smithi077.stdout:4/429: rename d4/f6b to d4/de/f93 0 2024-04-26T04:53:05.035 INFO:tasks.workunit.client.1.smithi078.stdout:3/843: creat d16/d9b/f125 x:0 0 0 2024-04-26T04:53:05.040 INFO:tasks.workunit.client.0.smithi077.stdout:6/500: dwrite d8/d1c/f64 [0,4194304] 0 2024-04-26T04:53:05.041 INFO:tasks.workunit.client.1.smithi078.stdout:4/834: link d0/de/d22/d27/d7b/f117 d0/d1d/de1/d20/d31/d47/db2/d4c/dba/f129 0 2024-04-26T04:53:05.045 INFO:tasks.workunit.client.1.smithi078.stdout:4/835: dread d0/d1d/db4/fd3 [0,4194304] 0 2024-04-26T04:53:05.046 INFO:tasks.workunit.client.1.smithi078.stdout:4/836: read d0/d1d/de1/dd8/f10d [3587259,1763] 0 2024-04-26T04:53:05.046 INFO:tasks.workunit.client.1.smithi078.stdout:9/771: getdents dc/df/d4b/d22/d2d 0 2024-04-26T04:53:05.048 INFO:tasks.workunit.client.0.smithi077.stdout:9/437: dwrite d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:05.048 INFO:tasks.workunit.client.0.smithi077.stdout:4/430: symlink d4/de/l94 0 2024-04-26T04:53:05.050 INFO:tasks.workunit.client.0.smithi077.stdout:9/438: dread - d9/d3a/d79/d50/f9a zero size 2024-04-26T04:53:05.053 INFO:tasks.workunit.client.1.smithi078.stdout:0/844: truncate d2/d9/d31/d4a/f7a 1575518 0 2024-04-26T04:53:05.054 INFO:tasks.workunit.client.0.smithi077.stdout:4/431: dwrite d4/f14 [0,4194304] 0 2024-04-26T04:53:05.055 INFO:tasks.workunit.client.1.smithi078.stdout:8/832: dwrite d0/d19/d52/d35/d44/d6b/fd5 [0,4194304] 0 2024-04-26T04:53:05.057 INFO:tasks.workunit.client.0.smithi077.stdout:3/330: mknod d0/d7/d1f/c68 0 2024-04-26T04:53:05.058 INFO:tasks.workunit.client.1.smithi078.stdout:6/900: dwrite d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:53:05.061 INFO:tasks.workunit.client.0.smithi077.stdout:9/439: write d9/d3a/d79/f33 [12320247,63297] 0 2024-04-26T04:53:05.063 INFO:tasks.workunit.client.0.smithi077.stdout:7/471: creat da/d13/fa1 x:0 0 0 2024-04-26T04:53:05.065 INFO:tasks.workunit.client.0.smithi077.stdout:3/331: rmdir d0/d9 39 2024-04-26T04:53:05.065 INFO:tasks.workunit.client.0.smithi077.stdout:4/432: dwrite d4/d17/f3d [0,4194304] 0 2024-04-26T04:53:05.068 INFO:tasks.workunit.client.1.smithi078.stdout:4/837: mknod d0/d1d/c12a 0 2024-04-26T04:53:05.068 INFO:tasks.workunit.client.1.smithi078.stdout:2/840: mkdir d8/d11/d18/d5d/d127 0 2024-04-26T04:53:05.072 INFO:tasks.workunit.client.0.smithi077.stdout:8/457: link d4/df/d18/d56/f72 d4/df/d22/d2d/d2b/f9c 0 2024-04-26T04:53:05.073 INFO:tasks.workunit.client.0.smithi077.stdout:9/440: unlink d9/d45/l4d 0 2024-04-26T04:53:05.074 INFO:tasks.workunit.client.0.smithi077.stdout:7/472: dwrite da/d13/f6d [0,4194304] 0 2024-04-26T04:53:05.075 INFO:tasks.workunit.client.0.smithi077.stdout:9/441: chown d9/f8c 2333160 1 2024-04-26T04:53:05.075 INFO:tasks.workunit.client.1.smithi078.stdout:1/892: getdents d7/d1e/d9e/db/d37 0 2024-04-26T04:53:05.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/442: write d9/f26 [4138926,81533] 0 2024-04-26T04:53:05.077 INFO:tasks.workunit.client.0.smithi077.stdout:8/458: read d4/fd [3686348,66030] 0 2024-04-26T04:53:05.077 INFO:tasks.workunit.client.0.smithi077.stdout:3/332: rename d0/d7/d15/c5a to d0/d7/d1f/d4d/c69 0 2024-04-26T04:53:05.078 INFO:tasks.workunit.client.0.smithi077.stdout:9/443: dread - d9/d3a/f89 zero size 2024-04-26T04:53:05.081 INFO:tasks.workunit.client.1.smithi078.stdout:0/845: mknod d2/d9/d31/c118 0 2024-04-26T04:53:05.083 INFO:tasks.workunit.client.0.smithi077.stdout:7/473: mknod da/d13/d22/d34/ca2 0 2024-04-26T04:53:05.083 INFO:tasks.workunit.client.1.smithi078.stdout:8/833: mkdir d0/d19/d52/d35/d44/d6b/d104/d58/d5b/d10e 0 2024-04-26T04:53:05.085 INFO:tasks.workunit.client.1.smithi078.stdout:6/901: rmdir d2/de/d27/da9/d11 39 2024-04-26T04:53:05.087 INFO:tasks.workunit.client.0.smithi077.stdout:7/474: dread da/d13/f6d [0,4194304] 0 2024-04-26T04:53:05.089 INFO:tasks.workunit.client.0.smithi077.stdout:8/459: rename d4/df/d22/d2d/l97 to d4/df/d22/d75/l9d 0 2024-04-26T04:53:05.091 INFO:tasks.workunit.client.1.smithi078.stdout:2/841: creat d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/d91/f128 x:0 0 0 2024-04-26T04:53:05.095 INFO:tasks.workunit.client.1.smithi078.stdout:5/797: rmdir d7/d48/d57/d65/d7a/d9c/d104/df9 0 2024-04-26T04:53:05.097 INFO:tasks.workunit.client.1.smithi078.stdout:7/795: sync 2024-04-26T04:53:05.097 INFO:tasks.workunit.client.1.smithi078.stdout:8/834: chown d0/d19/d25/d32/c65 51559507 1 2024-04-26T04:53:05.102 INFO:tasks.workunit.client.0.smithi077.stdout:9/444: link d9/dd/l65 d9/d3a/d79/d95/l9d 0 2024-04-26T04:53:05.103 INFO:tasks.workunit.client.1.smithi078.stdout:6/902: symlink d2/d1a/d38/dd5/d116/l123 0 2024-04-26T04:53:05.104 INFO:tasks.workunit.client.0.smithi077.stdout:8/460: creat d4/df/d22/f9e x:0 0 0 2024-04-26T04:53:05.105 INFO:tasks.workunit.client.1.smithi078.stdout:3/844: dread d16/f36 [0,4194304] 0 2024-04-26T04:53:05.106 INFO:tasks.workunit.client.1.smithi078.stdout:3/845: chown d16/d26/d2e/d5f/d49/d77/ff3 11 1 2024-04-26T04:53:05.107 INFO:tasks.workunit.client.1.smithi078.stdout:3/846: write d16/f57 [3501518,24572] 0 2024-04-26T04:53:05.113 INFO:tasks.workunit.client.1.smithi078.stdout:2/842: symlink d8/d11/d14/d69/d8f/d34/dd3/dd9/dfc/l129 0 2024-04-26T04:53:05.120 INFO:tasks.workunit.client.1.smithi078.stdout:4/838: link d0/d1d/db4/f11c d0/d1d/de1/d20/f12b 0 2024-04-26T04:53:05.121 INFO:tasks.workunit.client.1.smithi078.stdout:8/835: dread d0/d19/d52/f24 [0,4194304] 0 2024-04-26T04:53:05.121 INFO:tasks.workunit.client.1.smithi078.stdout:2/843: rmdir d8/d11/d18/d5d 39 2024-04-26T04:53:05.122 INFO:tasks.workunit.client.1.smithi078.stdout:9/772: dwrite dc/df/f28 [0,4194304] 0 2024-04-26T04:53:05.124 INFO:tasks.workunit.client.1.smithi078.stdout:5/798: dread d7/da/f62 [0,4194304] 0 2024-04-26T04:53:05.126 INFO:tasks.workunit.client.1.smithi078.stdout:1/893: mknod d7/d1e/d9e/dd/d20/d53/c11e 0 2024-04-26T04:53:05.128 INFO:tasks.workunit.client.0.smithi077.stdout:9/445: dread d9/f17 [0,4194304] 0 2024-04-26T04:53:05.128 INFO:tasks.workunit.client.0.smithi077.stdout:8/461: dread d4/df/f15 [0,4194304] 0 2024-04-26T04:53:05.131 INFO:tasks.workunit.client.0.smithi077.stdout:9/446: readlink d9/dd/l7a 0 2024-04-26T04:53:05.131 INFO:tasks.workunit.client.0.smithi077.stdout:8/462: write d4/df/d6f/f71 [3295527,35280] 0 2024-04-26T04:53:05.131 INFO:tasks.workunit.client.1.smithi078.stdout:6/903: link d2/de/d27/da9/da/fb8 d2/d1a/d38/dd5/ddf/f124 0 2024-04-26T04:53:05.136 INFO:tasks.workunit.client.0.smithi077.stdout:9/447: creat d9/dd/d3d/f9e x:0 0 0 2024-04-26T04:53:05.140 INFO:tasks.workunit.client.0.smithi077.stdout:8/463: rename d4/df/d22/d2d/d2b/d33/c82 to d4/df/d6f/c9f 0 2024-04-26T04:53:05.146 INFO:tasks.workunit.client.0.smithi077.stdout:9/448: mkdir d9/d35/d99/d9f 0 2024-04-26T04:53:05.146 INFO:tasks.workunit.client.0.smithi077.stdout:7/475: write da/d15/d42/f84 [1041190,3495] 0 2024-04-26T04:53:05.147 INFO:tasks.workunit.client.0.smithi077.stdout:8/464: dwrite d4/d87/f9b [0,4194304] 0 2024-04-26T04:53:05.147 INFO:tasks.workunit.client.0.smithi077.stdout:7/476: readlink da/l4c 0 2024-04-26T04:53:05.149 INFO:tasks.workunit.client.0.smithi077.stdout:7/477: write da/d15/f2f [1903628,8186] 0 2024-04-26T04:53:05.149 INFO:tasks.workunit.client.1.smithi078.stdout:0/846: write d2/d1e/f2a [1789880,90701] 0 2024-04-26T04:53:05.151 INFO:tasks.workunit.client.0.smithi077.stdout:7/478: chown da/d13/d22/d5d/f66 233708975 1 2024-04-26T04:53:05.156 INFO:tasks.workunit.client.1.smithi078.stdout:2/844: mkdir d8/d11/d14/d12a 0 2024-04-26T04:53:05.156 INFO:tasks.workunit.client.0.smithi077.stdout:3/333: dwrite d0/d9/f26 [0,4194304] 0 2024-04-26T04:53:05.158 INFO:tasks.workunit.client.0.smithi077.stdout:8/465: creat d4/d17/d35/d5b/fa0 x:0 0 0 2024-04-26T04:53:05.158 INFO:tasks.workunit.client.1.smithi078.stdout:7/796: write d9/da/d11/d13/d1a/d52/d5a/d41/d68/faa [466353,104527] 0 2024-04-26T04:53:05.163 INFO:tasks.workunit.client.0.smithi077.stdout:3/334: dread d0/d7/d15/f44 [0,4194304] 0 2024-04-26T04:53:05.164 INFO:tasks.workunit.client.1.smithi078.stdout:7/797: dwrite d9/da/d33/d4b/d4c/d5b/ff0 [0,4194304] 0 2024-04-26T04:53:05.165 INFO:tasks.workunit.client.1.smithi078.stdout:5/799: creat d7/d48/d57/d65/d46/dce/d109/f10d x:0 0 0 2024-04-26T04:53:05.168 INFO:tasks.workunit.client.1.smithi078.stdout:3/847: getdents d16/d26/d2e/d5f 0 2024-04-26T04:53:05.169 INFO:tasks.workunit.client.1.smithi078.stdout:7/798: read d9/fe [3146732,74130] 0 2024-04-26T04:53:05.171 INFO:tasks.workunit.client.1.smithi078.stdout:7/799: readlink d9/da/d11/d13/d1a/d52/d5a/d28/dab/lcd 0 2024-04-26T04:53:05.172 INFO:tasks.workunit.client.0.smithi077.stdout:3/335: dread d0/d7/d1f/d4d/f5f [0,4194304] 0 2024-04-26T04:53:05.173 INFO:tasks.workunit.client.1.smithi078.stdout:9/773: write dc/df/d4b/d22/d2a/d92/fad [5872900,89042] 0 2024-04-26T04:53:05.174 INFO:tasks.workunit.client.1.smithi078.stdout:2/845: symlink d8/d11/d6e/l12b 0 2024-04-26T04:53:05.175 INFO:tasks.workunit.client.1.smithi078.stdout:2/846: stat d8/d11/d6e/cb3 0 2024-04-26T04:53:05.176 INFO:tasks.workunit.client.0.smithi077.stdout:9/449: getdents d9/dd/d3d/d7c 0 2024-04-26T04:53:05.176 INFO:tasks.workunit.client.0.smithi077.stdout:9/450: stat d9/dd/d3d/d7c/c97 0 2024-04-26T04:53:05.178 INFO:tasks.workunit.client.1.smithi078.stdout:8/836: write d0/d19/d52/d35/d44/d6b/d104/f7f [2335278,110949] 0 2024-04-26T04:53:05.180 INFO:tasks.workunit.client.1.smithi078.stdout:6/904: dwrite d2/de/d27/da9/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:53:05.184 INFO:tasks.workunit.client.0.smithi077.stdout:9/451: dread d9/d3a/d79/f6b [0,4194304] 0 2024-04-26T04:53:05.213 INFO:tasks.workunit.client.1.smithi078.stdout:4/839: sync 2024-04-26T04:53:05.213 INFO:tasks.workunit.client.0.smithi077.stdout:5/446: sync 2024-04-26T04:53:05.215 INFO:tasks.workunit.client.0.smithi077.stdout:3/336: fdatasync d0/d7/d1f/f4a 0 2024-04-26T04:53:05.216 INFO:tasks.workunit.client.1.smithi078.stdout:4/840: dread d0/d1d/de1/d20/d31/d47/db2/fa3 [0,4194304] 0 2024-04-26T04:53:05.217 INFO:tasks.workunit.client.1.smithi078.stdout:1/894: creat d7/d1e/d9e/dd/d20/f11f x:0 0 0 2024-04-26T04:53:05.221 INFO:tasks.workunit.client.0.smithi077.stdout:9/452: rename d9/dd/d1b/d2b/d32/d36/c90 to d9/dd/d1b/d2b/ca0 0 2024-04-26T04:53:05.222 INFO:tasks.workunit.client.1.smithi078.stdout:0/847: dwrite d2/d9/d78/d84/f88 [0,4194304] 0 2024-04-26T04:53:05.222 INFO:tasks.workunit.client.1.smithi078.stdout:0/848: chown d2/d9/d18/d7d/da9 122909 1 2024-04-26T04:53:05.224 INFO:tasks.workunit.client.1.smithi078.stdout:9/774: creat dc/df/d27/d81/da8/fe8 x:0 0 0 2024-04-26T04:53:05.225 INFO:tasks.workunit.client.1.smithi078.stdout:2/847: creat d8/db7/f12c x:0 0 0 2024-04-26T04:53:05.226 INFO:tasks.workunit.client.1.smithi078.stdout:2/848: chown d8/d11/d14/d69/d8f/d34/dd3/de4/d52 481222 1 2024-04-26T04:53:05.226 INFO:tasks.workunit.client.0.smithi077.stdout:8/466: getdents d4/df/d22/d2d/d2b/d33 0 2024-04-26T04:53:05.230 INFO:tasks.workunit.client.1.smithi078.stdout:5/800: mkdir d7/d48/da9/deb/dfe/d10e 0 2024-04-26T04:53:05.231 INFO:tasks.workunit.client.1.smithi078.stdout:5/801: write d7/d48/d57/d65/d46/de7/fd8 [344437,127812] 0 2024-04-26T04:53:05.235 INFO:tasks.workunit.client.0.smithi077.stdout:9/453: dwrite d9/f62 [0,4194304] 0 2024-04-26T04:53:05.239 INFO:tasks.workunit.client.0.smithi077.stdout:8/467: creat d4/df/d22/d80/d86/fa1 x:0 0 0 2024-04-26T04:53:05.240 INFO:tasks.workunit.client.0.smithi077.stdout:8/468: write d4/df/d6f/f71 [2717536,12882] 0 2024-04-26T04:53:05.241 INFO:tasks.workunit.client.0.smithi077.stdout:8/469: dread - d4/d17/f3c zero size 2024-04-26T04:53:05.242 INFO:tasks.workunit.client.0.smithi077.stdout:8/470: write d4/fe [2141496,18486] 0 2024-04-26T04:53:05.242 INFO:tasks.workunit.client.0.smithi077.stdout:5/447: creat d1/fa5 x:0 0 0 2024-04-26T04:53:05.243 INFO:tasks.workunit.client.1.smithi078.stdout:4/841: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/fa6 [0,4194304] 0 2024-04-26T04:53:05.246 INFO:tasks.workunit.client.0.smithi077.stdout:3/337: link d0/d7/f33 d0/d9/d32/d29/f6a 0 2024-04-26T04:53:05.246 INFO:tasks.workunit.client.1.smithi078.stdout:4/842: truncate d0/de/d22/d27/d56/fe3 478263 0 2024-04-26T04:53:05.247 INFO:tasks.workunit.client.1.smithi078.stdout:0/849: read d2/d9/d31/d4a/f7a [648379,4110] 0 2024-04-26T04:53:05.247 INFO:tasks.workunit.client.1.smithi078.stdout:4/843: chown d0/d1d/de1/d20/d31/d47/db2/d44/d55/f7f 47 1 2024-04-26T04:53:05.248 INFO:tasks.workunit.client.0.smithi077.stdout:9/454: chown d9/dd/d1b/d2b/c64 34669488 1 2024-04-26T04:53:05.248 INFO:tasks.workunit.client.1.smithi078.stdout:9/775: symlink dc/df/d27/le9 0 2024-04-26T04:53:05.249 INFO:tasks.workunit.client.1.smithi078.stdout:4/844: dread d0/de/d22/d27/ffa [0,4194304] 0 2024-04-26T04:53:05.250 INFO:tasks.workunit.client.1.smithi078.stdout:8/837: mkdir d0/d19/d52/d35/d44/d6b/d104/d58/d10f 0 2024-04-26T04:53:05.253 INFO:tasks.workunit.client.1.smithi078.stdout:2/849: unlink d8/d11/d18/dbb/f107 0 2024-04-26T04:53:05.253 INFO:tasks.workunit.client.1.smithi078.stdout:0/850: dwrite d2/d9/d18/db7/fd1 [0,4194304] 0 2024-04-26T04:53:05.253 INFO:tasks.workunit.client.1.smithi078.stdout:3/848: truncate d16/d26/d2b/dd6/fe3 1778225 0 2024-04-26T04:53:05.254 INFO:tasks.workunit.client.1.smithi078.stdout:3/849: readlink d16/d26/d2b/lae 0 2024-04-26T04:53:05.254 INFO:tasks.workunit.client.0.smithi077.stdout:8/471: stat d4/d69/d73/c78 0 2024-04-26T04:53:05.255 INFO:tasks.workunit.client.1.smithi078.stdout:3/850: chown d16/d26/d2b/l10c 500817190 1 2024-04-26T04:53:05.256 INFO:tasks.workunit.client.0.smithi077.stdout:8/472: read d4/d17/f6a [61399,10938] 0 2024-04-26T04:53:05.256 INFO:tasks.workunit.client.1.smithi078.stdout:0/851: truncate d2/d9/d78/deb/f10d 147335 0 2024-04-26T04:53:05.256 INFO:tasks.workunit.client.0.smithi077.stdout:8/473: dread - d4/d17/d35/d5b/f8b zero size 2024-04-26T04:53:05.258 INFO:tasks.workunit.client.0.smithi077.stdout:5/448: mkdir d1/d8/d72/da6 0 2024-04-26T04:53:05.258 INFO:tasks.workunit.client.0.smithi077.stdout:8/474: read - d4/df/d22/d2d/f30 zero size 2024-04-26T04:53:05.260 INFO:tasks.workunit.client.0.smithi077.stdout:3/338: read d0/f1b [664373,54964] 0 2024-04-26T04:53:05.260 INFO:tasks.workunit.client.0.smithi077.stdout:8/475: fdatasync d4/df/d22/d2d/f94 0 2024-04-26T04:53:05.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/476: readlink d4/df/d22/d2d/l38 0 2024-04-26T04:53:05.261 INFO:tasks.workunit.client.1.smithi078.stdout:0/852: dwrite d2/d9/d78/deb/f101 [0,4194304] 0 2024-04-26T04:53:05.262 INFO:tasks.workunit.client.0.smithi077.stdout:9/455: creat d9/d7d/fa1 x:0 0 0 2024-04-26T04:53:05.263 INFO:tasks.workunit.client.1.smithi078.stdout:5/802: mkdir d7/d48/da9/d10f 0 2024-04-26T04:53:05.263 INFO:tasks.workunit.client.1.smithi078.stdout:5/803: chown d7/da/fe8 2 1 2024-04-26T04:53:05.265 INFO:tasks.workunit.client.1.smithi078.stdout:5/804: write d7/d18/f40 [2647806,84011] 0 2024-04-26T04:53:05.266 INFO:tasks.workunit.client.1.smithi078.stdout:5/805: dread - d7/da/d1e/d91/fd4 zero size 2024-04-26T04:53:05.267 INFO:tasks.workunit.client.1.smithi078.stdout:9/776: fsync dc/f4e 0 2024-04-26T04:53:05.270 INFO:tasks.workunit.client.0.smithi077.stdout:9/456: dwrite d9/f26 [0,4194304] 0 2024-04-26T04:53:05.278 INFO:tasks.workunit.client.0.smithi077.stdout:3/339: rename d0/f67 to d0/d9/d32/d2a/f6b 0 2024-04-26T04:53:05.280 INFO:tasks.workunit.client.1.smithi078.stdout:6/905: creat d2/de/d27/da9/d2a/f125 x:0 0 0 2024-04-26T04:53:05.288 INFO:tasks.workunit.client.0.smithi077.stdout:8/477: link d4/df/d18/d56/f7c d4/d69/fa2 0 2024-04-26T04:53:05.294 INFO:tasks.workunit.client.0.smithi077.stdout:8/478: unlink d4/df/d22/d80/c85 0 2024-04-26T04:53:05.295 INFO:tasks.workunit.client.1.smithi078.stdout:7/800: dwrite d9/da/d11/d13/d50/f59 [0,4194304] 0 2024-04-26T04:53:05.296 INFO:tasks.workunit.client.1.smithi078.stdout:7/801: stat d9/da/d11/d13/d50/f7c 0 2024-04-26T04:53:05.296 INFO:tasks.workunit.client.0.smithi077.stdout:8/479: write d4/df/d22/d2d/f94 [623603,110929] 0 2024-04-26T04:53:05.297 INFO:tasks.workunit.client.1.smithi078.stdout:7/802: fsync d9/da/d11/d13/d1a/dd4/fdf 0 2024-04-26T04:53:05.297 INFO:tasks.workunit.client.1.smithi078.stdout:9/777: stat dc/df/d4b/d3e/l63 0 2024-04-26T04:53:05.298 INFO:tasks.workunit.client.0.smithi077.stdout:5/449: dwrite d1/d42/da4/f34 [0,4194304] 0 2024-04-26T04:53:05.301 INFO:tasks.workunit.client.0.smithi077.stdout:8/480: readlink d4/df/d22/d80/d86/l96 0 2024-04-26T04:53:05.301 INFO:tasks.workunit.client.0.smithi077.stdout:5/450: write d1/d12/f50 [3748290,60296] 0 2024-04-26T04:53:05.301 INFO:tasks.workunit.client.1.smithi078.stdout:6/906: creat d2/de/d27/d58/d5b/f126 x:0 0 0 2024-04-26T04:53:05.302 INFO:tasks.workunit.client.1.smithi078.stdout:3/851: mknod d16/d26/d2b/d31/d78/dd1/d123/c126 0 2024-04-26T04:53:05.303 INFO:tasks.workunit.client.0.smithi077.stdout:8/481: read - d4/df/d22/d80/f8e zero size 2024-04-26T04:53:05.304 INFO:tasks.workunit.client.0.smithi077.stdout:8/482: stat d4/df/d18 0 2024-04-26T04:53:05.304 INFO:tasks.workunit.client.1.smithi078.stdout:7/803: dwrite d9/da/d33/d4b/d4c/fe5 [0,4194304] 0 2024-04-26T04:53:05.305 INFO:tasks.workunit.client.1.smithi078.stdout:1/895: rename d7/d1e/d9e/db/d37/d58/f5e to d7/d1e/d9e/d4c/d3d/f120 0 2024-04-26T04:53:05.309 INFO:tasks.workunit.client.0.smithi077.stdout:9/457: write d9/dd/d1b/d2b/d32/f39 [787163,35193] 0 2024-04-26T04:53:05.310 INFO:tasks.workunit.client.1.smithi078.stdout:0/853: link d2/d9/d78/d84/f8e d2/d9/d60/d6a/daa/f119 0 2024-04-26T04:53:05.313 INFO:tasks.workunit.client.1.smithi078.stdout:5/806: symlink d7/d48/d57/d65/d7a/d9c/d104/d96/df0/l110 0 2024-04-26T04:53:05.315 INFO:tasks.workunit.client.1.smithi078.stdout:8/838: dwrite d0/d19/d52/d35/d44/d6b/d104/d58/feb [0,4194304] 0 2024-04-26T04:53:05.316 INFO:tasks.workunit.client.0.smithi077.stdout:5/451: mkdir d1/d8d/da7 0 2024-04-26T04:53:05.317 INFO:tasks.workunit.client.1.smithi078.stdout:9/778: creat dc/df/d8a/fea x:0 0 0 2024-04-26T04:53:05.319 INFO:tasks.workunit.client.0.smithi077.stdout:3/340: dwrite d0/d7/d15/f50 [0,4194304] 0 2024-04-26T04:53:05.319 INFO:tasks.workunit.client.0.smithi077.stdout:9/458: mknod d9/d35/d99/ca2 0 2024-04-26T04:53:05.320 INFO:tasks.workunit.client.0.smithi077.stdout:5/452: symlink d1/d8/d72/la8 0 2024-04-26T04:53:05.322 INFO:tasks.workunit.client.1.smithi078.stdout:9/779: chown dc/df/d4b/d22/d2d 1257 1 2024-04-26T04:53:05.322 INFO:tasks.workunit.client.1.smithi078.stdout:6/907: dwrite d2/de/d27/da9/da/d36/fe7 [0,4194304] 0 2024-04-26T04:53:05.326 INFO:tasks.workunit.client.1.smithi078.stdout:3/852: dwrite d16/d26/d2b/d31/f3d [0,4194304] 0 2024-04-26T04:53:05.326 INFO:tasks.workunit.client.1.smithi078.stdout:8/839: read d0/d19/d52/d35/d44/f26 [402806,84723] 0 2024-04-26T04:53:05.327 INFO:tasks.workunit.client.0.smithi077.stdout:3/341: fsync d0/d9/d32/d2a/f4e 0 2024-04-26T04:53:05.332 INFO:tasks.workunit.client.1.smithi078.stdout:5/807: write d7/d48/d57/d65/d46/dce/d109/f10d [185504,17672] 0 2024-04-26T04:53:05.332 INFO:tasks.workunit.client.0.smithi077.stdout:5/453: rename d1/d8/d72/c7b to d1/d42/d62/d6b/ca9 0 2024-04-26T04:53:05.334 INFO:tasks.workunit.client.0.smithi077.stdout:5/454: write d1/d12/f50 [4118086,67275] 0 2024-04-26T04:53:05.336 INFO:tasks.workunit.client.1.smithi078.stdout:8/840: fsync d0/d19/d52/d35/d44/d6b/d6c/fe9 0 2024-04-26T04:53:05.336 INFO:tasks.workunit.client.1.smithi078.stdout:9/780: write dc/df/d4b/d22/d2a/d92/fad [2275016,91352] 0 2024-04-26T04:53:05.336 INFO:tasks.workunit.client.1.smithi078.stdout:1/896: dwrite d7/d1e/d9e/db/d47/f73 [0,4194304] 0 2024-04-26T04:53:05.339 INFO:tasks.workunit.client.1.smithi078.stdout:9/781: chown dc/df/d4b/d22/d2d/d98/fd4 1035797856 1 2024-04-26T04:53:05.339 INFO:tasks.workunit.client.1.smithi078.stdout:4/845: write d0/d1d/de1/d20/f4f [3455216,106607] 0 2024-04-26T04:53:05.348 INFO:tasks.workunit.client.1.smithi078.stdout:2/850: getdents d8/db7 0 2024-04-26T04:53:05.348 INFO:tasks.workunit.client.1.smithi078.stdout:6/908: readlink d2/le9 0 2024-04-26T04:53:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 -- 172.21.15.77:0/832886847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fd2f0 msgr2=0x7f04940fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 --2- 172.21.15.77:0/832886847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fd2f0 0x7f04940fd700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0484004940 tx=0x7f0484009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 -- 172.21.15.77:0/832886847 shutdown_connections 2024-04-26T04:53:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 --2- 172.21.15.77:0/832886847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fdc40 0x7f04940fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 --2- 172.21.15.77:0/832886847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fd2f0 0x7f04940fd700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.351+0000 7f0499064700 1 -- 172.21.15.77:0/832886847 >> 172.21.15.77:0/832886847 conn(0x7f04940f6bb0 msgr2=0x7f04940f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 -- 172.21.15.77:0/832886847 shutdown_connections 2024-04-26T04:53:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 -- 172.21.15.77:0/832886847 wait complete. 2024-04-26T04:53:05.354 INFO:tasks.workunit.client.1.smithi078.stdout:2/851: dwrite d8/f1f [0,4194304] 0 2024-04-26T04:53:05.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 Processor -- start 2024-04-26T04:53:05.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 -- start start 2024-04-26T04:53:05.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fd2f0 0x7f049410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049410e490 con 0x7f04940fdc40 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.352+0000 7f0499064700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049410e600 con 0x7f04940fd2f0 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58318/0 (socket says 172.21.15.77:58318) 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 -- 172.21.15.77:0/202681104 learned_addr learned my addr 172.21.15.77:0/202681104 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f0492d9d700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fd2f0 0x7f049410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 -- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fd2f0 msgr2=0x7f049410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fd2f0 0x7f049410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 -- 172.21.15.77:0/202681104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484005040 con 0x7f04940fdc40 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f049259c700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0488002a50 tx=0x7f04880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0488007cc0 con 0x7f04940fdc40 2024-04-26T04:53:05.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0494113060 con 0x7f04940fdc40 2024-04-26T04:53:05.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f048800f040 con 0x7f04940fdc40 2024-04-26T04:53:05.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048800db30 con 0x7f04940fdc40 2024-04-26T04:53:05.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.353+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0494113620 con 0x7f04940fdc40 2024-04-26T04:53:05.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.354+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494101bf0 con 0x7f04940fdc40 2024-04-26T04:53:05.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.355+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f048800dc90 con 0x7f04940fdc40 2024-04-26T04:53:05.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.355+0000 7f047b7fe700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 0x7f047c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.355+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f048808a5e0 con 0x7f04940fdc40 2024-04-26T04:53:05.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.356+0000 7f0492d9d700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 0x7f047c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:05.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.357+0000 7f0492d9d700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 0x7f047c06dc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f048400b3b0 tx=0x7f0484009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:05.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.363+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0488057ca0 con 0x7f04940fdc40 2024-04-26T04:53:05.365 INFO:tasks.workunit.client.1.smithi078.stdout:0/854: mknod d2/d9/d18/d7d/da9/dab/dfd/c11a 0 2024-04-26T04:53:05.366 INFO:tasks.workunit.client.1.smithi078.stdout:5/808: mkdir d7/d11/dee/d111 0 2024-04-26T04:53:05.367 INFO:tasks.workunit.client.1.smithi078.stdout:8/841: symlink d0/d7a/d83/df3/l110 0 2024-04-26T04:53:05.370 INFO:tasks.workunit.client.1.smithi078.stdout:9/782: creat dc/df/d27/feb x:0 0 0 2024-04-26T04:53:05.371 INFO:tasks.workunit.client.1.smithi078.stdout:6/909: mknod d2/de/d27/da9/da8/c127 0 2024-04-26T04:53:05.374 INFO:tasks.workunit.client.1.smithi078.stdout:6/910: chown d2/de/d27/d58/l87 2903 1 2024-04-26T04:53:05.374 INFO:tasks.workunit.client.1.smithi078.stdout:5/809: symlink d7/d18/d32/d84/l112 0 2024-04-26T04:53:05.375 INFO:tasks.workunit.client.1.smithi078.stdout:0/855: symlink d2/d9/d60/d6a/daa/d112/l11b 0 2024-04-26T04:53:05.375 INFO:tasks.workunit.client.1.smithi078.stdout:4/846: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121/l12c 0 2024-04-26T04:53:05.377 INFO:tasks.workunit.client.1.smithi078.stdout:2/852: dread d8/d11/d18/d5d/d8b/db0/fda [0,4194304] 0 2024-04-26T04:53:05.377 INFO:tasks.workunit.client.1.smithi078.stdout:9/783: fdatasync dc/df/f59 0 2024-04-26T04:53:05.377 INFO:tasks.workunit.client.1.smithi078.stdout:1/897: dread d7/d1e/d9e/dd/d20/fb3 [0,4194304] 0 2024-04-26T04:53:05.381 INFO:tasks.workunit.client.1.smithi078.stdout:0/856: mkdir d2/d9/d31/d4a/d11c 0 2024-04-26T04:53:05.382 INFO:tasks.workunit.client.1.smithi078.stdout:4/847: chown d0/d1d/de1/d20/d31/d47/db2/d4c/dba/cd2 579387388 1 2024-04-26T04:53:05.383 INFO:tasks.workunit.client.1.smithi078.stdout:9/784: fsync dc/df/d4b/d22/d2d/d98/fb4 0 2024-04-26T04:53:05.384 INFO:tasks.workunit.client.1.smithi078.stdout:6/911: dwrite d2/de/f104 [0,4194304] 0 2024-04-26T04:53:05.387 INFO:tasks.workunit.client.1.smithi078.stdout:4/848: fsync d0/d1d/de1/d20/d31/d47/db2/d44/d55/f5d 0 2024-04-26T04:53:05.390 INFO:tasks.workunit.client.1.smithi078.stdout:5/810: rename d7/d48/da9/deb/dfe/d10e to d7/d18/d32/d37/d79/da7/d113 0 2024-04-26T04:53:05.390 INFO:tasks.workunit.client.1.smithi078.stdout:3/853: dread d16/d26/d2e/d5f/d49/f4b [0,4194304] 0 2024-04-26T04:53:05.390 INFO:tasks.workunit.client.1.smithi078.stdout:5/811: chown d7/d18/d32 11879 1 2024-04-26T04:53:05.392 INFO:tasks.workunit.client.1.smithi078.stdout:9/785: write dc/df/d4b/d22/d2a/d57/fa4 [1478963,72984] 0 2024-04-26T04:53:05.403 INFO:tasks.workunit.client.1.smithi078.stdout:1/898: link d7/l5d d7/d1e/d9e/db/d49/dba/d87/l121 0 2024-04-26T04:53:05.407 INFO:tasks.workunit.client.0.smithi077.stdout:9/459: dwrite d9/d45/f4a [0,4194304] 0 2024-04-26T04:53:05.408 INFO:tasks.workunit.client.1.smithi078.stdout:9/786: link dc/df/f28 dc/d55/fec 0 2024-04-26T04:53:05.413 INFO:tasks.workunit.client.1.smithi078.stdout:0/857: dread d2/d9/f30 [0,4194304] 0 2024-04-26T04:53:05.415 INFO:tasks.workunit.client.1.smithi078.stdout:1/899: symlink d7/d1e/d9e/dfc/l122 0 2024-04-26T04:53:05.416 INFO:tasks.workunit.client.1.smithi078.stdout:5/812: getdents d7/d48/d57/d65/d46/dce/d109 0 2024-04-26T04:53:05.416 INFO:tasks.workunit.client.1.smithi078.stdout:4/849: dread d0/de/f78 [0,4194304] 0 2024-04-26T04:53:05.420 INFO:tasks.workunit.client.0.smithi077.stdout:9/460: link d9/d45/f7e d9/d35/d99/fa3 0 2024-04-26T04:53:05.421 INFO:tasks.workunit.client.1.smithi078.stdout:5/813: dwrite d7/d18/f1a [0,4194304] 0 2024-04-26T04:53:05.423 INFO:tasks.workunit.client.0.smithi077.stdout:3/342: write d0/d9/d32/d29/f6a [269235,48120] 0 2024-04-26T04:53:05.425 INFO:tasks.workunit.client.1.smithi078.stdout:1/900: unlink d7/d1e/d9e/db/d37/d58/d5c/dc0/l27 0 2024-04-26T04:53:05.427 INFO:tasks.workunit.client.0.smithi077.stdout:9/461: symlink d9/dd/la4 0 2024-04-26T04:53:05.428 INFO:tasks.workunit.client.0.smithi077.stdout:9/462: write d9/dd/d1b/f66 [1643908,126734] 0 2024-04-26T04:53:05.432 INFO:tasks.workunit.client.0.smithi077.stdout:3/343: write d0/d7/d1f/d4d/f56 [509803,36886] 0 2024-04-26T04:53:05.432 INFO:tasks.workunit.client.1.smithi078.stdout:0/858: link d2/d9/d31/c58 d2/d9/d60/d6a/daa/d112/c11d 0 2024-04-26T04:53:05.434 INFO:tasks.workunit.client.1.smithi078.stdout:0/859: truncate d2/d9/d78/d84/dd7/ff4 264872 0 2024-04-26T04:53:05.435 INFO:tasks.workunit.client.1.smithi078.stdout:0/860: chown d2/d9/d31/d4a/f7a 1193 1 2024-04-26T04:53:05.436 INFO:tasks.workunit.client.1.smithi078.stdout:4/850: creat d0/de/d22/d27/f12d x:0 0 0 2024-04-26T04:53:05.437 INFO:tasks.workunit.client.0.smithi077.stdout:3/344: fdatasync d0/d7/d1f/d4d/f56 0 2024-04-26T04:53:05.437 INFO:tasks.workunit.client.0.smithi077.stdout:3/345: fdatasync d0/d7/f33 0 2024-04-26T04:53:05.438 INFO:tasks.workunit.client.1.smithi078.stdout:4/851: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/f119 [103766,63174] 0 2024-04-26T04:53:05.438 INFO:tasks.workunit.client.1.smithi078.stdout:1/901: mkdir d7/d1e/d9e/d123 0 2024-04-26T04:53:05.440 INFO:tasks.workunit.client.1.smithi078.stdout:0/861: creat d2/d9/d31/df2/f11e x:0 0 0 2024-04-26T04:53:05.442 INFO:tasks.workunit.client.0.smithi077.stdout:3/346: mknod d0/d7/d15/c6c 0 2024-04-26T04:53:05.444 INFO:tasks.workunit.client.0.smithi077.stdout:3/347: creat d0/d9/d32/d29/f6d x:0 0 0 2024-04-26T04:53:05.445 INFO:tasks.workunit.client.1.smithi078.stdout:0/862: symlink d2/d9/d60/d6a/daa/d112/l11f 0 2024-04-26T04:53:05.447 INFO:tasks.workunit.client.1.smithi078.stdout:1/902: dwrite d7/f57 [0,4194304] 0 2024-04-26T04:53:05.447 INFO:tasks.workunit.client.1.smithi078.stdout:7/804: dwrite d9/da/faf [0,4194304] 0 2024-04-26T04:53:05.449 INFO:tasks.workunit.client.0.smithi077.stdout:3/348: mknod d0/c6e 0 2024-04-26T04:53:05.450 INFO:tasks.workunit.client.1.smithi078.stdout:4/852: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/df6/f101 [0,4194304] 0 2024-04-26T04:53:05.453 INFO:tasks.workunit.client.0.smithi077.stdout:3/349: truncate d0/d7/f3e 2775329 0 2024-04-26T04:53:05.457 INFO:tasks.workunit.client.1.smithi078.stdout:8/842: sync 2024-04-26T04:53:05.460 INFO:tasks.workunit.client.1.smithi078.stdout:2/853: dwrite d8/d11/d18/d5d/d61/f4a [0,4194304] 0 2024-04-26T04:53:05.463 INFO:tasks.workunit.client.0.smithi077.stdout:3/350: dread d0/d9/d32/d29/f6a [0,4194304] 0 2024-04-26T04:53:05.464 INFO:tasks.workunit.client.1.smithi078.stdout:6/912: write d2/de/d27/da9/da/d35/fae [233932,78] 0 2024-04-26T04:53:05.470 INFO:tasks.workunit.client.0.smithi077.stdout:0/384: sync 2024-04-26T04:53:05.470 INFO:tasks.workunit.client.0.smithi077.stdout:1/430: sync 2024-04-26T04:53:05.470 INFO:tasks.workunit.client.1.smithi078.stdout:2/854: rename d8/d53/cfb to d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/dec/c12d 0 2024-04-26T04:53:05.472 INFO:tasks.workunit.client.1.smithi078.stdout:6/913: dwrite d2/de/d27/da9/d3a/d69/dee/f10f [0,4194304] 0 2024-04-26T04:53:05.472 INFO:tasks.workunit.client.1.smithi078.stdout:2/855: chown d8/d11/f45 1 1 2024-04-26T04:53:05.474 INFO:tasks.workunit.client.1.smithi078.stdout:9/787: write dc/df/d27/d81/f91 [8628182,29924] 0 2024-04-26T04:53:05.475 INFO:tasks.workunit.client.1.smithi078.stdout:3/854: dwrite d16/d26/d2b/d31/f59 [0,4194304] 0 2024-04-26T04:53:05.477 INFO:tasks.workunit.client.1.smithi078.stdout:3/855: write d16/d26/d2e/d5f/d49/d77/ff3 [136053,43173] 0 2024-04-26T04:53:05.478 INFO:tasks.workunit.client.0.smithi077.stdout:1/431: dwrite d9/db/dd/d68/f7b [0,4194304] 0 2024-04-26T04:53:05.480 INFO:tasks.workunit.client.1.smithi078.stdout:3/856: dread d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/fcd [0,4194304] 0 2024-04-26T04:53:05.492 INFO:tasks.workunit.client.1.smithi078.stdout:1/903: getdents d7/d1e/d9e/db/d49 0 2024-04-26T04:53:05.495 INFO:tasks.workunit.client.0.smithi077.stdout:0/385: link d0/d19/d1e/d2f/f44 d0/d2c/f88 0 2024-04-26T04:53:05.496 INFO:tasks.workunit.client.1.smithi078.stdout:5/814: dwrite f5 [4194304,4194304] 0 2024-04-26T04:53:05.499 INFO:tasks.workunit.client.1.smithi078.stdout:9/788: rename dc/df/d17/l50 to dc/df/d4b/d22/dbe/led 0 2024-04-26T04:53:05.500 INFO:tasks.workunit.client.1.smithi078.stdout:9/789: readlink dc/df/d27/l34 0 2024-04-26T04:53:05.503 INFO:tasks.workunit.client.1.smithi078.stdout:3/857: dread d16/d26/d2b/d31/d71/d8a/fc3 [0,4194304] 0 2024-04-26T04:53:05.504 INFO:tasks.workunit.client.1.smithi078.stdout:1/904: readlink d7/l5d 0 2024-04-26T04:53:05.504 INFO:tasks.workunit.client.0.smithi077.stdout:0/386: read d0/f4 [437695,106312] 0 2024-04-26T04:53:05.506 INFO:tasks.workunit.client.1.smithi078.stdout:6/914: symlink d2/de/d27/d58/d10c/l128 0 2024-04-26T04:53:05.507 INFO:tasks.workunit.client.1.smithi078.stdout:6/915: truncate d2/d1a/d38/dd5/ffd 309064 0 2024-04-26T04:53:05.507 INFO:tasks.workunit.client.1.smithi078.stdout:5/815: creat d7/d18/d32/d37/d79/f114 x:0 0 0 2024-04-26T04:53:05.509 INFO:tasks.workunit.client.1.smithi078.stdout:6/916: read d2/de/d27/da9/da/f4a [1603961,45379] 0 2024-04-26T04:53:05.511 INFO:tasks.workunit.client.1.smithi078.stdout:0/863: write d2/d9/d31/d4a/f64 [2020897,22961] 0 2024-04-26T04:53:05.512 INFO:tasks.workunit.client.1.smithi078.stdout:3/858: symlink d16/d26/d2e/d5f/d49/dad/df0/l127 0 2024-04-26T04:53:05.512 INFO:tasks.workunit.client.1.smithi078.stdout:4/853: write d0/d1d/de1/d20/d31/d47/db2/d44/fb1 [156690,101751] 0 2024-04-26T04:53:05.515 INFO:tasks.workunit.client.1.smithi078.stdout:6/917: creat d2/de/d27/da9/d3a/d69/dee/f129 x:0 0 0 2024-04-26T04:53:05.517 INFO:tasks.workunit.client.0.smithi077.stdout:3/351: write d0/d9/d32/d2a/f6b [229095,70216] 0 2024-04-26T04:53:05.520 INFO:tasks.workunit.client.1.smithi078.stdout:3/859: rename d16/d26/d2e to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/d128 22 2024-04-26T04:53:05.522 INFO:tasks.workunit.client.1.smithi078.stdout:2/856: write d8/d11/d6e/ff4 [577378,56528] 0 2024-04-26T04:53:05.522 INFO:tasks.workunit.client.1.smithi078.stdout:7/805: dwrite d9/da/d11/d13/d1a/d52/d5a/d28/f100 [0,4194304] 0 2024-04-26T04:53:05.523 INFO:tasks.workunit.client.1.smithi078.stdout:8/843: dwrite d0/d19/d52/d35/d44/d6b/d104/d58/f70 [0,4194304] 0 2024-04-26T04:53:05.524 INFO:tasks.workunit.client.1.smithi078.stdout:0/864: dread d2/d9/d78/deb/f101 [0,4194304] 0 2024-04-26T04:53:05.525 INFO:tasks.workunit.client.1.smithi078.stdout:3/860: write d16/d26/d2b/d31/fc5 [4082410,6592] 0 2024-04-26T04:53:05.525 INFO:tasks.workunit.client.1.smithi078.stdout:1/905: getdents d7/d1e/d9e/dd 0 2024-04-26T04:53:05.527 INFO:tasks.workunit.client.0.smithi077.stdout:1/432: dwrite d9/db/d2b/d3f/f52 [0,4194304] 0 2024-04-26T04:53:05.527 INFO:tasks.workunit.client.1.smithi078.stdout:2/857: chown d8/d11/d18/dbb/lcb 30 1 2024-04-26T04:53:05.528 INFO:tasks.workunit.client.0.smithi077.stdout:3/352: dread d0/f19 [0,4194304] 0 2024-04-26T04:53:05.528 INFO:tasks.workunit.client.1.smithi078.stdout:7/806: chown d9/da/d33/d4b/d4c/d5b/dc3 510007 1 2024-04-26T04:53:05.529 INFO:tasks.workunit.client.0.smithi077.stdout:3/353: stat d0/d7/d15/f44 0 2024-04-26T04:53:05.529 INFO:tasks.workunit.client.0.smithi077.stdout:1/433: chown d9/db/dd/f49 3 1 2024-04-26T04:53:05.531 INFO:tasks.workunit.client.1.smithi078.stdout:4/854: creat d0/d1d/de1/f12e x:0 0 0 2024-04-26T04:53:05.534 INFO:tasks.workunit.client.1.smithi078.stdout:6/918: symlink d2/de/d27/da9/d2a/dbf/l12a 0 2024-04-26T04:53:05.535 INFO:tasks.workunit.client.0.smithi077.stdout:1/434: mkdir d9/d53/d6e/d7c 0 2024-04-26T04:53:05.536 INFO:tasks.workunit.client.0.smithi077.stdout:1/435: chown d9/db 0 1 2024-04-26T04:53:05.536 INFO:tasks.workunit.client.1.smithi078.stdout:5/816: dread d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:53:05.539 INFO:tasks.workunit.client.0.smithi077.stdout:3/354: truncate d0/f3d 3112257 0 2024-04-26T04:53:05.539 INFO:tasks.workunit.client.1.smithi078.stdout:9/790: dwrite dc/f54 [0,4194304] 0 2024-04-26T04:53:05.545 INFO:tasks.workunit.client.0.smithi077.stdout:3/355: truncate d0/d7/f8 3610800 0 2024-04-26T04:53:05.545 INFO:tasks.workunit.client.1.smithi078.stdout:8/844: dwrite d0/d19/d25/d32/d8a/d92/da4/fed [0,4194304] 0 2024-04-26T04:53:05.546 INFO:tasks.workunit.client.1.smithi078.stdout:8/845: chown d0/d7a/d83 1701 1 2024-04-26T04:53:05.547 INFO:tasks.workunit.client.0.smithi077.stdout:1/436: dread d9/db/dd/f5b [0,4194304] 0 2024-04-26T04:53:05.548 INFO:tasks.workunit.client.0.smithi077.stdout:1/437: readlink d9/db/l37 0 2024-04-26T04:53:05.549 INFO:tasks.workunit.client.1.smithi078.stdout:4/855: mknod d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/c12f 0 2024-04-26T04:53:05.553 INFO:tasks.workunit.client.1.smithi078.stdout:4/856: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [3402429,16655] 0 2024-04-26T04:53:05.553 INFO:tasks.workunit.client.1.smithi078.stdout:5/817: rmdir d7/d48/d57/d65/d70 39 2024-04-26T04:53:05.555 INFO:tasks.workunit.client.0.smithi077.stdout:1/438: dwrite d9/db/d2b/f61 [4194304,4194304] 0 2024-04-26T04:53:05.559 INFO:tasks.workunit.client.1.smithi078.stdout:2/858: mkdir d8/d11/d18/d5d/d127/d12e 0 2024-04-26T04:53:05.560 INFO:tasks.workunit.client.0.smithi077.stdout:1/439: stat d9/f42 0 2024-04-26T04:53:05.562 INFO:tasks.workunit.client.1.smithi078.stdout:7/807: mknod d9/da/d11/d13/d1a/d52/d5a/d41/c108 0 2024-04-26T04:53:05.563 INFO:tasks.workunit.client.1.smithi078.stdout:3/861: mknod d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/c129 0 2024-04-26T04:53:05.564 INFO:tasks.workunit.client.0.smithi077.stdout:1/440: dread d9/db/dd/d34/f51 [0,4194304] 0 2024-04-26T04:53:05.565 INFO:tasks.workunit.client.1.smithi078.stdout:0/865: write d2/d9/d18/f8a [5236216,3479] 0 2024-04-26T04:53:05.568 INFO:tasks.workunit.client.1.smithi078.stdout:0/866: truncate d2/d9/d78/d84/dd7/ff4 820349 0 2024-04-26T04:53:05.570 INFO:tasks.workunit.client.1.smithi078.stdout:3/862: dwrite d16/d26/d2e/d5f/d49/d77/ff3 [0,4194304] 0 2024-04-26T04:53:05.570 INFO:tasks.workunit.client.1.smithi078.stdout:4/857: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 160206 1 2024-04-26T04:53:05.571 INFO:tasks.workunit.client.0.smithi077.stdout:1/441: mkdir d9/d53/d7d 0 2024-04-26T04:53:05.572 INFO:tasks.workunit.client.1.smithi078.stdout:4/858: chown d0/d1d/de1/d20/d31/d47/db2/f125 116308 1 2024-04-26T04:53:05.574 INFO:tasks.workunit.client.1.smithi078.stdout:1/906: creat d7/d1e/d9e/d4c/d3d/f124 x:0 0 0 2024-04-26T04:53:05.575 INFO:tasks.workunit.client.1.smithi078.stdout:2/859: rename d8/d11/d14/fbf to d8/d11/d18/dbb/f12f 0 2024-04-26T04:53:05.576 INFO:tasks.workunit.client.0.smithi077.stdout:3/356: dwrite d0/d9/f2b [0,4194304] 0 2024-04-26T04:53:05.577 INFO:tasks.workunit.client.1.smithi078.stdout:8/846: symlink d0/d19/d52/de7/l111 0 2024-04-26T04:53:05.578 INFO:tasks.workunit.client.1.smithi078.stdout:8/847: fdatasync d0/d19/d52/d5c/df0/ffe 0 2024-04-26T04:53:05.580 INFO:tasks.workunit.client.1.smithi078.stdout:4/859: dwrite d0/d1d/de1/d20/d31/d47/db2/f40 [0,4194304] 0 2024-04-26T04:53:05.584 INFO:tasks.workunit.client.1.smithi078.stdout:6/919: creat d2/de/d27/da9/d2a/dbf/f12b x:0 0 0 2024-04-26T04:53:05.588 INFO:tasks.workunit.client.1.smithi078.stdout:3/863: rmdir d16/d26/d2b 39 2024-04-26T04:53:05.591 INFO:tasks.workunit.client.1.smithi078.stdout:9/791: dwrite dc/df/d27/d81/f9e [0,4194304] 0 2024-04-26T04:53:05.593 INFO:tasks.workunit.client.1.smithi078.stdout:8/848: rmdir d0/d19/d25/d32 39 2024-04-26T04:53:05.598 INFO:tasks.workunit.client.1.smithi078.stdout:7/808: write d9/da/d11/d13/d1a/d52/d5a/f26 [334401,55556] 0 2024-04-26T04:53:05.599 INFO:tasks.workunit.client.1.smithi078.stdout:6/920: creat d2/de/d27/da9/da/d36/db6/f12c x:0 0 0 2024-04-26T04:53:05.600 INFO:tasks.workunit.client.1.smithi078.stdout:7/809: read - d9/da/d11/d13/d50/fc8 zero size 2024-04-26T04:53:05.601 INFO:tasks.workunit.client.1.smithi078.stdout:5/818: link d7/d21/d2f/fc2 d7/d18/d32/d37/d79/f115 0 2024-04-26T04:53:05.605 INFO:tasks.workunit.client.1.smithi078.stdout:4/860: dwrite d0/de/f78 [0,4194304] 0 2024-04-26T04:53:05.606 INFO:tasks.workunit.client.1.smithi078.stdout:9/792: rmdir dc/df/d8a 39 2024-04-26T04:53:05.606 INFO:tasks.workunit.client.0.smithi077.stdout:2/428: sync 2024-04-26T04:53:05.606 INFO:tasks.workunit.client.1.smithi078.stdout:8/849: rmdir d0/d19/d52/d35/d44/d6b/d104/d58/d5b 39 2024-04-26T04:53:05.607 INFO:tasks.workunit.client.0.smithi077.stdout:2/429: mknod d9/d15/d44/c8f 0 2024-04-26T04:53:05.608 INFO:tasks.workunit.client.0.smithi077.stdout:2/430: write d9/fe [1623722,40868] 0 2024-04-26T04:53:05.609 INFO:tasks.workunit.client.1.smithi078.stdout:3/864: symlink d16/d26/d2b/d31/d78/dd1/l12a 0 2024-04-26T04:53:05.609 INFO:tasks.workunit.client.0.smithi077.stdout:2/431: chown d9/dc/d1d/d34/d39/d6b 188 1 2024-04-26T04:53:05.612 INFO:tasks.workunit.client.1.smithi078.stdout:6/921: unlink d2/de/d27/da9/d2a/dbf/l12a 0 2024-04-26T04:53:05.613 INFO:tasks.workunit.client.1.smithi078.stdout:4/861: chown d0/d1d/de1/d20/f2a 1012 1 2024-04-26T04:53:05.616 INFO:tasks.workunit.client.0.smithi077.stdout:2/432: dwrite d9/f72 [0,4194304] 0 2024-04-26T04:53:05.619 INFO:tasks.workunit.client.1.smithi078.stdout:1/907: creat d7/f125 x:0 0 0 2024-04-26T04:53:05.620 INFO:tasks.workunit.client.1.smithi078.stdout:3/865: dread - d16/d26/d2b/d31/d71/d8a/dbc/fbd zero size 2024-04-26T04:53:05.621 INFO:tasks.workunit.client.1.smithi078.stdout:1/908: stat d7/d1e/d9e/db/d37/d58/d5c/l118 0 2024-04-26T04:53:05.621 INFO:tasks.workunit.client.1.smithi078.stdout:3/866: fsync d16/d3c/f5b 0 2024-04-26T04:53:05.624 INFO:tasks.workunit.client.0.smithi077.stdout:1/442: dread d9/db/dd/f2f [0,4194304] 0 2024-04-26T04:53:05.629 INFO:tasks.workunit.client.1.smithi078.stdout:0/867: dwrite d2/d1e/d5a/f71 [0,4194304] 0 2024-04-26T04:53:05.631 INFO:tasks.workunit.client.1.smithi078.stdout:0/868: truncate d2/d9/d60/d6a/daa/f10b 708396 0 2024-04-26T04:53:05.635 INFO:tasks.workunit.client.0.smithi077.stdout:3/357: fdatasync d0/d9/f2b 0 2024-04-26T04:53:05.636 INFO:tasks.workunit.client.1.smithi078.stdout:7/810: dread d9/da/d11/d13/d1a/d52/d5a/d41/f47 [0,4194304] 0 2024-04-26T04:53:05.639 INFO:tasks.workunit.client.1.smithi078.stdout:0/869: dwrite d2/d9/d60/dd5/f108 [0,4194304] 0 2024-04-26T04:53:05.640 INFO:tasks.workunit.client.1.smithi078.stdout:0/870: stat d2/d1e/fe6 0 2024-04-26T04:53:05.641 INFO:tasks.workunit.client.1.smithi078.stdout:0/871: readlink d2/d9/d60/d7f/l86 0 2024-04-26T04:53:05.646 INFO:tasks.workunit.client.1.smithi078.stdout:2/860: truncate d8/d11/d14/d69/d8f/d34/dd3/de4/f58 3487561 0 2024-04-26T04:53:05.656 INFO:tasks.workunit.client.1.smithi078.stdout:9/793: truncate dc/fd7 524624 0 2024-04-26T04:53:05.658 INFO:tasks.workunit.client.1.smithi078.stdout:7/811: dread d9/da/d33/f70 [0,4194304] 0 2024-04-26T04:53:05.660 INFO:tasks.workunit.client.0.smithi077.stdout:3/358: dread d0/d7/f8 [0,4194304] 0 2024-04-26T04:53:05.662 INFO:tasks.workunit.client.1.smithi078.stdout:7/812: dread d9/da/d20/f9b [0,4194304] 0 2024-04-26T04:53:05.662 INFO:tasks.workunit.client.0.smithi077.stdout:2/433: dwrite d9/f12 [0,4194304] 0 2024-04-26T04:53:05.662 INFO:tasks.workunit.client.0.smithi077.stdout:1/443: write d9/db/d2b/d3f/f50 [814564,107345] 0 2024-04-26T04:53:05.663 INFO:tasks.workunit.client.0.smithi077.stdout:2/434: chown d9/dc/d1d/l57 6316105 1 2024-04-26T04:53:05.664 INFO:tasks.workunit.client.0.smithi077.stdout:2/435: dread - d9/f66 zero size 2024-04-26T04:53:05.668 INFO:tasks.workunit.client.0.smithi077.stdout:2/436: dread d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:53:05.675 INFO:tasks.workunit.client.0.smithi077.stdout:2/437: mknod d9/d15/c90 0 2024-04-26T04:53:05.676 INFO:tasks.workunit.client.0.smithi077.stdout:2/438: stat d9/dc/d1d/d34/d39/d63/d79/l2d 0 2024-04-26T04:53:05.677 INFO:tasks.workunit.client.0.smithi077.stdout:3/359: rename d0/f1d to d0/d7/f6f 0 2024-04-26T04:53:05.679 INFO:tasks.workunit.client.0.smithi077.stdout:2/439: mkdir d9/d15/d91 0 2024-04-26T04:53:05.680 INFO:tasks.workunit.client.0.smithi077.stdout:3/360: rmdir d0/d7/d1f/d4d 39 2024-04-26T04:53:05.683 INFO:tasks.workunit.client.0.smithi077.stdout:2/440: rename d9/dc/d1d/d1e/f55 to d9/dc/d1d/d1e/d77/f92 0 2024-04-26T04:53:05.683 INFO:tasks.workunit.client.0.smithi077.stdout:2/441: chown d9/f10 2 1 2024-04-26T04:53:05.684 INFO:tasks.workunit.client.0.smithi077.stdout:2/442: fdatasync d9/f72 0 2024-04-26T04:53:05.685 INFO:tasks.workunit.client.1.smithi078.stdout:5/819: creat d7/da/f116 x:0 0 0 2024-04-26T04:53:05.686 INFO:tasks.workunit.client.1.smithi078.stdout:8/850: mkdir d0/d19/d52/d35/d44/d6b/d104/d58/d112 0 2024-04-26T04:53:05.693 INFO:tasks.workunit.client.0.smithi077.stdout:2/443: creat d9/d15/f93 x:0 0 0 2024-04-26T04:53:05.694 INFO:tasks.workunit.client.1.smithi078.stdout:3/867: mknod d16/d26/d2e/d5f/c12b 0 2024-04-26T04:53:05.695 INFO:tasks.workunit.client.1.smithi078.stdout:1/909: rename d7/d1e/d9e/dd/d20/l64 to d7/d1e/d9e/db/d49/dba/d87/l126 0 2024-04-26T04:53:05.697 INFO:tasks.workunit.client.1.smithi078.stdout:5/820: dread d7/d48/d57/faa [0,4194304] 0 2024-04-26T04:53:05.698 INFO:tasks.workunit.client.0.smithi077.stdout:2/444: creat d9/dc/d1d/d34/d39/d63/d79/f94 x:0 0 0 2024-04-26T04:53:05.701 INFO:tasks.workunit.client.1.smithi078.stdout:0/872: symlink d2/d9/d60/d7f/l120 0 2024-04-26T04:53:05.702 INFO:tasks.workunit.client.1.smithi078.stdout:2/861: mkdir d8/d11/d18/d5d/d127/d130 0 2024-04-26T04:53:05.706 INFO:tasks.workunit.client.1.smithi078.stdout:8/851: readlink d0/d19/d52/d5c/lec 0 2024-04-26T04:53:05.708 INFO:tasks.workunit.client.1.smithi078.stdout:2/862: dwrite d8/d11/d18/d5d/d8b/db0/db4/ff1 [0,4194304] 0 2024-04-26T04:53:05.711 INFO:tasks.workunit.client.1.smithi078.stdout:2/863: write d8/d11/d14/d69/d8f/d34/dd3/de4/d8e/f112 [777236,107244] 0 2024-04-26T04:53:05.711 INFO:tasks.workunit.client.1.smithi078.stdout:7/813: fdatasync d9/da/d33/fff 0 2024-04-26T04:53:05.713 INFO:tasks.workunit.client.1.smithi078.stdout:4/862: creat d0/de/d22/d27/f130 x:0 0 0 2024-04-26T04:53:05.713 INFO:tasks.workunit.client.1.smithi078.stdout:2/864: readlink d8/d11/d14/d69/d8f/d34/d70/lc4 0 2024-04-26T04:53:05.714 INFO:tasks.workunit.client.1.smithi078.stdout:7/814: write d9/da/d33/d4b/d4c/d5b/dc3/fdc [1320409,118257] 0 2024-04-26T04:53:05.716 INFO:tasks.workunit.client.1.smithi078.stdout:7/815: read - d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 zero size 2024-04-26T04:53:05.717 INFO:tasks.workunit.client.0.smithi077.stdout:2/445: dread d9/dc/d1d/d34/f61 [0,4194304] 0 2024-04-26T04:53:05.717 INFO:tasks.workunit.client.1.smithi078.stdout:8/852: mkdir d0/d19/d52/d5c/da2/d113 0 2024-04-26T04:53:05.722 INFO:tasks.workunit.client.1.smithi078.stdout:0/873: link d2/d1e/d5a/da3/fe3 d2/d9/d31/d73/f121 0 2024-04-26T04:53:05.725 INFO:tasks.workunit.client.0.smithi077.stdout:2/446: read d9/dc/f5c [1531718,102387] 0 2024-04-26T04:53:05.726 INFO:tasks.workunit.client.0.smithi077.stdout:3/361: rename d0/d7/f6f to d0/d7/f70 0 2024-04-26T04:53:05.727 INFO:tasks.workunit.client.1.smithi078.stdout:6/922: truncate d2/d1a/f6 5693034 0 2024-04-26T04:53:05.728 INFO:tasks.workunit.client.0.smithi077.stdout:2/447: dread d9/dc/d1d/d1e/f5e [0,4194304] 0 2024-04-26T04:53:05.734 INFO:tasks.workunit.client.0.smithi077.stdout:3/362: creat d0/d9/d32/d29/f71 x:0 0 0 2024-04-26T04:53:05.735 INFO:tasks.workunit.client.1.smithi078.stdout:4/863: mkdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121/d131 0 2024-04-26T04:53:05.736 INFO:tasks.workunit.client.0.smithi077.stdout:2/448: mknod d9/dc/d5f/d78/c95 0 2024-04-26T04:53:05.737 INFO:tasks.workunit.client.0.smithi077.stdout:2/449: dread - d9/f66 zero size 2024-04-26T04:53:05.737 INFO:tasks.workunit.client.1.smithi078.stdout:4/864: write d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [1495538,40717] 0 2024-04-26T04:53:05.739 INFO:tasks.workunit.client.1.smithi078.stdout:1/910: link d7/d1e/d9e/dd/l9d d7/d1e/d9e/db/d49/dba/dc5/dcf/l127 0 2024-04-26T04:53:05.740 INFO:tasks.workunit.client.0.smithi077.stdout:2/450: write d9/dc/d1d/d34/d39/d63/d79/f54 [4008519,96504] 0 2024-04-26T04:53:05.741 INFO:tasks.workunit.client.1.smithi078.stdout:0/874: truncate d2/d9/d18/d7d/da9/ff0 1372234 0 2024-04-26T04:53:05.741 INFO:tasks.workunit.client.1.smithi078.stdout:0/875: stat d2/d1e/d5a/dda 0 2024-04-26T04:53:05.742 INFO:tasks.workunit.client.1.smithi078.stdout:5/821: creat d7/da/f117 x:0 0 0 2024-04-26T04:53:05.742 INFO:tasks.workunit.client.0.smithi077.stdout:2/451: truncate d9/dc/d1d/d34/d39/f68 1536054 0 2024-04-26T04:53:05.743 INFO:tasks.workunit.client.1.smithi078.stdout:6/923: mknod d2/de/d27/d58/c12d 0 2024-04-26T04:53:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.745+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f049402b410 con 0x7f047c06b7a0 2024-04-26T04:53:05.747 INFO:tasks.workunit.client.0.smithi077.stdout:2/452: rename d9/dc/d1d/d34/f87 to d9/d15/d91/f96 0 2024-04-26T04:53:05.747 INFO:tasks.workunit.client.1.smithi078.stdout:3/868: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f68 [3025353,47644] 0 2024-04-26T04:53:05.748 INFO:tasks.workunit.client.0.smithi077.stdout:2/453: write d9/f72 [3214320,59888] 0 2024-04-26T04:53:05.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.746+0000 7f047b7fe700 1 -- 172.21.15.77:0/202681104 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f049402b410 con 0x7f047c06b7a0 2024-04-26T04:53:05.749 INFO:tasks.workunit.client.1.smithi078.stdout:6/924: dwrite d2/de/f104 [0,4194304] 0 2024-04-26T04:53:05.750 INFO:tasks.workunit.client.1.smithi078.stdout:3/869: read d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f82 [944597,10683] 0 2024-04-26T04:53:05.752 INFO:tasks.workunit.client.1.smithi078.stdout:7/816: dread d9/da/d11/d13/d1a/f1c [0,4194304] 0 2024-04-26T04:53:05.752 INFO:tasks.workunit.client.1.smithi078.stdout:6/925: write d2/d1a/d5c/ff0 [738817,66594] 0 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 msgr2=0x7f047c06dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 0x7f047c06dc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f048400b3b0 tx=0x7f0484009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 msgr2=0x7f049410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0488002a50 tx=0x7f04880046b0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 shutdown_connections 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f047c06b7a0 0x7f047c06dc50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04940fd2f0 0x7f049410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 --2- 172.21.15.77:0/202681104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04940fdc40 0x7f049410de90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.749+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 >> 172.21.15.77:0/202681104 conn(0x7f04940f6bb0 msgr2=0x7f04940f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:05.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.750+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 shutdown_connections 2024-04-26T04:53:05.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.750+0000 7f0499064700 1 -- 172.21.15.77:0/202681104 wait complete. 2024-04-26T04:53:05.753 INFO:tasks.workunit.client.1.smithi078.stdout:6/926: fsync d2/d52/f103 0 2024-04-26T04:53:05.754 INFO:tasks.workunit.client.1.smithi078.stdout:4/865: creat d0/d1d/d46/d100/f132 x:0 0 0 2024-04-26T04:53:05.756 INFO:tasks.workunit.client.0.smithi077.stdout:2/454: fdatasync d9/f64 0 2024-04-26T04:53:05.759 INFO:tasks.workunit.client.1.smithi078.stdout:0/876: symlink d2/d9/d60/d6a/daa/l122 0 2024-04-26T04:53:05.761 INFO:tasks.workunit.client.1.smithi078.stdout:2/865: write d8/d11/d14/d5b/f10e [4543349,14957] 0 2024-04-26T04:53:05.766 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:53:05.770 INFO:tasks.workunit.client.1.smithi078.stdout:7/817: mkdir d9/da/d33/d4b/d4c/d6c/d109 0 2024-04-26T04:53:05.771 INFO:tasks.workunit.client.1.smithi078.stdout:4/866: rmdir d0/de/d22/d27/d7b 39 2024-04-26T04:53:05.777 INFO:tasks.workunit.client.1.smithi078.stdout:2/866: rename d8/d11/d18/dbb/d123 to d8/d11/d18/d5d/d61/df5/d131 0 2024-04-26T04:53:05.778 INFO:tasks.workunit.client.1.smithi078.stdout:3/870: symlink d16/d26/d2b/l12c 0 2024-04-26T04:53:05.778 INFO:tasks.workunit.client.1.smithi078.stdout:3/871: chown d16/d26/d2b/d66/laf 0 1 2024-04-26T04:53:05.779 INFO:tasks.workunit.client.1.smithi078.stdout:6/927: creat d2/de/d27/da9/de5/df4/d11f/f12e x:0 0 0 2024-04-26T04:53:05.779 INFO:tasks.workunit.client.1.smithi078.stdout:3/872: chown d16/d26/d2e/f4d 4466 1 2024-04-26T04:53:05.779 INFO:tasks.workunit.client.0.smithi077.stdout:4/433: sync 2024-04-26T04:53:05.780 INFO:tasks.workunit.client.0.smithi077.stdout:6/501: sync 2024-04-26T04:53:05.780 INFO:tasks.workunit.client.1.smithi078.stdout:7/818: creat d9/da/d11/d13/d1a/f10a x:0 0 0 2024-04-26T04:53:05.780 INFO:tasks.workunit.client.1.smithi078.stdout:3/873: write d16/d26/d2b/d31/d71/d8a/f107 [248502,34549] 0 2024-04-26T04:53:05.782 INFO:tasks.workunit.client.1.smithi078.stdout:4/867: rmdir d0/de/d22/d27 39 2024-04-26T04:53:05.786 INFO:tasks.workunit.client.1.smithi078.stdout:5/822: creat d7/d11/f118 x:0 0 0 2024-04-26T04:53:05.787 INFO:tasks.workunit.client.1.smithi078.stdout:7/819: dwrite d9/da/d11/d13/d1a/d52/d5a/f44 [8388608,4194304] 0 2024-04-26T04:53:05.788 INFO:tasks.workunit.client.0.smithi077.stdout:6/502: rename d8/d55/f7b to d8/d52/d53/d5e/fae 0 2024-04-26T04:53:05.789 INFO:tasks.workunit.client.1.smithi078.stdout:0/877: mkdir d2/d123 0 2024-04-26T04:53:05.790 INFO:tasks.workunit.client.0.smithi077.stdout:4/434: chown d4/de/d15/d1f/d25/c68 78 1 2024-04-26T04:53:05.792 INFO:tasks.workunit.client.0.smithi077.stdout:2/455: write d9/dc/d1d/d34/d39/d63/d79/f3f [1341254,43669] 0 2024-04-26T04:53:05.792 INFO:tasks.workunit.client.0.smithi077.stdout:4/435: chown d4/de/d15/f5b 3 1 2024-04-26T04:53:05.794 INFO:tasks.workunit.client.0.smithi077.stdout:2/456: read d9/dc/d1d/d34/d39/d63/d79/f3f [1366188,111557] 0 2024-04-26T04:53:05.800 INFO:tasks.workunit.client.0.smithi077.stdout:4/436: creat d4/d17/d18/d43/d6e/f95 x:0 0 0 2024-04-26T04:53:05.801 INFO:tasks.workunit.client.1.smithi078.stdout:1/911: getdents d7/d1e/d9e/db/d49/d93 0 2024-04-26T04:53:05.802 INFO:tasks.workunit.client.1.smithi078.stdout:1/912: fdatasync d7/d1e/d9e/db/f5a 0 2024-04-26T04:53:05.803 INFO:tasks.workunit.client.1.smithi078.stdout:5/823: creat d7/d18/d32/d37/d8f/f119 x:0 0 0 2024-04-26T04:53:05.805 INFO:tasks.workunit.client.1.smithi078.stdout:7/820: creat d9/da/d11/d13/d1a/d2d/f10b x:0 0 0 2024-04-26T04:53:05.805 INFO:tasks.workunit.client.1.smithi078.stdout:1/913: dread d7/d1e/d9e/db/d37/fa9 [0,4194304] 0 2024-04-26T04:53:05.808 INFO:tasks.workunit.client.0.smithi077.stdout:4/437: dread d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:05.809 INFO:tasks.workunit.client.1.smithi078.stdout:3/874: dread d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f58 [0,4194304] 0 2024-04-26T04:53:05.811 INFO:tasks.workunit.client.0.smithi077.stdout:4/438: creat d4/d17/f96 x:0 0 0 2024-04-26T04:53:05.812 INFO:tasks.workunit.client.1.smithi078.stdout:3/875: dread d16/d26/d2b/d31/f93 [0,4194304] 0 2024-04-26T04:53:05.813 INFO:tasks.workunit.client.0.smithi077.stdout:4/439: symlink d4/d4e/l97 0 2024-04-26T04:53:05.814 INFO:tasks.workunit.client.1.smithi078.stdout:6/928: dwrite d2/de/d27/da9/da8/fb7 [0,4194304] 0 2024-04-26T04:53:05.814 INFO:tasks.workunit.client.1.smithi078.stdout:5/824: rmdir d7/d48/d57/d65/d7a 39 2024-04-26T04:53:05.815 INFO:tasks.workunit.client.1.smithi078.stdout:5/825: chown d7/d18/d1c/d8e/f103 30 1 2024-04-26T04:53:05.817 INFO:tasks.workunit.client.1.smithi078.stdout:2/867: creat d8/d11/f132 x:0 0 0 2024-04-26T04:53:05.818 INFO:tasks.workunit.client.1.smithi078.stdout:5/826: truncate d7/da/d1e/d91/ffa 190510 0 2024-04-26T04:53:05.818 INFO:tasks.workunit.client.0.smithi077.stdout:2/457: dwrite d9/dc/d1d/d34/d39/d63/d79/f30 [0,4194304] 0 2024-04-26T04:53:05.819 INFO:tasks.workunit.client.0.smithi077.stdout:4/440: dread d4/d17/d18/d43/f74 [0,4194304] 0 2024-04-26T04:53:05.821 INFO:tasks.workunit.client.1.smithi078.stdout:1/914: mknod d7/d1e/d9e/dd/d20/dca/d115/c128 0 2024-04-26T04:53:05.822 INFO:tasks.workunit.client.1.smithi078.stdout:7/821: symlink d9/da/l10c 0 2024-04-26T04:53:05.825 INFO:tasks.workunit.client.0.smithi077.stdout:4/441: dread d4/f14 [0,4194304] 0 2024-04-26T04:53:05.825 INFO:tasks.workunit.client.1.smithi078.stdout:0/878: creat d2/d9/d18/d7d/f124 x:0 0 0 2024-04-26T04:53:05.827 INFO:tasks.workunit.client.1.smithi078.stdout:2/868: creat d8/d11/d18/dfe/f133 x:0 0 0 2024-04-26T04:53:05.827 INFO:tasks.workunit.client.1.smithi078.stdout:5/827: creat d7/da/d1e/d91/dd6/f11a x:0 0 0 2024-04-26T04:53:05.828 INFO:tasks.workunit.client.0.smithi077.stdout:2/458: symlink d9/dc/d1d/d34/l97 0 2024-04-26T04:53:05.829 INFO:tasks.workunit.client.1.smithi078.stdout:1/915: creat d7/d1e/d9e/db/d49/dba/d87/f129 x:0 0 0 2024-04-26T04:53:05.831 INFO:tasks.workunit.client.1.smithi078.stdout:7/822: rmdir d9/da/d11/d13/d1a/d65 39 2024-04-26T04:53:05.832 INFO:tasks.workunit.client.1.smithi078.stdout:7/823: write d9/da/d11/fe7 [813444,86768] 0 2024-04-26T04:53:05.833 INFO:tasks.workunit.client.0.smithi077.stdout:2/459: creat d9/dc/d5f/d78/f98 x:0 0 0 2024-04-26T04:53:05.837 INFO:tasks.workunit.client.1.smithi078.stdout:5/828: mknod d7/d21/d2f/d90/c11b 0 2024-04-26T04:53:05.838 INFO:tasks.workunit.client.1.smithi078.stdout:3/876: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f6c [1158772,54801] 0 2024-04-26T04:53:05.839 INFO:tasks.workunit.client.0.smithi077.stdout:4/442: getdents d4/de/d15/d1f/d25 0 2024-04-26T04:53:05.840 INFO:tasks.workunit.client.1.smithi078.stdout:3/877: write d16/d26/f7f [8387599,66651] 0 2024-04-26T04:53:05.841 INFO:tasks.workunit.client.1.smithi078.stdout:7/824: rename d9/da/d11/d13/d1a/d2d/l78 to d9/da/d33/d4b/d4c/d6c/d109/l10d 0 2024-04-26T04:53:05.842 INFO:tasks.workunit.client.1.smithi078.stdout:0/879: symlink d2/d9/d78/d84/d103/l125 0 2024-04-26T04:53:05.843 INFO:tasks.workunit.client.1.smithi078.stdout:7/825: write d9/da/faf [3738938,115010] 0 2024-04-26T04:53:05.843 INFO:tasks.workunit.client.1.smithi078.stdout:0/880: chown d2/d1e/d5a/da3 472 1 2024-04-26T04:53:05.843 INFO:tasks.workunit.client.0.smithi077.stdout:4/443: creat d4/d17/f98 x:0 0 0 2024-04-26T04:53:05.844 INFO:tasks.workunit.client.0.smithi077.stdout:2/460: getdents d9/d15/d91 0 2024-04-26T04:53:05.845 INFO:tasks.workunit.client.0.smithi077.stdout:4/444: read d4/de/f2b [1122437,49067] 0 2024-04-26T04:53:05.846 INFO:tasks.workunit.client.0.smithi077.stdout:4/445: dread - d4/d17/f7a zero size 2024-04-26T04:53:05.846 INFO:tasks.workunit.client.1.smithi078.stdout:1/916: link d7/d1e/d9e/db/d37/fe6 d7/d1e/d9e/db/d49/d93/f12a 0 2024-04-26T04:53:05.847 INFO:tasks.workunit.client.0.smithi077.stdout:4/446: stat d4/de/d15/l22 0 2024-04-26T04:53:05.848 INFO:tasks.workunit.client.1.smithi078.stdout:3/878: mknod d16/d26/d2b/d31/d78/c12d 0 2024-04-26T04:53:05.849 INFO:tasks.workunit.client.1.smithi078.stdout:3/879: chown d16/d26/d2b/d31/d78/dd1/l11d 45130305 1 2024-04-26T04:53:05.851 INFO:tasks.workunit.client.1.smithi078.stdout:1/917: dwrite d7/d1e/fe2 [0,4194304] 0 2024-04-26T04:53:05.851 INFO:tasks.workunit.client.1.smithi078.stdout:0/881: creat d2/d9/d18/db7/d92/f126 x:0 0 0 2024-04-26T04:53:05.852 INFO:tasks.workunit.client.0.smithi077.stdout:2/461: dwrite d9/f3d [0,4194304] 0 2024-04-26T04:53:05.853 INFO:tasks.workunit.client.1.smithi078.stdout:2/869: dread d8/d11/fbe [0,4194304] 0 2024-04-26T04:53:05.856 INFO:tasks.workunit.client.0.smithi077.stdout:4/447: symlink d4/d17/d7f/l99 0 2024-04-26T04:53:05.861 INFO:tasks.workunit.client.0.smithi077.stdout:2/462: mknod d9/dc/d1d/d34/d39/d63/c99 0 2024-04-26T04:53:05.861 INFO:tasks.workunit.client.0.smithi077.stdout:2/463: chown d9/dc/d1d/f83 1 1 2024-04-26T04:53:05.864 INFO:tasks.workunit.client.1.smithi078.stdout:1/918: mknod d7/d1e/d9e/db/d49/dba/d87/c12b 0 2024-04-26T04:53:05.866 INFO:tasks.workunit.client.1.smithi078.stdout:1/919: write d7/d1e/d9e/db/d49/dba/d87/f11b [100608,40409] 0 2024-04-26T04:53:05.868 INFO:tasks.workunit.client.1.smithi078.stdout:1/920: dread - d7/d1e/d9e/db/d37/d58/d5c/f103 zero size 2024-04-26T04:53:05.868 INFO:tasks.workunit.client.0.smithi077.stdout:4/448: unlink d4/de/d15/c89 0 2024-04-26T04:53:05.870 INFO:tasks.workunit.client.1.smithi078.stdout:7/826: rename d9/da/d11/d13/d1a/d52/d5a/d28/l2e to d9/da/d11/l10e 0 2024-04-26T04:53:05.872 INFO:tasks.workunit.client.1.smithi078.stdout:0/882: mknod d2/d1e/d5a/dda/c127 0 2024-04-26T04:53:05.877 INFO:tasks.workunit.client.0.smithi077.stdout:4/449: mknod d4/c9a 0 2024-04-26T04:53:05.878 INFO:tasks.workunit.client.0.smithi077.stdout:4/450: fdatasync d4/d4e/f91 0 2024-04-26T04:53:05.878 INFO:tasks.workunit.client.1.smithi078.stdout:5/829: dread d7/f8 [0,4194304] 0 2024-04-26T04:53:05.879 INFO:tasks.workunit.client.1.smithi078.stdout:5/830: chown d7/d48/d57/d65/d46/lae 498822 1 2024-04-26T04:53:05.880 INFO:tasks.workunit.client.1.smithi078.stdout:3/880: rename d16/d26/d2b to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e 0 2024-04-26T04:53:05.882 INFO:tasks.workunit.client.1.smithi078.stdout:7/827: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/f10f x:0 0 0 2024-04-26T04:53:05.886 INFO:tasks.workunit.client.0.smithi077.stdout:2/464: write d9/dc/d5f/f62 [25135,94696] 0 2024-04-26T04:53:05.886 INFO:tasks.workunit.client.1.smithi078.stdout:2/870: creat d8/d11/d18/d5d/d61/f134 x:0 0 0 2024-04-26T04:53:05.888 INFO:tasks.workunit.client.0.smithi077.stdout:4/451: rename d4/de/l3c to d4/de/d15/d1f/d35/l9b 0 2024-04-26T04:53:05.895 INFO:tasks.workunit.client.1.smithi078.stdout:1/921: dwrite d7/d1e/d9e/db/d49/df8/f10f [0,4194304] 0 2024-04-26T04:53:05.897 INFO:tasks.workunit.client.1.smithi078.stdout:7/828: truncate d9/da/d11/d13/d50/f7c 3453531 0 2024-04-26T04:53:05.900 INFO:tasks.workunit.client.1.smithi078.stdout:1/922: dwrite f1 [0,4194304] 0 2024-04-26T04:53:05.901 INFO:tasks.workunit.client.1.smithi078.stdout:1/923: write d7/d1e/d9e/dd/d20/dca/ff6 [47356,107702] 0 2024-04-26T04:53:05.902 INFO:tasks.workunit.client.0.smithi077.stdout:2/465: rename d9/dc/d1d/d34/d39/d63/d6a/c7e to d9/dc/d1d/d34/d39/d63/d79/d5b/c9a 0 2024-04-26T04:53:05.905 INFO:tasks.workunit.client.0.smithi077.stdout:2/466: rmdir d9/d15/d56 39 2024-04-26T04:53:05.906 INFO:tasks.workunit.client.1.smithi078.stdout:2/871: symlink d8/d11/d6e/l135 0 2024-04-26T04:53:05.906 INFO:tasks.workunit.client.1.smithi078.stdout:5/831: symlink d7/d48/da9/d10f/l11c 0 2024-04-26T04:53:05.906 INFO:tasks.workunit.client.1.smithi078.stdout:5/832: readlink d7/d21/lf4 0 2024-04-26T04:53:05.907 INFO:tasks.workunit.client.0.smithi077.stdout:2/467: readlink d9/d15/l86 0 2024-04-26T04:53:05.907 INFO:tasks.workunit.client.1.smithi078.stdout:3/881: symlink d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/l12f 0 2024-04-26T04:53:05.908 INFO:tasks.workunit.client.0.smithi077.stdout:4/452: dwrite d4/de/f2b [0,4194304] 0 2024-04-26T04:53:05.909 INFO:tasks.workunit.client.1.smithi078.stdout:5/833: dread d7/d48/d57/d65/d46/dce/d109/f10d [0,4194304] 0 2024-04-26T04:53:05.911 INFO:tasks.workunit.client.1.smithi078.stdout:0/883: creat d2/d9/f128 x:0 0 0 2024-04-26T04:53:05.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/880083109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 msgr2=0x7fd59c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/880083109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c0fc150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004940 tx=0x7fd58c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:05.916 INFO:tasks.workunit.client.1.smithi078.stdout:9/794: sync 2024-04-26T04:53:05.916 INFO:tasks.workunit.client.1.smithi078.stdout:8/853: sync 2024-04-26T04:53:05.916 INFO:tasks.workunit.client.1.smithi078.stdout:4/868: sync 2024-04-26T04:53:05.916 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:05 smithi077 ceph-mon[26311]: pgmap v22: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 64 MiB/s rd, 200 MiB/s wr, 386 op/s 2024-04-26T04:53:05.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/880083109 shutdown_connections 2024-04-26T04:53:05.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/880083109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c0fcff0 0x7fd59c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/880083109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c0fc150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/880083109 >> 172.21.15.77:0/880083109 conn(0x7fd59c0f74c0 msgr2=0x7fd59c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:05.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/880083109 shutdown_connections 2024-04-26T04:53:05.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.914+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/880083109 wait complete. 2024-04-26T04:53:05.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.915+0000 7fd5a3ff8700 1 Processor -- start 2024-04-26T04:53:05.919 INFO:tasks.workunit.client.1.smithi078.stdout:9/795: chown dc/df/d89 297 1 2024-04-26T04:53:05.919 INFO:tasks.workunit.client.1.smithi078.stdout:8/854: readlink d0/d19/d25/ld2 0 2024-04-26T04:53:05.919 INFO:tasks.workunit.client.1.smithi078.stdout:1/924: dwrite d7/f11 [0,4194304] 0 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.915+0000 7fd5a3ff8700 1 -- start start 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.915+0000 7fd5a3ff8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.915+0000 7fd5a3ff8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c0fcff0 0x7fd59c10ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a3ff8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c10d4b0 con 0x7fd59c0fbd40 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a3ff8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c10dca0 con 0x7fd59c0fcff0 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58332/0 (socket says 172.21.15.77:58332) 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 -- 172.21.15.77:0/3914203429 learned_addr learned my addr 172.21.15.77:0/3914203429 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 -- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c0fcff0 msgr2=0x7fd59c10ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:05.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c0fcff0 0x7fd59c10ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 -- 172.21.15.77:0/3914203429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c005040 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a1d94700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009fd0 tx=0x7fd58c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58c00e040 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c10df20 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c10e4e0 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd58c01dad0 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.916+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58c0139d0 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.918+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5800052e0 con 0x7fd59c0fbd40 2024-04-26T04:53:05.921 INFO:tasks.workunit.client.0.smithi077.stdout:4/453: creat d4/f9c x:0 0 0 2024-04-26T04:53:05.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.919+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fd58c013c40 con 0x7fd59c0fbd40 2024-04-26T04:53:05.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.919+0000 7fd5927fc700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 0x7fd58806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:05.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.920+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd58c08fc40 con 0x7fd59c0fbd40 2024-04-26T04:53:05.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.920+0000 7fd5a1593700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 0x7fd58806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:05.923 INFO:tasks.workunit.client.1.smithi078.stdout:7/829: mkdir d9/da/d11/d13/d1a/d52/d110 0 2024-04-26T04:53:05.923 INFO:tasks.workunit.client.1.smithi078.stdout:4/869: dread - d0/d1d/de1/d20/d31/d47/db2/f125 zero size 2024-04-26T04:53:05.923 INFO:tasks.workunit.client.1.smithi078.stdout:5/834: write d7/d11/f47 [3565324,82909] 0 2024-04-26T04:53:05.923 INFO:tasks.workunit.client.1.smithi078.stdout:5/835: chown d7/d11/dee/d111 46410762 1 2024-04-26T04:53:05.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.922+0000 7fd5a1593700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 0x7fd58806dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd598006fd0 tx=0x7fd598006db0 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:05.924 INFO:tasks.workunit.client.0.smithi077.stdout:4/454: symlink d4/d17/d18/d43/d6e/l9d 0 2024-04-26T04:53:05.927 INFO:tasks.workunit.client.1.smithi078.stdout:9/796: creat dc/df/d4b/d22/d2a/d92/ddb/fee x:0 0 0 2024-04-26T04:53:05.930 INFO:tasks.workunit.client.1.smithi078.stdout:7/830: mkdir d9/da/d33/d4b/d4c/d6c/d111 0 2024-04-26T04:53:05.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:05.929+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd58c05d680 con 0x7fd59c0fbd40 2024-04-26T04:53:05.932 INFO:tasks.workunit.client.0.smithi077.stdout:4/455: dwrite d4/d17/d18/d43/d53/d64/f81 [0,4194304] 0 2024-04-26T04:53:05.933 INFO:tasks.workunit.client.0.smithi077.stdout:4/456: readlink d4/de/l13 0 2024-04-26T04:53:05.934 INFO:tasks.workunit.client.1.smithi078.stdout:9/797: dwrite dc/d55/fca [0,4194304] 0 2024-04-26T04:53:05.938 INFO:tasks.workunit.client.1.smithi078.stdout:4/870: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/fbe [0,4194304] 0 2024-04-26T04:53:05.940 INFO:tasks.workunit.client.0.smithi077.stdout:4/457: rename d4/d4e/f77 to d4/d17/d7f/f9e 0 2024-04-26T04:53:05.941 INFO:tasks.workunit.client.1.smithi078.stdout:0/884: creat d2/d9/d18/d7d/f129 x:0 0 0 2024-04-26T04:53:05.941 INFO:tasks.workunit.client.0.smithi077.stdout:4/458: chown d4/d17/d18/d43/f74 2503 1 2024-04-26T04:53:05.942 INFO:tasks.workunit.client.1.smithi078.stdout:2/872: getdents d8/d53 0 2024-04-26T04:53:05.942 INFO:tasks.workunit.client.0.smithi077.stdout:4/459: chown d4/d17/d18/d43/d6e/d6c 27673433 1 2024-04-26T04:53:05.943 INFO:tasks.workunit.client.1.smithi078.stdout:8/855: getdents d0/d19/d52/d35/d44/d6b/d104/d58/d112 0 2024-04-26T04:53:05.946 INFO:tasks.workunit.client.0.smithi077.stdout:7/479: sync 2024-04-26T04:53:05.946 INFO:tasks.workunit.client.0.smithi077.stdout:8/483: sync 2024-04-26T04:53:05.946 INFO:tasks.workunit.client.0.smithi077.stdout:0/387: sync 2024-04-26T04:53:05.947 INFO:tasks.workunit.client.0.smithi077.stdout:0/388: chown d0/d47/f50 45601687 1 2024-04-26T04:53:05.947 INFO:tasks.workunit.client.0.smithi077.stdout:7/480: fsync da/d15/f94 0 2024-04-26T04:53:05.948 INFO:tasks.workunit.client.0.smithi077.stdout:2/468: truncate d9/dc/d1d/d34/d39/d63/d79/f54 3524426 0 2024-04-26T04:53:05.952 INFO:tasks.workunit.client.0.smithi077.stdout:8/484: mkdir d4/df/d6f/da3 0 2024-04-26T04:53:05.953 INFO:tasks.workunit.client.0.smithi077.stdout:2/469: dread d9/dc/d1d/d34/f61 [0,4194304] 0 2024-04-26T04:53:05.954 INFO:tasks.workunit.client.1.smithi078.stdout:1/925: creat d7/d1e/d9e/f12c x:0 0 0 2024-04-26T04:53:05.955 INFO:tasks.workunit.client.1.smithi078.stdout:9/798: dread dc/fd7 [0,4194304] 0 2024-04-26T04:53:05.955 INFO:tasks.workunit.client.1.smithi078.stdout:3/882: dwrite d16/d26/d2e/d74/f76 [0,4194304] 0 2024-04-26T04:53:05.959 INFO:tasks.workunit.client.0.smithi077.stdout:2/470: creat d9/dc/d1d/d1e/f9b x:0 0 0 2024-04-26T04:53:05.960 INFO:tasks.workunit.client.1.smithi078.stdout:0/885: mkdir d2/d9/d60/d6a/da5/d12a 0 2024-04-26T04:53:05.962 INFO:tasks.workunit.client.0.smithi077.stdout:7/481: mknod da/d15/d42/d8c/ca3 0 2024-04-26T04:53:05.962 INFO:tasks.workunit.client.0.smithi077.stdout:4/460: dwrite d4/d17/d18/d43/d53/d64/f81 [4194304,4194304] 0 2024-04-26T04:53:05.964 INFO:tasks.workunit.client.1.smithi078.stdout:6/929: sync 2024-04-26T04:53:05.964 INFO:tasks.workunit.client.1.smithi078.stdout:2/873: creat d8/d11/d18/dfe/f136 x:0 0 0 2024-04-26T04:53:05.964 INFO:tasks.workunit.client.1.smithi078.stdout:6/930: readlink d2/d1a/d38/lc9 0 2024-04-26T04:53:05.965 INFO:tasks.workunit.client.1.smithi078.stdout:2/874: write d8/d11/d14/f67 [3224501,130508] 0 2024-04-26T04:53:05.969 INFO:tasks.workunit.client.1.smithi078.stdout:7/831: dwrite d9/da/d33/d4b/d4c/f55 [0,4194304] 0 2024-04-26T04:53:05.972 INFO:tasks.workunit.client.0.smithi077.stdout:2/471: creat d9/dc/d1d/d34/d39/d63/f9c x:0 0 0 2024-04-26T04:53:05.973 INFO:tasks.workunit.client.1.smithi078.stdout:8/856: write d0/d19/d25/d32/d8a/d92/da4/fc6 [478625,12214] 0 2024-04-26T04:53:05.974 INFO:tasks.workunit.client.0.smithi077.stdout:4/461: dwrite d4/f9c [0,4194304] 0 2024-04-26T04:53:05.976 INFO:tasks.workunit.client.0.smithi077.stdout:4/462: chown d4/de/d15/d1f/d35/c54 247617 1 2024-04-26T04:53:05.977 INFO:tasks.workunit.client.1.smithi078.stdout:8/857: write d0/d19/d52/d35/d44/d6b/d104/d58/feb [181459,9044] 0 2024-04-26T04:53:05.981 INFO:tasks.workunit.client.0.smithi077.stdout:8/485: dread d4/df/d22/d2d/f4a [0,4194304] 0 2024-04-26T04:53:05.981 INFO:tasks.workunit.client.1.smithi078.stdout:3/883: fdatasync d16/d26/d2e/d5f/d49/d77/dba/fc7 0 2024-04-26T04:53:05.982 INFO:tasks.workunit.client.1.smithi078.stdout:3/884: chown f6 624375968 1 2024-04-26T04:53:05.986 INFO:tasks.workunit.client.1.smithi078.stdout:0/886: creat d2/dc1/f12b x:0 0 0 2024-04-26T04:53:05.989 INFO:tasks.workunit.client.0.smithi077.stdout:0/389: truncate d0/d2c/f2e 4149223 0 2024-04-26T04:53:05.995 INFO:tasks.workunit.client.1.smithi078.stdout:5/836: getdents d7/d21/d2f 0 2024-04-26T04:53:05.996 INFO:tasks.workunit.client.1.smithi078.stdout:5/837: dread - d7/da/f117 zero size 2024-04-26T04:53:06.000 INFO:tasks.workunit.client.1.smithi078.stdout:8/858: creat d0/d19/d25/d32/f114 x:0 0 0 2024-04-26T04:53:06.002 INFO:tasks.workunit.client.1.smithi078.stdout:4/871: link d0/d1d/de1/d20/d31/f3f d0/d1d/de1/dd8/f133 0 2024-04-26T04:53:06.005 INFO:tasks.workunit.client.1.smithi078.stdout:9/799: dwrite dc/fd7 [0,4194304] 0 2024-04-26T04:53:06.006 INFO:tasks.workunit.client.0.smithi077.stdout:7/482: dread da/d15/f44 [0,4194304] 0 2024-04-26T04:53:06.006 INFO:tasks.workunit.client.1.smithi078.stdout:9/800: fsync dc/df/fd1 0 2024-04-26T04:53:06.006 INFO:tasks.workunit.client.1.smithi078.stdout:0/887: mkdir d2/dc1/d12c 0 2024-04-26T04:53:06.008 INFO:tasks.workunit.client.1.smithi078.stdout:6/931: truncate d2/de/d27/da9/d2a/fb0 3932398 0 2024-04-26T04:53:06.010 INFO:tasks.workunit.client.0.smithi077.stdout:2/472: mkdir d9/dc/d1d/d1e/d77/d9d 0 2024-04-26T04:53:06.011 INFO:tasks.workunit.client.0.smithi077.stdout:2/473: stat d9/dc/d1d/d34/f61 0 2024-04-26T04:53:06.012 INFO:tasks.workunit.client.0.smithi077.stdout:2/474: readlink d9/d15/l86 0 2024-04-26T04:53:06.018 INFO:tasks.workunit.client.1.smithi078.stdout:7/832: creat d9/da/d33/d4b/d4c/d5b/d83/da5/f112 x:0 0 0 2024-04-26T04:53:06.019 INFO:tasks.workunit.client.1.smithi078.stdout:5/838: mknod d7/d18/d32/d37/d8f/c11d 0 2024-04-26T04:53:06.020 INFO:tasks.workunit.client.1.smithi078.stdout:5/839: readlink d7/da/d1e/l20 0 2024-04-26T04:53:06.025 INFO:tasks.workunit.client.1.smithi078.stdout:4/872: mkdir d0/d1d/db4/d134 0 2024-04-26T04:53:06.025 INFO:tasks.workunit.client.1.smithi078.stdout:4/873: dread - d0/f108 zero size 2024-04-26T04:53:06.025 INFO:tasks.workunit.client.1.smithi078.stdout:9/801: dread dc/df/d27/d81/f66 [0,4194304] 0 2024-04-26T04:53:06.028 INFO:tasks.workunit.client.0.smithi077.stdout:8/486: symlink d4/d17/d35/la4 0 2024-04-26T04:53:06.029 INFO:tasks.workunit.client.0.smithi077.stdout:1/444: sync 2024-04-26T04:53:06.029 INFO:tasks.workunit.client.0.smithi077.stdout:5/455: sync 2024-04-26T04:53:06.029 INFO:tasks.workunit.client.0.smithi077.stdout:3/363: sync 2024-04-26T04:53:06.029 INFO:tasks.workunit.client.0.smithi077.stdout:9/463: sync 2024-04-26T04:53:06.029 INFO:tasks.workunit.client.0.smithi077.stdout:6/503: sync 2024-04-26T04:53:06.034 INFO:tasks.workunit.client.0.smithi077.stdout:7/483: dread da/d13/d22/f99 [0,4194304] 0 2024-04-26T04:53:06.034 INFO:tasks.workunit.client.0.smithi077.stdout:9/464: chown d9/dd/d3d/f3e 32636 1 2024-04-26T04:53:06.034 INFO:tasks.workunit.client.0.smithi077.stdout:6/504: write d8/d55/f62 [1268952,101926] 0 2024-04-26T04:53:06.034 INFO:tasks.workunit.client.0.smithi077.stdout:4/463: link d4/de/c60 d4/d17/d18/d43/d53/d75/c9f 0 2024-04-26T04:53:06.036 INFO:tasks.workunit.client.0.smithi077.stdout:4/464: fdatasync d4/f8 0 2024-04-26T04:53:06.036 INFO:tasks.workunit.client.1.smithi078.stdout:6/932: unlink d2/de/d27/da9/d2a/dbf/cd9 0 2024-04-26T04:53:06.038 INFO:tasks.workunit.client.0.smithi077.stdout:8/487: creat d4/d17/d35/d5b/fa5 x:0 0 0 2024-04-26T04:53:06.039 INFO:tasks.workunit.client.0.smithi077.stdout:1/445: dwrite d9/db/dd/d34/d63/f76 [0,4194304] 0 2024-04-26T04:53:06.040 INFO:tasks.workunit.client.1.smithi078.stdout:7/833: mkdir d9/da/d33/d4b/d4c/d5b/d83/da5/d113 0 2024-04-26T04:53:06.041 INFO:tasks.workunit.client.1.smithi078.stdout:2/875: truncate d8/d11/d14/d69/d8f/d34/d70/f75 2849941 0 2024-04-26T04:53:06.041 INFO:tasks.workunit.client.1.smithi078.stdout:1/926: sync 2024-04-26T04:53:06.044 INFO:tasks.workunit.client.0.smithi077.stdout:4/465: dwrite d4/de/f2b [0,4194304] 0 2024-04-26T04:53:06.045 INFO:tasks.workunit.client.1.smithi078.stdout:2/876: dwrite d8/fc6 [0,4194304] 0 2024-04-26T04:53:06.048 INFO:tasks.workunit.client.1.smithi078.stdout:2/877: truncate d8/d11/d18/d5d/d8b/db0/d10d/d110/f124 638683 0 2024-04-26T04:53:06.052 INFO:tasks.workunit.client.1.smithi078.stdout:2/878: dread d8/d53/f71 [0,4194304] 0 2024-04-26T04:53:06.054 INFO:tasks.workunit.client.1.smithi078.stdout:3/885: rename d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d78/dd1 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130 0 2024-04-26T04:53:06.056 INFO:tasks.workunit.client.0.smithi077.stdout:5/456: mknod d1/d30/d33/d66/caa 0 2024-04-26T04:53:06.057 INFO:tasks.workunit.client.1.smithi078.stdout:3/886: read d16/d26/f7f [5419330,17953] 0 2024-04-26T04:53:06.059 INFO:tasks.workunit.client.0.smithi077.stdout:7/484: creat da/d6b/fa4 x:0 0 0 2024-04-26T04:53:06.061 INFO:tasks.workunit.client.0.smithi077.stdout:7/485: write da/d13/d22/f58 [152455,129276] 0 2024-04-26T04:53:06.063 INFO:tasks.workunit.client.0.smithi077.stdout:6/505: creat d8/d52/d53/d5e/d6c/faf x:0 0 0 2024-04-26T04:53:06.064 INFO:tasks.workunit.client.0.smithi077.stdout:2/475: dwrite d9/dc/d1d/f4a [0,4194304] 0 2024-04-26T04:53:06.066 INFO:tasks.workunit.client.0.smithi077.stdout:6/506: truncate d8/f27 1223635 0 2024-04-26T04:53:06.066 INFO:tasks.workunit.client.0.smithi077.stdout:2/476: chown d9/dc/d5f 12767 1 2024-04-26T04:53:06.078 INFO:tasks.workunit.client.0.smithi077.stdout:8/488: dread d4/f10 [0,4194304] 0 2024-04-26T04:53:06.078 INFO:tasks.workunit.client.0.smithi077.stdout:8/489: chown d4/d87/f9b 5866375 1 2024-04-26T04:53:06.079 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:05 smithi078 ceph-mon[33746]: pgmap v22: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 64 MiB/s rd, 200 MiB/s wr, 386 op/s 2024-04-26T04:53:06.080 INFO:tasks.workunit.client.1.smithi078.stdout:5/840: dwrite d7/da/f93 [0,4194304] 0 2024-04-26T04:53:06.080 INFO:tasks.workunit.client.0.smithi077.stdout:1/446: truncate d9/d53/d6e/f30 185202 0 2024-04-26T04:53:06.082 INFO:tasks.workunit.client.0.smithi077.stdout:0/390: link d0/f2 d0/d19/d1e/f89 0 2024-04-26T04:53:06.084 INFO:tasks.workunit.client.0.smithi077.stdout:4/466: write d4/d17/d18/d43/d6e/f86 [67923,113979] 0 2024-04-26T04:53:06.088 INFO:tasks.workunit.client.1.smithi078.stdout:2/879: unlink d8/d11/d18/dfe/f100 0 2024-04-26T04:53:06.089 INFO:tasks.workunit.client.1.smithi078.stdout:2/880: read - d8/d11/d14/d69/d8f/d34/d70/f103 zero size 2024-04-26T04:53:06.090 INFO:tasks.workunit.client.0.smithi077.stdout:2/477: dread d9/dc/d1d/d34/d39/d63/d79/f1c [0,4194304] 0 2024-04-26T04:53:06.090 INFO:tasks.workunit.client.0.smithi077.stdout:5/457: mknod d1/d30/d33/d66/cab 0 2024-04-26T04:53:06.092 INFO:tasks.workunit.client.1.smithi078.stdout:8/859: rename d0/d19/fa1 to d0/d19/d52/d35/d44/d6b/d6c/f115 0 2024-04-26T04:53:06.094 INFO:tasks.workunit.client.1.smithi078.stdout:9/802: creat dc/d55/de2/fef x:0 0 0 2024-04-26T04:53:06.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/486: unlink da/d15/d42/c8a 0 2024-04-26T04:53:06.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/487: chown da/d6b/fa4 1426782 1 2024-04-26T04:53:06.096 INFO:tasks.workunit.client.1.smithi078.stdout:3/887: creat d16/d26/d2e/d5f/d49/dad/df0/f131 x:0 0 0 2024-04-26T04:53:06.097 INFO:tasks.workunit.client.1.smithi078.stdout:9/803: dread dc/f60 [0,4194304] 0 2024-04-26T04:53:06.097 INFO:tasks.workunit.client.0.smithi077.stdout:6/507: truncate d8/d2a/d86/f94 1040875 0 2024-04-26T04:53:06.098 INFO:tasks.workunit.client.1.smithi078.stdout:0/888: rmdir d2/d9/d60/d6a/da5/d12a 0 2024-04-26T04:53:06.102 INFO:tasks.workunit.client.0.smithi077.stdout:8/490: creat d4/df/d18/d56/fa6 x:0 0 0 2024-04-26T04:53:06.103 INFO:tasks.workunit.client.1.smithi078.stdout:6/933: mknod d2/de/d27/da9/da/c12f 0 2024-04-26T04:53:06.105 INFO:tasks.workunit.client.1.smithi078.stdout:7/834: creat d9/da/d11/d13/d1a/d65/f114 x:0 0 0 2024-04-26T04:53:06.106 INFO:tasks.workunit.client.1.smithi078.stdout:7/835: chown d9/da/d11/d13/d50/d8a/fea 16118130 1 2024-04-26T04:53:06.107 INFO:tasks.workunit.client.0.smithi077.stdout:0/391: rename d0/d28/d40/l60 to d0/d19/d1e/d59/d5a/l8a 0 2024-04-26T04:53:06.108 INFO:tasks.workunit.client.1.smithi078.stdout:7/836: read d9/da/d11/d13/d1a/d2d/f3f [4165477,49392] 0 2024-04-26T04:53:06.108 INFO:tasks.workunit.client.1.smithi078.stdout:1/927: mknod d7/d1e/d9e/dd/d20/c12d 0 2024-04-26T04:53:06.108 INFO:tasks.workunit.client.1.smithi078.stdout:7/837: write d9/da/faf [1182467,120943] 0 2024-04-26T04:53:06.109 INFO:tasks.workunit.client.0.smithi077.stdout:4/467: unlink d4/d17/d18/c4c 0 2024-04-26T04:53:06.112 INFO:tasks.workunit.client.0.smithi077.stdout:3/364: creat d0/d7/d1f/f72 x:0 0 0 2024-04-26T04:53:06.114 INFO:tasks.workunit.client.1.smithi078.stdout:5/841: read - d7/d18/d32/fbe zero size 2024-04-26T04:53:06.116 INFO:tasks.workunit.client.1.smithi078.stdout:7/838: dwrite d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fe9 [4194304,4194304] 0 2024-04-26T04:53:06.117 INFO:tasks.workunit.client.1.smithi078.stdout:4/874: mknod d0/de/d22/d27/d7b/c135 0 2024-04-26T04:53:06.119 INFO:tasks.workunit.client.0.smithi077.stdout:2/478: symlink d9/dc/d1d/d1e/l9e 0 2024-04-26T04:53:06.122 INFO:tasks.workunit.client.0.smithi077.stdout:3/365: dwrite d0/d9/d32/d2a/f6b [0,4194304] 0 2024-04-26T04:53:06.123 INFO:tasks.workunit.client.0.smithi077.stdout:3/366: readlink d0/d7/d1f/l55 0 2024-04-26T04:53:06.127 INFO:tasks.workunit.client.0.smithi077.stdout:7/488: symlink da/d13/d22/d5d/la5 0 2024-04-26T04:53:06.128 INFO:tasks.workunit.client.0.smithi077.stdout:1/447: dread d9/f43 [0,4194304] 0 2024-04-26T04:53:06.133 INFO:tasks.workunit.client.0.smithi077.stdout:8/491: fsync d4/df/d22/d2d/f30 0 2024-04-26T04:53:06.134 INFO:tasks.workunit.client.0.smithi077.stdout:8/492: read - d4/d17/d35/d5b/f8b zero size 2024-04-26T04:53:06.135 INFO:tasks.workunit.client.1.smithi078.stdout:8/860: fdatasync d0/d19/d25/d32/d8a/f57 0 2024-04-26T04:53:06.135 INFO:tasks.workunit.client.0.smithi077.stdout:5/458: write d1/d42/da4/d25/f4b [1125811,91449] 0 2024-04-26T04:53:06.137 INFO:tasks.workunit.client.0.smithi077.stdout:5/459: truncate d1/fa5 503842 0 2024-04-26T04:53:06.139 INFO:tasks.workunit.client.0.smithi077.stdout:6/508: rename d8/d52/da0/f75 to d8/d52/d53/d5e/fb0 0 2024-04-26T04:53:06.139 INFO:tasks.workunit.client.0.smithi077.stdout:1/448: dread d9/db/dd/f74 [0,4194304] 0 2024-04-26T04:53:06.139 INFO:tasks.workunit.client.0.smithi077.stdout:6/509: fsync d8/f5a 0 2024-04-26T04:53:06.141 INFO:tasks.workunit.client.0.smithi077.stdout:6/510: write d8/d55/f76 [902658,31563] 0 2024-04-26T04:53:06.141 INFO:tasks.workunit.client.1.smithi078.stdout:3/888: symlink d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/l132 0 2024-04-26T04:53:06.142 INFO:tasks.workunit.client.0.smithi077.stdout:6/511: stat d8/d52/da0/f77 0 2024-04-26T04:53:06.145 INFO:tasks.workunit.client.1.smithi078.stdout:0/889: mknod d2/d1e/c12d 0 2024-04-26T04:53:06.145 INFO:tasks.workunit.client.1.smithi078.stdout:2/881: write d8/d11/d14/d69/d8f/fba [236128,105234] 0 2024-04-26T04:53:06.146 INFO:tasks.workunit.client.1.smithi078.stdout:0/890: chown d2/d9/d78/deb/f101 8 1 2024-04-26T04:53:06.147 INFO:tasks.workunit.client.1.smithi078.stdout:2/882: chown d8/d11/d14/d69/d8f/d34/d70 3157512 1 2024-04-26T04:53:06.147 INFO:tasks.workunit.client.0.smithi077.stdout:9/465: getdents d9/dd 0 2024-04-26T04:53:06.149 INFO:tasks.workunit.client.0.smithi077.stdout:2/479: stat d9/dc/d1d/d34/d39/d63/c7d 0 2024-04-26T04:53:06.150 INFO:tasks.workunit.client.0.smithi077.stdout:2/480: readlink d9/dc/d1d/d34/d39/d63/d79/l6f 0 2024-04-26T04:53:06.151 INFO:tasks.workunit.client.0.smithi077.stdout:3/367: rmdir d0/d7/d1f 39 2024-04-26T04:53:06.152 INFO:tasks.workunit.client.0.smithi077.stdout:7/489: fsync da/d15/f32 0 2024-04-26T04:53:06.155 INFO:tasks.workunit.client.1.smithi078.stdout:5/842: dread - d7/d48/d57/d65/d7a/d9c/d104/fdb zero size 2024-04-26T04:53:06.159 INFO:tasks.workunit.client.0.smithi077.stdout:8/493: dread d4/df/d22/d2d/d2b/f54 [0,4194304] 0 2024-04-26T04:53:06.160 INFO:tasks.workunit.client.1.smithi078.stdout:4/875: rmdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85 39 2024-04-26T04:53:06.162 INFO:tasks.workunit.client.0.smithi077.stdout:5/460: rmdir d1/d42/da4/d25/d31/d3c 39 2024-04-26T04:53:06.164 INFO:tasks.workunit.client.0.smithi077.stdout:0/392: symlink d0/d47/l8b 0 2024-04-26T04:53:06.165 INFO:tasks.workunit.client.1.smithi078.stdout:7/839: write d9/da/d20/f9b [820812,76952] 0 2024-04-26T04:53:06.165 INFO:tasks.workunit.client.0.smithi077.stdout:9/466: dread f2 [0,4194304] 0 2024-04-26T04:53:06.165 INFO:tasks.workunit.client.1.smithi078.stdout:8/861: stat d0/f23 0 2024-04-26T04:53:06.166 INFO:tasks.workunit.client.0.smithi077.stdout:9/467: readlink d9/d3a/l9b 0 2024-04-26T04:53:06.168 INFO:tasks.workunit.client.0.smithi077.stdout:4/468: symlink d4/la0 0 2024-04-26T04:53:06.169 INFO:tasks.workunit.client.1.smithi078.stdout:9/804: unlink dc/df/d4b/d22/dbe/led 0 2024-04-26T04:53:06.170 INFO:tasks.workunit.client.0.smithi077.stdout:4/469: fdatasync d4/d17/d18/d43/d6e/f86 0 2024-04-26T04:53:06.172 INFO:tasks.workunit.client.1.smithi078.stdout:6/934: mknod d2/d1a/c130 0 2024-04-26T04:53:06.172 INFO:tasks.workunit.client.1.smithi078.stdout:6/935: dread - d2/de/d27/da9/d3a/d42/f108 zero size 2024-04-26T04:53:06.173 INFO:tasks.workunit.client.1.smithi078.stdout:6/936: chown d2/de/d27/da9/da/d35/fae 5987877 1 2024-04-26T04:53:06.173 INFO:tasks.workunit.client.0.smithi077.stdout:0/393: dwrite d0/d19/d1e/d59/f6e [0,4194304] 0 2024-04-26T04:53:06.174 INFO:tasks.workunit.client.1.smithi078.stdout:0/891: unlink d2/d9/d31/f8d 0 2024-04-26T04:53:06.176 INFO:tasks.workunit.client.0.smithi077.stdout:1/449: write d9/db/dd/f3e [2025918,89079] 0 2024-04-26T04:53:06.177 INFO:tasks.workunit.client.0.smithi077.stdout:1/450: readlink d9/db/dd/l17 0 2024-04-26T04:53:06.183 INFO:tasks.workunit.client.1.smithi078.stdout:5/843: creat d7/d48/d57/d65/d7a/d9c/d104/d96/f11e x:0 0 0 2024-04-26T04:53:06.185 INFO:tasks.workunit.client.0.smithi077.stdout:1/451: dread d9/db/f60 [0,4194304] 0 2024-04-26T04:53:06.185 INFO:tasks.workunit.client.0.smithi077.stdout:8/494: dread - d4/df/d22/d2d/f30 zero size 2024-04-26T04:53:06.187 INFO:tasks.workunit.client.0.smithi077.stdout:5/461: dread - d1/d42/da4/d25/d31/f71 zero size 2024-04-26T04:53:06.188 INFO:tasks.workunit.client.0.smithi077.stdout:9/468: mkdir d9/dd/d1b/d2b/d32/d36/d7f/da5 0 2024-04-26T04:53:06.192 INFO:tasks.workunit.client.1.smithi078.stdout:9/805: creat dc/df/d4b/d22/d2a/d92/ff0 x:0 0 0 2024-04-26T04:53:06.193 INFO:tasks.workunit.client.0.smithi077.stdout:8/495: dwrite d4/df/d22/d2d/d2b/f4d [4194304,4194304] 0 2024-04-26T04:53:06.193 INFO:tasks.workunit.client.0.smithi077.stdout:4/470: stat d4/lb 0 2024-04-26T04:53:06.194 INFO:tasks.workunit.client.0.smithi077.stdout:8/496: write d4/d17/d35/f52 [6144159,8743] 0 2024-04-26T04:53:06.198 INFO:tasks.workunit.client.0.smithi077.stdout:4/471: dread d4/d17/f3d [0,4194304] 0 2024-04-26T04:53:06.199 INFO:tasks.workunit.client.0.smithi077.stdout:6/512: rename d8/l1a to d8/d52/d53/d5e/lb1 0 2024-04-26T04:53:06.199 INFO:tasks.workunit.client.0.smithi077.stdout:7/490: rename da to da/d13/d22/d5d/da6 22 2024-04-26T04:53:06.200 INFO:tasks.workunit.client.0.smithi077.stdout:7/491: read da/f87 [436593,99468] 0 2024-04-26T04:53:06.201 INFO:tasks.workunit.client.1.smithi078.stdout:6/937: dread d2/fe1 [0,4194304] 0 2024-04-26T04:53:06.202 INFO:tasks.workunit.client.1.smithi078.stdout:8/862: dread d0/f10 [0,4194304] 0 2024-04-26T04:53:06.203 INFO:tasks.workunit.client.1.smithi078.stdout:7/840: write d9/da/d33/d4b/d4c/d6c/fcf [1006665,116422] 0 2024-04-26T04:53:06.204 INFO:tasks.workunit.client.1.smithi078.stdout:8/863: fdatasync d0/d19/d25/d32/d8a/d92/da4/fc6 0 2024-04-26T04:53:06.204 INFO:tasks.workunit.client.1.smithi078.stdout:6/938: write d2/de/d27/da9/da/d36/fe7 [3139838,121476] 0 2024-04-26T04:53:06.207 INFO:tasks.workunit.client.1.smithi078.stdout:1/928: rename d7/d1e/d9e/db/d37/d58/d5c/d68/de8 to d7/d1e/d9e/dd/d20/d12e 0 2024-04-26T04:53:06.207 INFO:tasks.workunit.client.1.smithi078.stdout:0/892: dread d2/f48 [0,4194304] 0 2024-04-26T04:53:06.207 INFO:tasks.workunit.client.1.smithi078.stdout:6/939: dread d2/de/d27/da9/da/d35/fae [0,4194304] 0 2024-04-26T04:53:06.207 INFO:tasks.workunit.client.0.smithi077.stdout:3/368: symlink d0/d9/d32/l73 0 2024-04-26T04:53:06.209 INFO:tasks.workunit.client.1.smithi078.stdout:4/876: truncate d0/d1d/de1/d20/fa5 590112 0 2024-04-26T04:53:06.213 INFO:tasks.workunit.client.0.smithi077.stdout:5/462: mkdir d1/d42/dac 0 2024-04-26T04:53:06.225 INFO:tasks.workunit.client.1.smithi078.stdout:7/841: symlink d9/da/d33/d4b/d4c/d6c/d109/l115 0 2024-04-26T04:53:06.227 INFO:tasks.workunit.client.0.smithi077.stdout:4/472: symlink d4/d17/d18/d43/d53/d64/la1 0 2024-04-26T04:53:06.227 INFO:tasks.workunit.client.0.smithi077.stdout:1/452: write d9/f44 [310859,66983] 0 2024-04-26T04:53:06.231 INFO:tasks.workunit.client.1.smithi078.stdout:7/842: dwrite d9/da/d11/d13/d1a/d2d/f3d [4194304,4194304] 0 2024-04-26T04:53:06.233 INFO:tasks.workunit.client.1.smithi078.stdout:7/843: write d9/da/d11/d13/d1a/d52/d5a/f44 [2937348,16478] 0 2024-04-26T04:53:06.234 INFO:tasks.workunit.client.0.smithi077.stdout:7/492: mkdir da/d13/d22/d5d/d9f/da7 0 2024-04-26T04:53:06.238 INFO:tasks.workunit.client.0.smithi077.stdout:8/497: dread d4/df/d22/d2d/f3d [0,4194304] 0 2024-04-26T04:53:06.241 INFO:tasks.workunit.client.0.smithi077.stdout:6/513: write d8/d2a/f34 [1280209,60522] 0 2024-04-26T04:53:06.241 INFO:tasks.workunit.client.0.smithi077.stdout:2/481: creat d9/d15/f9f x:0 0 0 2024-04-26T04:53:06.242 INFO:tasks.workunit.client.1.smithi078.stdout:3/889: getdents d16 0 2024-04-26T04:53:06.243 INFO:tasks.workunit.client.1.smithi078.stdout:2/883: getdents d8/d11/d14/d69/d8f/d34/dd3/de4/d52 0 2024-04-26T04:53:06.244 INFO:tasks.workunit.client.1.smithi078.stdout:3/890: chown d16/d7e/f116 165 1 2024-04-26T04:53:06.246 INFO:tasks.workunit.client.0.smithi077.stdout:9/469: fsync d9/d45/f7e 0 2024-04-26T04:53:06.247 INFO:tasks.workunit.client.0.smithi077.stdout:9/470: chown d9/dd/d3d/f5a 3694053 1 2024-04-26T04:53:06.248 INFO:tasks.workunit.client.1.smithi078.stdout:5/844: dwrite d7/d21/f42 [0,4194304] 0 2024-04-26T04:53:06.250 INFO:tasks.workunit.client.0.smithi077.stdout:1/453: truncate d9/db/dd/f5b 951478 0 2024-04-26T04:53:06.250 INFO:tasks.workunit.client.1.smithi078.stdout:6/940: dread d2/de/f2b [0,4194304] 0 2024-04-26T04:53:06.250 INFO:tasks.workunit.client.1.smithi078.stdout:5/845: chown d7/d18/d32/d37/d8f/f6e 13 1 2024-04-26T04:53:06.252 INFO:tasks.workunit.client.1.smithi078.stdout:5/846: read d7/da/f75 [586636,82739] 0 2024-04-26T04:53:06.253 INFO:tasks.workunit.client.1.smithi078.stdout:9/806: dread dc/df/d4b/d22/d2a/d57/fa4 [0,4194304] 0 2024-04-26T04:53:06.254 INFO:tasks.workunit.client.0.smithi077.stdout:0/394: creat d0/d47/f8c x:0 0 0 2024-04-26T04:53:06.254 INFO:tasks.workunit.client.1.smithi078.stdout:9/807: chown dc/df/d27/d81/lc2 835 1 2024-04-26T04:53:06.255 INFO:tasks.workunit.client.0.smithi077.stdout:7/493: mknod da/d15/d1a/d39/ca8 0 2024-04-26T04:53:06.256 INFO:tasks.workunit.client.1.smithi078.stdout:9/808: dread dc/f60 [0,4194304] 0 2024-04-26T04:53:06.257 INFO:tasks.workunit.client.1.smithi078.stdout:1/929: truncate d7/d1e/d9e/db/d49/fb7 2512842 0 2024-04-26T04:53:06.257 INFO:tasks.workunit.client.0.smithi077.stdout:3/369: stat d0/d7/d1f/d4d/f61 0 2024-04-26T04:53:06.258 INFO:tasks.workunit.client.0.smithi077.stdout:9/471: dwrite d9/f8b [0,4194304] 0 2024-04-26T04:53:06.260 INFO:tasks.workunit.client.0.smithi077.stdout:5/463: dwrite d1/d42/da4/d25/d31/f71 [0,4194304] 0 2024-04-26T04:53:06.260 INFO:tasks.workunit.client.0.smithi077.stdout:8/498: creat d4/df/d22/d2d/d2b/d33/fa7 x:0 0 0 2024-04-26T04:53:06.262 INFO:tasks.workunit.client.0.smithi077.stdout:9/472: read - d9/d7d/fa1 zero size 2024-04-26T04:53:06.265 INFO:tasks.workunit.client.0.smithi077.stdout:2/482: creat d9/dc/d1d/d34/d39/d6b/fa0 x:0 0 0 2024-04-26T04:53:06.266 INFO:tasks.workunit.client.0.smithi077.stdout:2/483: chown d9/dc/d1d/c29 106386357 1 2024-04-26T04:53:06.270 INFO:tasks.workunit.client.1.smithi078.stdout:3/891: dread d16/d26/f32 [0,4194304] 0 2024-04-26T04:53:06.270 INFO:tasks.workunit.client.1.smithi078.stdout:3/892: stat f6 0 2024-04-26T04:53:06.275 INFO:tasks.workunit.client.1.smithi078.stdout:8/864: creat d0/f116 x:0 0 0 2024-04-26T04:53:06.276 INFO:tasks.workunit.client.0.smithi077.stdout:1/454: rmdir d9/d53 39 2024-04-26T04:53:06.280 INFO:tasks.workunit.client.1.smithi078.stdout:5/847: mknod d7/d48/da9/deb/dfe/c11f 0 2024-04-26T04:53:06.281 INFO:tasks.workunit.client.0.smithi077.stdout:0/395: fsync d0/d2c/d4e/f14 0 2024-04-26T04:53:06.282 INFO:tasks.workunit.client.1.smithi078.stdout:5/848: fdatasync d7/da/d1e/d91/fd4 0 2024-04-26T04:53:06.282 INFO:tasks.workunit.client.1.smithi078.stdout:7/844: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fb8 [0,4194304] 0 2024-04-26T04:53:06.283 INFO:tasks.workunit.client.0.smithi077.stdout:3/370: mknod d0/d9/d32/d29/c74 0 2024-04-26T04:53:06.283 INFO:tasks.workunit.client.1.smithi078.stdout:5/849: fsync d7/d48/d57/d65/d7a/d9c/d104/fe5 0 2024-04-26T04:53:06.285 INFO:tasks.workunit.client.0.smithi077.stdout:8/499: unlink d4/df/d22/l66 0 2024-04-26T04:53:06.288 INFO:tasks.workunit.client.0.smithi077.stdout:8/500: write d4/df/d22/d2d/d2b/f4d [6472639,2127] 0 2024-04-26T04:53:06.289 INFO:tasks.workunit.client.0.smithi077.stdout:8/501: readlink d4/df/d22/l99 0 2024-04-26T04:53:06.290 INFO:tasks.workunit.client.1.smithi078.stdout:1/930: chown d7/d1e/d9e/db/d49/dba/d87/l126 20979337 1 2024-04-26T04:53:06.291 INFO:tasks.workunit.client.1.smithi078.stdout:0/893: sync 2024-04-26T04:53:06.293 INFO:tasks.workunit.client.1.smithi078.stdout:1/931: write d7/d1e/d9e/db/d49/d93/f109 [769451,21621] 0 2024-04-26T04:53:06.294 INFO:tasks.workunit.client.0.smithi077.stdout:9/473: mknod d9/d35/ca6 0 2024-04-26T04:53:06.296 INFO:tasks.workunit.client.1.smithi078.stdout:6/941: dwrite d2/d1a/d38/dd5/ddf/f124 [4194304,4194304] 0 2024-04-26T04:53:06.298 INFO:tasks.workunit.client.1.smithi078.stdout:3/893: creat d16/d26/d2e/f133 x:0 0 0 2024-04-26T04:53:06.298 INFO:tasks.workunit.client.1.smithi078.stdout:3/894: stat d16/d26/d2e/d5f/d49/d77/fc6 0 2024-04-26T04:53:06.307 INFO:tasks.workunit.client.1.smithi078.stdout:8/865: unlink d0/d19/d52/c15 0 2024-04-26T04:53:06.308 INFO:tasks.workunit.client.1.smithi078.stdout:8/866: fsync d0/fe4 0 2024-04-26T04:53:06.309 INFO:tasks.workunit.client.0.smithi077.stdout:5/464: dwrite d1/d8/f55 [0,4194304] 0 2024-04-26T04:53:06.310 INFO:tasks.workunit.client.1.smithi078.stdout:8/867: chown d0/d19/d25/de0/cf9 140168 1 2024-04-26T04:53:06.310 INFO:tasks.workunit.client.0.smithi077.stdout:0/396: mkdir d0/d19/d8d 0 2024-04-26T04:53:06.310 INFO:tasks.workunit.client.0.smithi077.stdout:0/397: chown d0/d28/d40 5919777 1 2024-04-26T04:53:06.311 INFO:tasks.workunit.client.1.smithi078.stdout:5/850: dread d7/da/f62 [0,4194304] 0 2024-04-26T04:53:06.312 INFO:tasks.workunit.client.0.smithi077.stdout:6/514: rename d8/d52/d53/c82 to d8/d2a/cb2 0 2024-04-26T04:53:06.313 INFO:tasks.workunit.client.0.smithi077.stdout:2/484: dwrite d9/dc/d1d/d1e/f13 [0,4194304] 0 2024-04-26T04:53:06.314 INFO:tasks.workunit.client.1.smithi078.stdout:7/845: chown d9/da/d11/d13/f91 1308263670 1 2024-04-26T04:53:06.314 INFO:tasks.workunit.client.0.smithi077.stdout:6/515: dread - d8/d55/f85 zero size 2024-04-26T04:53:06.315 INFO:tasks.workunit.client.0.smithi077.stdout:3/371: mknod d0/d7/d1f/d4d/c75 0 2024-04-26T04:53:06.320 INFO:tasks.workunit.client.1.smithi078.stdout:4/877: getdents d0/de/d22 0 2024-04-26T04:53:06.322 INFO:tasks.workunit.client.1.smithi078.stdout:0/894: creat d2/d9/d31/f12e x:0 0 0 2024-04-26T04:53:06.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.321+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd580000bf0 con 0x7fd58806b790 2024-04-26T04:53:06.323 INFO:tasks.workunit.client.1.smithi078.stdout:1/932: creat d7/d1e/d9e/db/d49/dba/d87/db5/f12f x:0 0 0 2024-04-26T04:53:06.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.323+0000 7fd5927fc700 1 -- 172.21.15.77:0/3914203429 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd580000bf0 con 0x7fd58806b790 2024-04-26T04:53:06.328 INFO:tasks.workunit.client.0.smithi077.stdout:9/474: fdatasync d9/d3a/f6e 0 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 msgr2=0x7fd58806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 0x7fd58806dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd598006fd0 tx=0x7fd598006db0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 msgr2=0x7fd59c10c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009fd0 tx=0x7fd58c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 shutdown_connections 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd58806b790 0x7fd58806dc40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c0fbd40 0x7fd59c10c8e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 --2- 172.21.15.77:0/3914203429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c0fcff0 0x7fd59c10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 >> 172.21.15.77:0/3914203429 conn(0x7fd59c0f74c0 msgr2=0x7fd59c0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:06.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.327+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 shutdown_connections 2024-04-26T04:53:06.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.328+0000 7fd5a3ff8700 1 -- 172.21.15.77:0/3914203429 wait complete. 2024-04-26T04:53:06.330 INFO:tasks.workunit.client.1.smithi078.stdout:6/942: creat d2/de/d27/da9/d2a/dbf/df6/f131 x:0 0 0 2024-04-26T04:53:06.332 INFO:tasks.workunit.client.0.smithi077.stdout:9/475: dread d9/f8b [0,4194304] 0 2024-04-26T04:53:06.335 INFO:tasks.workunit.client.1.smithi078.stdout:6/943: dwrite d2/d1a/d38/f79 [0,4194304] 0 2024-04-26T04:53:06.336 INFO:tasks.workunit.client.0.smithi077.stdout:4/473: link d4/de/c30 d4/de/d15/d1f/d25/ca2 0 2024-04-26T04:53:06.336 INFO:tasks.workunit.client.1.smithi078.stdout:2/884: rename d8/d11/d14/d69/d8f to d8/d11/d18/d137 0 2024-04-26T04:53:06.337 INFO:tasks.workunit.client.1.smithi078.stdout:3/895: fdatasync d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f82 0 2024-04-26T04:53:06.337 INFO:tasks.workunit.client.1.smithi078.stdout:8/868: dread d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:53:06.340 INFO:tasks.workunit.client.1.smithi078.stdout:8/869: chown d0/d19/d25/d32/d8a/d92/da4/da7 160389992 1 2024-04-26T04:53:06.348 INFO:tasks.workunit.client.0.smithi077.stdout:5/465: symlink d1/d8/lad 0 2024-04-26T04:53:06.350 INFO:tasks.workunit.client.1.smithi078.stdout:5/851: fsync d7/da/f75 0 2024-04-26T04:53:06.353 INFO:tasks.workunit.client.1.smithi078.stdout:9/809: link dc/df/d4b/d3e/c56 dc/df/d4b/dae/cf1 0 2024-04-26T04:53:06.356 INFO:tasks.workunit.client.0.smithi077.stdout:6/516: creat d8/d52/fb3 x:0 0 0 2024-04-26T04:53:06.360 INFO:tasks.workunit.client.0.smithi077.stdout:8/502: mknod d4/d17/d90/ca8 0 2024-04-26T04:53:06.361 INFO:tasks.workunit.client.0.smithi077.stdout:7/494: link da/d15/d1a/d39/l62 da/d15/d42/la9 0 2024-04-26T04:53:06.362 INFO:tasks.workunit.client.1.smithi078.stdout:6/944: symlink d2/de/d27/da9/da/dab/l132 0 2024-04-26T04:53:06.365 INFO:tasks.workunit.client.1.smithi078.stdout:4/878: dread d0/f93 [0,4194304] 0 2024-04-26T04:53:06.366 INFO:tasks.workunit.client.0.smithi077.stdout:8/503: dwrite d4/fe [0,4194304] 0 2024-04-26T04:53:06.367 INFO:tasks.workunit.client.0.smithi077.stdout:8/504: chown d4/df/f8c 45493728 1 2024-04-26T04:53:06.367 INFO:tasks.workunit.client.1.smithi078.stdout:2/885: rmdir d8/d11/d18/dfe 39 2024-04-26T04:53:06.372 INFO:tasks.workunit.client.0.smithi077.stdout:4/474: mkdir d4/d17/d7f/da3 0 2024-04-26T04:53:06.374 INFO:tasks.workunit.client.1.smithi078.stdout:8/870: symlink d0/d19/d52/d5c/df0/l117 0 2024-04-26T04:53:06.374 INFO:tasks.workunit.client.0.smithi077.stdout:0/398: dread d0/d2c/f2e [0,4194304] 0 2024-04-26T04:53:06.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/399: chown d0/c5b 1137915889 1 2024-04-26T04:53:06.375 INFO:tasks.workunit.client.0.smithi077.stdout:1/455: creat d9/d53/d7d/f7e x:0 0 0 2024-04-26T04:53:06.380 INFO:tasks.workunit.client.0.smithi077.stdout:5/466: creat d1/d30/d33/d66/fae x:0 0 0 2024-04-26T04:53:06.387 INFO:tasks.workunit.client.0.smithi077.stdout:5/467: dwrite d1/d42/da4/d25/d31/f92 [0,4194304] 0 2024-04-26T04:53:06.388 INFO:tasks.workunit.client.0.smithi077.stdout:5/468: stat d1/d12/d45 0 2024-04-26T04:53:06.389 INFO:tasks.workunit.client.0.smithi077.stdout:3/372: mkdir d0/d7/d1f/d76 0 2024-04-26T04:53:06.389 INFO:tasks.workunit.client.0.smithi077.stdout:5/469: chown d1/d42/da4/c23 389275 1 2024-04-26T04:53:06.389 INFO:tasks.workunit.client.0.smithi077.stdout:3/373: readlink d0/d7/l10 0 2024-04-26T04:53:06.391 INFO:tasks.workunit.client.1.smithi078.stdout:0/895: mknod d2/d9/d31/d4a/d11c/c12f 0 2024-04-26T04:53:06.394 INFO:tasks.workunit.client.1.smithi078.stdout:3/896: dwrite d16/d26/d2e/d5f/d49/dad/db0/fef [0,4194304] 0 2024-04-26T04:53:06.396 INFO:tasks.workunit.client.1.smithi078.stdout:1/933: symlink d7/d1e/d9e/db/d49/dba/d87/d119/l130 0 2024-04-26T04:53:06.396 INFO:tasks.workunit.client.1.smithi078.stdout:3/897: write d16/d26/d2e/d5f/d49/d77/fc6 [6202874,9936] 0 2024-04-26T04:53:06.397 INFO:tasks.workunit.client.0.smithi077.stdout:2/485: write d9/dc/f41 [1623748,19071] 0 2024-04-26T04:53:06.399 INFO:tasks.workunit.client.1.smithi078.stdout:7/846: rename d9/da/d11/d13/c18 to d9/dfd/c116 0 2024-04-26T04:53:06.401 INFO:tasks.workunit.client.1.smithi078.stdout:4/879: symlink d0/d1d/de1/dd8/l136 0 2024-04-26T04:53:06.402 INFO:tasks.workunit.client.0.smithi077.stdout:9/476: rename d9/f8b to d9/dd/d1b/d2b/d32/d36/fa7 0 2024-04-26T04:53:06.403 INFO:tasks.workunit.client.0.smithi077.stdout:4/475: write d4/d17/d18/d43/f74 [1403091,28282] 0 2024-04-26T04:53:06.404 INFO:tasks.workunit.client.0.smithi077.stdout:9/477: chown d9/dd/d1b/d2b/d32/d36/l3f 246603263 1 2024-04-26T04:53:06.406 INFO:tasks.workunit.client.0.smithi077.stdout:6/517: symlink d8/d2a/lb4 0 2024-04-26T04:53:06.407 INFO:tasks.workunit.client.1.smithi078.stdout:5/852: creat d7/d11/dee/d111/f120 x:0 0 0 2024-04-26T04:53:06.409 INFO:tasks.workunit.client.1.smithi078.stdout:9/810: symlink dc/df/d4b/d22/d2a/d57/d93/lf2 0 2024-04-26T04:53:06.410 INFO:tasks.workunit.client.1.smithi078.stdout:9/811: read dc/df/d4b/d22/f97 [129376,26079] 0 2024-04-26T04:53:06.411 INFO:tasks.workunit.client.1.smithi078.stdout:0/896: fsync f1 0 2024-04-26T04:53:06.412 INFO:tasks.workunit.client.0.smithi077.stdout:8/505: mknod d4/df/d22/d2d/d2b/ca9 0 2024-04-26T04:53:06.414 INFO:tasks.workunit.client.1.smithi078.stdout:9/812: dread dc/fd7 [0,4194304] 0 2024-04-26T04:53:06.415 INFO:tasks.workunit.client.1.smithi078.stdout:1/934: unlink d7/c9a 0 2024-04-26T04:53:06.416 INFO:tasks.workunit.client.1.smithi078.stdout:6/945: write d2/de/d27/f64 [639120,89271] 0 2024-04-26T04:53:06.416 INFO:tasks.workunit.client.1.smithi078.stdout:1/935: read d7/d1e/d9e/db/f5a [676227,113280] 0 2024-04-26T04:53:06.417 INFO:tasks.workunit.client.0.smithi077.stdout:4/476: dread d4/de/d15/f50 [0,4194304] 0 2024-04-26T04:53:06.417 INFO:tasks.workunit.client.0.smithi077.stdout:9/478: fdatasync d9/d43/f56 0 2024-04-26T04:53:06.418 INFO:tasks.workunit.client.0.smithi077.stdout:9/479: chown d9/d35/d99 26821 1 2024-04-26T04:53:06.419 INFO:tasks.workunit.client.0.smithi077.stdout:9/480: truncate d9/f2d 4505828 0 2024-04-26T04:53:06.420 INFO:tasks.workunit.client.0.smithi077.stdout:6/518: readlink d8/l1e 0 2024-04-26T04:53:06.421 INFO:tasks.workunit.client.0.smithi077.stdout:6/519: chown d8/d1c/f25 10 1 2024-04-26T04:53:06.421 INFO:tasks.workunit.client.0.smithi077.stdout:6/520: chown d8/d52/da0/l8b 30358963 1 2024-04-26T04:53:06.424 INFO:tasks.workunit.client.1.smithi078.stdout:3/898: truncate d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d66/fc2 706629 0 2024-04-26T04:53:06.426 INFO:tasks.workunit.client.1.smithi078.stdout:2/886: symlink d8/d11/d18/d137/l138 0 2024-04-26T04:53:06.430 INFO:tasks.workunit.client.0.smithi077.stdout:8/506: write d4/d17/f9a [254642,80196] 0 2024-04-26T04:53:06.434 INFO:tasks.workunit.client.1.smithi078.stdout:5/853: rename d7/d18/d32/d84/l112 to d7/d48/d57/d65/d46/l121 0 2024-04-26T04:53:06.435 INFO:tasks.workunit.client.0.smithi077.stdout:4/477: chown d4/de/d15/d1f/d76/f87 43783 1 2024-04-26T04:53:06.437 INFO:tasks.workunit.client.0.smithi077.stdout:9/481: rmdir d9/dd/d1b 39 2024-04-26T04:53:06.440 INFO:tasks.workunit.client.0.smithi077.stdout:3/374: dwrite d0/d7/f3e [0,4194304] 0 2024-04-26T04:53:06.442 INFO:tasks.workunit.client.0.smithi077.stdout:6/521: truncate d8/f22 820332 0 2024-04-26T04:53:06.442 INFO:tasks.workunit.client.1.smithi078.stdout:0/897: fsync d2/d9/d31/d73/fa7 0 2024-04-26T04:53:06.443 INFO:tasks.workunit.client.0.smithi077.stdout:7/495: getdents da/d6b/d63 0 2024-04-26T04:53:06.444 INFO:tasks.workunit.client.1.smithi078.stdout:9/813: mkdir dc/d55/de2/df3 0 2024-04-26T04:53:06.445 INFO:tasks.workunit.client.0.smithi077.stdout:2/486: dwrite d9/dc/d1d/d34/f84 [0,4194304] 0 2024-04-26T04:53:06.445 INFO:tasks.workunit.client.1.smithi078.stdout:9/814: fdatasync dc/fe6 0 2024-04-26T04:53:06.446 INFO:tasks.workunit.client.0.smithi077.stdout:2/487: truncate d9/f10 5218198 0 2024-04-26T04:53:06.449 INFO:tasks.workunit.client.0.smithi077.stdout:0/400: rename d0/d19/d1e/d81/f83 to d0/d28/f8e 0 2024-04-26T04:53:06.450 INFO:tasks.workunit.client.1.smithi078.stdout:6/946: truncate d2/de/d27/da9/d2a/d4d/fa7 49951 0 2024-04-26T04:53:06.454 INFO:tasks.workunit.client.0.smithi077.stdout:4/478: mknod d4/d17/d18/d43/ca4 0 2024-04-26T04:53:06.455 INFO:tasks.workunit.client.1.smithi078.stdout:3/899: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/f134 x:0 0 0 2024-04-26T04:53:06.458 INFO:tasks.workunit.client.1.smithi078.stdout:2/887: chown d8/d11/d18/d137/c21 324057 1 2024-04-26T04:53:06.459 INFO:tasks.workunit.client.0.smithi077.stdout:6/522: mkdir d8/d1c/db5 0 2024-04-26T04:53:06.466 INFO:tasks.workunit.client.0.smithi077.stdout:8/507: rename d4/l8 to d4/d17/laa 0 2024-04-26T04:53:06.468 INFO:tasks.workunit.client.0.smithi077.stdout:9/482: chown d9/dd/l20 58522840 1 2024-04-26T04:53:06.477 INFO:tasks.workunit.client.1.smithi078.stdout:9/815: fsync dc/df/d17/f88 0 2024-04-26T04:53:06.478 INFO:tasks.workunit.client.0.smithi077.stdout:4/479: dread d4/de/d15/d1f/d35/d5c/f8c [0,4194304] 0 2024-04-26T04:53:06.478 INFO:tasks.workunit.client.0.smithi077.stdout:2/488: mknod d9/ca1 0 2024-04-26T04:53:06.480 INFO:tasks.workunit.client.0.smithi077.stdout:0/401: mknod d0/d28/c8f 0 2024-04-26T04:53:06.481 INFO:tasks.workunit.client.0.smithi077.stdout:0/402: write d0/d19/f2d [745646,88782] 0 2024-04-26T04:53:06.484 INFO:tasks.workunit.client.0.smithi077.stdout:8/508: rename d4/f10 to d4/d17/d90/fab 0 2024-04-26T04:53:06.485 INFO:tasks.workunit.client.1.smithi078.stdout:2/888: rmdir d8/d11/d18/d5d/d127 39 2024-04-26T04:53:06.485 INFO:tasks.workunit.client.1.smithi078.stdout:1/936: truncate d7/f8e 2696564 0 2024-04-26T04:53:06.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 -- 172.21.15.77:0/2098272784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 msgr2=0x7f1554102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 --2- 172.21.15.77:0/2098272784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f1554102300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1544004940 tx=0x7f1544009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:06.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 -- 172.21.15.77:0/2098272784 shutdown_connections 2024-04-26T04:53:06.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 --2- 172.21.15.77:0/2098272784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f1554102300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 --2- 172.21.15.77:0/2098272784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f15540ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 -- 172.21.15.77:0/2098272784 >> 172.21.15.77:0/2098272784 conn(0x7f15540f7430 msgr2=0x7f15540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 -- 172.21.15.77:0/2098272784 shutdown_connections 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.487+0000 7f155b589700 1 -- 172.21.15.77:0/2098272784 wait complete. 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 Processor -- start 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 -- start start 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f155406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155406fd40 con 0x7f15540fff20 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f155b589700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155406feb0 con 0x7f1554066e90 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f1558b24700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f1559325700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f155406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f1559325700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f155406e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:52208/0 (socket says 172.21.15.77:52208) 2024-04-26T04:53:06.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f1558b24700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58338/0 (socket says 172.21.15.77:58338) 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.488+0000 7f1559325700 1 -- 172.21.15.77:0/3689703552 learned_addr learned my addr 172.21.15.77:0/3689703552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1558b24700 1 -- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 msgr2=0x7f155406e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1558b24700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f155406e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1558b24700 1 -- 172.21.15.77:0/3689703552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544005040 con 0x7f15540fff20 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1558b24700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1544009fd0 tx=0x7f1544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1544021070 con 0x7f15540fff20 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155406ee00 con 0x7f15540fff20 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f154401c400 con 0x7f15540fff20 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154400e070 con 0x7f15540fff20 2024-04-26T04:53:06.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.489+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155411df00 con 0x7f15540fff20 2024-04-26T04:53:06.493 INFO:tasks.workunit.client.1.smithi078.stdout:4/880: link d0/d1d/de1/d20/d31/d47/db2/fa3 d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121/f137 0 2024-04-26T04:53:06.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.490+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155404c710 con 0x7f15540fff20 2024-04-26T04:53:06.493 INFO:tasks.workunit.client.0.smithi077.stdout:3/375: link d0/d7/d15/c6c d0/d7/d1f/d76/c77 0 2024-04-26T04:53:06.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.491+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f1544013af0 con 0x7f15540fff20 2024-04-26T04:53:06.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.492+0000 7f1549ffb700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 0x7f154006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:06.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.492+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f154408b540 con 0x7f15540fff20 2024-04-26T04:53:06.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.492+0000 7f1559325700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 0x7f154006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:06.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.492+0000 7f1559325700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 0x7f154006dc50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1550004ba0 tx=0x7f155000b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:06.497 INFO:tasks.workunit.client.1.smithi078.stdout:5/854: fdatasync d7/d18/d1c/f9e 0 2024-04-26T04:53:06.497 INFO:tasks.workunit.client.1.smithi078.stdout:4/881: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/f119 [0,4194304] 0 2024-04-26T04:53:06.498 INFO:tasks.workunit.client.0.smithi077.stdout:6/523: write d8/d2a/f32 [4782128,21653] 0 2024-04-26T04:53:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.499+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f1544058cc0 con 0x7f15540fff20 2024-04-26T04:53:06.503 INFO:tasks.workunit.client.1.smithi078.stdout:0/898: truncate d2/d9/d31/d4a/f5d 416854 0 2024-04-26T04:53:06.505 INFO:tasks.workunit.client.0.smithi077.stdout:2/489: dwrite d9/dc/d1d/d1e/d77/f92 [8388608,4194304] 0 2024-04-26T04:53:06.509 INFO:tasks.workunit.client.1.smithi078.stdout:9/816: fsync dc/df/f1e 0 2024-04-26T04:53:06.510 INFO:tasks.workunit.client.0.smithi077.stdout:9/483: creat d9/dd/d1b/d2b/d32/d36/d7f/fa8 x:0 0 0 2024-04-26T04:53:06.511 INFO:tasks.workunit.client.1.smithi078.stdout:9/817: write dc/df/fd1 [131757,107758] 0 2024-04-26T04:53:06.511 INFO:tasks.workunit.client.1.smithi078.stdout:7/847: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d104 0 2024-04-26T04:53:06.514 INFO:tasks.workunit.client.1.smithi078.stdout:1/937: creat d7/d1e/d9e/db/d49/dba/d87/f131 x:0 0 0 2024-04-26T04:53:06.515 INFO:tasks.workunit.client.1.smithi078.stdout:1/938: chown d7/d1e/d9e/dd/d20/c12d 241073 1 2024-04-26T04:53:06.515 INFO:tasks.workunit.client.0.smithi077.stdout:7/496: getdents da/d15/d42/d8c 0 2024-04-26T04:53:06.517 INFO:tasks.workunit.client.1.smithi078.stdout:3/900: rename d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/c10f to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130/d123/c135 0 2024-04-26T04:53:06.520 INFO:tasks.workunit.client.0.smithi077.stdout:6/524: stat d8/d2a/cb2 0 2024-04-26T04:53:06.525 INFO:tasks.workunit.client.0.smithi077.stdout:8/509: dwrite d4/d17/f6a [0,4194304] 0 2024-04-26T04:53:06.537 INFO:tasks.workunit.client.0.smithi077.stdout:3/376: creat d0/d9/f78 x:0 0 0 2024-04-26T04:53:06.538 INFO:tasks.workunit.client.0.smithi077.stdout:9/484: mknod d9/dd/d3d/ca9 0 2024-04-26T04:53:06.539 INFO:tasks.workunit.client.0.smithi077.stdout:7/497: dwrite da/d13/d22/f99 [4194304,4194304] 0 2024-04-26T04:53:06.540 INFO:tasks.workunit.client.1.smithi078.stdout:1/939: mknod d7/d1e/d9e/db/d49/dba/d87/d119/c132 0 2024-04-26T04:53:06.541 INFO:tasks.workunit.client.0.smithi077.stdout:6/525: creat d8/d1c/da4/fb6 x:0 0 0 2024-04-26T04:53:06.541 INFO:tasks.workunit.client.1.smithi078.stdout:6/947: sync 2024-04-26T04:53:06.542 INFO:tasks.workunit.client.1.smithi078.stdout:6/948: stat d2/de/d27/da9/d3a/d42/ced 0 2024-04-26T04:53:06.544 INFO:tasks.workunit.client.0.smithi077.stdout:0/403: getdents d0/d19 0 2024-04-26T04:53:06.546 INFO:tasks.workunit.client.1.smithi078.stdout:8/871: link d0/d19/d52/d5c/fcd d0/d19/d52/d35/d44/d6b/d104/d58/d5b/dee/f118 0 2024-04-26T04:53:06.546 INFO:tasks.workunit.client.0.smithi077.stdout:0/404: truncate d0/d47/f8c 987058 0 2024-04-26T04:53:06.547 INFO:tasks.workunit.client.1.smithi078.stdout:8/872: chown d0/f116 2448 1 2024-04-26T04:53:06.551 INFO:tasks.workunit.client.0.smithi077.stdout:6/526: dread d8/d52/d53/f80 [0,4194304] 0 2024-04-26T04:53:06.554 INFO:tasks.workunit.client.1.smithi078.stdout:4/882: mknod d0/d1d/de1/d20/d31/d47/db2/d44/d55/c138 0 2024-04-26T04:53:06.558 INFO:tasks.workunit.client.0.smithi077.stdout:2/490: getdents d9/dc/d5f 0 2024-04-26T04:53:06.559 INFO:tasks.workunit.client.1.smithi078.stdout:9/818: getdents dc/d55/de2/df3 0 2024-04-26T04:53:06.559 INFO:tasks.workunit.client.0.smithi077.stdout:2/491: fdatasync d9/d15/f37 0 2024-04-26T04:53:06.560 INFO:tasks.workunit.client.0.smithi077.stdout:0/405: mkdir d0/d19/d1e/d2f/d90 0 2024-04-26T04:53:06.562 INFO:tasks.workunit.client.0.smithi077.stdout:0/406: truncate d0/d2c/d4e/f4c 940475 0 2024-04-26T04:53:06.563 INFO:tasks.workunit.client.1.smithi078.stdout:7/848: rename d9/da/d11/d13/d50/d8a/d8f to d9/da/d11/d13/d1a/d65/d117 0 2024-04-26T04:53:06.564 INFO:tasks.workunit.client.0.smithi077.stdout:7/498: dread da/d15/d1a/f79 [0,4194304] 0 2024-04-26T04:53:06.565 INFO:tasks.workunit.client.0.smithi077.stdout:7/499: dread - da/d6b/d63/f9e zero size 2024-04-26T04:53:06.566 INFO:tasks.workunit.client.1.smithi078.stdout:2/889: symlink d8/d11/d18/d5d/d127/d130/l139 0 2024-04-26T04:53:06.567 INFO:tasks.workunit.client.1.smithi078.stdout:5/855: write d7/d18/f2e [3697922,75864] 0 2024-04-26T04:53:06.569 INFO:tasks.workunit.client.1.smithi078.stdout:5/856: dread - d7/d48/d57/d65/d7a/d9c/d104/d96/fdd zero size 2024-04-26T04:53:06.571 INFO:tasks.workunit.client.1.smithi078.stdout:5/857: chown d7/d48/d57/d65/d7a/f98 29180 1 2024-04-26T04:53:06.573 INFO:tasks.workunit.client.1.smithi078.stdout:2/890: dwrite d8/d11/d18/d5d/d61/f134 [0,4194304] 0 2024-04-26T04:53:06.573 INFO:tasks.workunit.client.1.smithi078.stdout:5/858: readlink d7/d21/d2f/l3c 0 2024-04-26T04:53:06.574 INFO:tasks.workunit.client.1.smithi078.stdout:5/859: chown d7/d18/f27 244406 1 2024-04-26T04:53:06.576 INFO:tasks.workunit.client.1.smithi078.stdout:6/949: write d2/de/d27/da9/d3a/d69/f99 [896929,15664] 0 2024-04-26T04:53:06.578 INFO:tasks.workunit.client.1.smithi078.stdout:9/819: rename dc/df/d37/cb7 to dc/df/d4b/d3e/d80/cf4 0 2024-04-26T04:53:06.580 INFO:tasks.workunit.client.1.smithi078.stdout:7/849: creat d9/da/d33/d4b/d4c/d5b/d83/de6/f118 x:0 0 0 2024-04-26T04:53:06.581 INFO:tasks.workunit.client.1.smithi078.stdout:1/940: mknod d7/d1e/d9e/db/d37/d58/d5c/d68/c133 0 2024-04-26T04:53:06.584 INFO:tasks.workunit.client.1.smithi078.stdout:5/860: rmdir d7/d18 39 2024-04-26T04:53:06.585 INFO:tasks.workunit.client.1.smithi078.stdout:8/873: link d0/d19/d25/d32/d8a/d92/da4/fed d0/d19/d52/d5c/df0/f119 0 2024-04-26T04:53:06.586 INFO:tasks.workunit.client.1.smithi078.stdout:0/899: getdents d2/d9/d78/d84 0 2024-04-26T04:53:06.588 INFO:tasks.workunit.client.1.smithi078.stdout:9/820: creat dc/df/d4b/dae/ff5 x:0 0 0 2024-04-26T04:53:06.589 INFO:tasks.workunit.client.1.smithi078.stdout:3/901: getdents d16/d3c/d88 0 2024-04-26T04:53:06.590 INFO:tasks.workunit.client.1.smithi078.stdout:1/941: chown d7/d1e/d9e/dd/lad 960877108 1 2024-04-26T04:53:06.591 INFO:tasks.workunit.client.1.smithi078.stdout:1/942: write d7/d1e/d9e/db/f105 [643053,86118] 0 2024-04-26T04:53:06.593 INFO:tasks.workunit.client.1.smithi078.stdout:0/900: mknod d2/d9/d60/c130 0 2024-04-26T04:53:06.595 INFO:tasks.workunit.client.0.smithi077.stdout:7/500: readlink da/d15/d42/l4f 0 2024-04-26T04:53:06.597 INFO:tasks.workunit.client.0.smithi077.stdout:6/527: write d8/d1c/da4/fa8 [754143,75053] 0 2024-04-26T04:53:06.597 INFO:tasks.workunit.client.0.smithi077.stdout:8/510: getdents d4/df/d22/d2d/d2b 0 2024-04-26T04:53:06.598 INFO:tasks.workunit.client.1.smithi078.stdout:4/883: truncate d0/d1d/de1/f19 417525 0 2024-04-26T04:53:06.598 INFO:tasks.workunit.client.0.smithi077.stdout:8/511: chown d4/d17/l45 45630 1 2024-04-26T04:53:06.598 INFO:tasks.workunit.client.0.smithi077.stdout:6/528: dread - d8/d1c/da4/fb6 zero size 2024-04-26T04:53:06.599 INFO:tasks.workunit.client.1.smithi078.stdout:2/891: dread d8/d11/f45 [0,4194304] 0 2024-04-26T04:53:06.600 INFO:tasks.workunit.client.0.smithi077.stdout:8/512: fsync d4/df/d22/d2d/d2b/f7b 0 2024-04-26T04:53:06.600 INFO:tasks.workunit.client.0.smithi077.stdout:3/377: rename d0/d7/f8 to d0/d9/f79 0 2024-04-26T04:53:06.600 INFO:tasks.workunit.client.0.smithi077.stdout:3/378: chown d0/c6e 1388075788 1 2024-04-26T04:53:06.601 INFO:tasks.workunit.client.0.smithi077.stdout:0/407: getdents d0/d19/d8d 0 2024-04-26T04:53:06.602 INFO:tasks.workunit.client.1.smithi078.stdout:7/850: truncate d9/da/d11/d13/d1a/f40 1825124 0 2024-04-26T04:53:06.603 INFO:tasks.workunit.client.1.smithi078.stdout:1/943: fdatasync f3 0 2024-04-26T04:53:06.608 INFO:tasks.workunit.client.1.smithi078.stdout:8/874: dread d0/d19/d52/d5c/fb3 [0,4194304] 0 2024-04-26T04:53:06.608 INFO:tasks.workunit.client.1.smithi078.stdout:9/821: dread dc/d55/f52 [0,4194304] 0 2024-04-26T04:53:06.646 INFO:tasks.workunit.client.1.smithi078.stdout:3/902: dread d16/f3f [0,4194304] 0 2024-04-26T04:53:06.646 INFO:tasks.workunit.client.1.smithi078.stdout:6/950: rename d2/de/d27/da9/d3a/d69/daa/fea to d2/de/d27/da9/d2a/d4d/f133 0 2024-04-26T04:53:06.649 INFO:tasks.workunit.client.1.smithi078.stdout:4/884: fdatasync d0/de/d22/d27/d7b/fdb 0 2024-04-26T04:53:06.650 INFO:tasks.workunit.client.1.smithi078.stdout:2/892: symlink d8/d53/l13a 0 2024-04-26T04:53:06.651 INFO:tasks.workunit.client.1.smithi078.stdout:5/861: dwrite d7/d18/d1c/f7d [0,4194304] 0 2024-04-26T04:53:06.653 INFO:tasks.workunit.client.1.smithi078.stdout:5/862: stat d7/d48/d57/d65/d7a/d9c/d104/d96/f11e 0 2024-04-26T04:53:06.659 INFO:tasks.workunit.client.0.smithi077.stdout:7/501: truncate da/d15/d1a/d39/f7a 2566186 0 2024-04-26T04:53:06.663 INFO:tasks.workunit.client.0.smithi077.stdout:8/513: dread d4/df/f1f [0,4194304] 0 2024-04-26T04:53:06.666 INFO:tasks.workunit.client.0.smithi077.stdout:6/529: getdents d8/d52/d53/d5e/da6 0 2024-04-26T04:53:06.668 INFO:tasks.workunit.client.0.smithi077.stdout:8/514: creat d4/df/d22/d75/fac x:0 0 0 2024-04-26T04:53:06.671 INFO:tasks.workunit.client.0.smithi077.stdout:3/379: write d0/d9/f79 [2217794,89162] 0 2024-04-26T04:53:06.674 INFO:tasks.workunit.client.0.smithi077.stdout:0/408: truncate d0/d47/ff 1127084 0 2024-04-26T04:53:06.676 INFO:tasks.workunit.client.0.smithi077.stdout:0/409: write d0/d2c/d4e/f4c [416718,27330] 0 2024-04-26T04:53:06.679 INFO:tasks.workunit.client.1.smithi078.stdout:9/822: dwrite dc/d55/f52 [0,4194304] 0 2024-04-26T04:53:06.679 INFO:tasks.workunit.client.0.smithi077.stdout:3/380: dwrite d0/d7/d15/f50 [4194304,4194304] 0 2024-04-26T04:53:06.685 INFO:tasks.workunit.client.0.smithi077.stdout:0/410: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:53:06.688 INFO:tasks.workunit.client.0.smithi077.stdout:7/502: truncate da/ff 1687394 0 2024-04-26T04:53:06.692 INFO:tasks.workunit.client.0.smithi077.stdout:8/515: symlink d4/d69/lad 0 2024-04-26T04:53:06.694 INFO:tasks.workunit.client.0.smithi077.stdout:6/530: write d8/d52/d53/d5e/f69 [4622277,33042] 0 2024-04-26T04:53:06.701 INFO:tasks.workunit.client.0.smithi077.stdout:8/516: rename d4/d17/f9a to d4/df/d22/d80/fae 0 2024-04-26T04:53:06.703 INFO:tasks.workunit.client.0.smithi077.stdout:6/531: fdatasync d8/d1c/f28 0 2024-04-26T04:53:06.704 INFO:tasks.workunit.client.0.smithi077.stdout:6/532: truncate d8/d1c/f79 1438271 0 2024-04-26T04:53:06.705 INFO:tasks.workunit.client.0.smithi077.stdout:0/411: symlink d0/d19/d1e/d82/l91 0 2024-04-26T04:53:06.705 INFO:tasks.workunit.client.0.smithi077.stdout:0/412: dread - d0/d2c/d4e/f66 zero size 2024-04-26T04:53:06.711 INFO:tasks.workunit.client.0.smithi077.stdout:6/533: dwrite d8/d2a/f32 [0,4194304] 0 2024-04-26T04:53:06.713 INFO:tasks.workunit.client.0.smithi077.stdout:6/534: chown d8/d52/d53/d5e/d6c/f84 26 1 2024-04-26T04:53:06.715 INFO:tasks.workunit.client.0.smithi077.stdout:8/517: write d4/df/f15 [7329015,18890] 0 2024-04-26T04:53:06.715 INFO:tasks.workunit.client.0.smithi077.stdout:8/518: readlink d4/df/d22/l7d 0 2024-04-26T04:53:06.723 INFO:tasks.workunit.client.0.smithi077.stdout:6/535: rmdir d8/d55 39 2024-04-26T04:53:06.726 INFO:tasks.workunit.client.1.smithi078.stdout:7/851: sync 2024-04-26T04:53:06.727 INFO:tasks.workunit.client.0.smithi077.stdout:8/519: creat d4/d17/d90/faf x:0 0 0 2024-04-26T04:53:06.731 INFO:tasks.workunit.client.0.smithi077.stdout:6/536: dwrite f4 [0,4194304] 0 2024-04-26T04:53:06.734 INFO:tasks.workunit.client.0.smithi077.stdout:0/413: link d0/d47/l6b d0/d19/d1e/d81/l92 0 2024-04-26T04:53:06.741 INFO:tasks.workunit.client.0.smithi077.stdout:3/381: dwrite d0/f19 [0,4194304] 0 2024-04-26T04:53:06.742 INFO:tasks.workunit.client.0.smithi077.stdout:7/503: dread da/ff [0,4194304] 0 2024-04-26T04:53:06.752 INFO:tasks.workunit.client.0.smithi077.stdout:0/414: dread d0/d47/f16 [0,4194304] 0 2024-04-26T04:53:06.754 INFO:tasks.workunit.client.0.smithi077.stdout:8/520: link d4/d17/d35/f70 d4/df/d18/fb0 0 2024-04-26T04:53:06.756 INFO:tasks.workunit.client.1.smithi078.stdout:0/901: mknod d2/d9/d60/dae/de4/c131 0 2024-04-26T04:53:06.756 INFO:tasks.workunit.client.0.smithi077.stdout:8/521: dread - d4/df/d22/d2d/f30 zero size 2024-04-26T04:53:06.758 INFO:tasks.workunit.client.0.smithi077.stdout:8/522: chown d4/f13 12 1 2024-04-26T04:53:06.759 INFO:tasks.workunit.client.0.smithi077.stdout:7/504: fsync da/d13/f6d 0 2024-04-26T04:53:06.759 INFO:tasks.workunit.client.0.smithi077.stdout:0/415: dwrite d0/d47/f50 [0,4194304] 0 2024-04-26T04:53:06.760 INFO:tasks.workunit.client.0.smithi077.stdout:8/523: fsync d4/d17/d35/d5b/fa5 0 2024-04-26T04:53:06.762 INFO:tasks.workunit.client.0.smithi077.stdout:3/382: fsync d0/d9/d32/f3b 0 2024-04-26T04:53:06.763 INFO:tasks.workunit.client.0.smithi077.stdout:7/505: dread da/ff [0,4194304] 0 2024-04-26T04:53:06.764 INFO:tasks.workunit.client.0.smithi077.stdout:7/506: chown da/d15/d1a 9407092 1 2024-04-26T04:53:06.766 INFO:tasks.workunit.client.0.smithi077.stdout:7/507: write da/d13/d22/f99 [586999,109615] 0 2024-04-26T04:53:06.766 INFO:tasks.workunit.client.1.smithi078.stdout:4/885: rename d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/f7a to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/f139 0 2024-04-26T04:53:06.766 INFO:tasks.workunit.client.1.smithi078.stdout:2/893: sync 2024-04-26T04:53:06.768 INFO:tasks.workunit.client.1.smithi078.stdout:5/863: dread - d7/d48/d57/d65/d7a/d9c/fe9 zero size 2024-04-26T04:53:06.784 INFO:tasks.workunit.client.1.smithi078.stdout:9/823: rename dc/df/d4b/d22/l6d to dc/d55/de2/df3/lf6 0 2024-04-26T04:53:06.786 INFO:tasks.workunit.client.1.smithi078.stdout:7/852: creat d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/f119 x:0 0 0 2024-04-26T04:53:06.788 INFO:tasks.workunit.client.1.smithi078.stdout:0/902: dread d2/d9/d60/d6a/f87 [0,4194304] 0 2024-04-26T04:53:06.788 INFO:tasks.workunit.client.0.smithi077.stdout:3/383: dread d0/d9/f30 [4194304,4194304] 0 2024-04-26T04:53:06.788 INFO:tasks.workunit.client.0.smithi077.stdout:6/537: getdents d8/d1c/da4 0 2024-04-26T04:53:06.788 INFO:tasks.workunit.client.0.smithi077.stdout:3/384: write d0/d7/d15/f50 [6250109,23466] 0 2024-04-26T04:53:06.789 INFO:tasks.workunit.client.1.smithi078.stdout:3/903: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/f8c [0,4194304] 0 2024-04-26T04:53:06.790 INFO:tasks.workunit.client.0.smithi077.stdout:7/508: rename da/d15/f78 to da/d15/faa 0 2024-04-26T04:53:06.792 INFO:tasks.workunit.client.1.smithi078.stdout:6/951: mkdir d2/de/d27/da9/d3a/d69/daa/d134 0 2024-04-26T04:53:06.792 INFO:tasks.workunit.client.0.smithi077.stdout:7/509: read da/d15/d42/f77 [2465842,64472] 0 2024-04-26T04:53:06.793 INFO:tasks.workunit.client.0.smithi077.stdout:7/510: chown da/d15/d1a 1505065 1 2024-04-26T04:53:06.793 INFO:tasks.workunit.client.0.smithi077.stdout:0/416: mkdir d0/d47/d93 0 2024-04-26T04:53:06.794 INFO:tasks.workunit.client.0.smithi077.stdout:7/511: write da/d13/d22/f99 [430556,103075] 0 2024-04-26T04:53:06.796 INFO:tasks.workunit.client.1.smithi078.stdout:6/952: read d2/de/d27/da9/d3a/d42/f59 [3469542,78461] 0 2024-04-26T04:53:06.819 INFO:tasks.workunit.client.0.smithi077.stdout:8/524: getdents d4/df/d18 0 2024-04-26T04:53:06.821 INFO:tasks.workunit.client.0.smithi077.stdout:3/385: getdents d0/d9/d32/d29 0 2024-04-26T04:53:06.822 INFO:tasks.workunit.client.0.smithi077.stdout:0/417: write d0/d19/d1e/f89 [810009,17282] 0 2024-04-26T04:53:06.824 INFO:tasks.workunit.client.0.smithi077.stdout:3/386: dread - d0/d9/d32/d29/f71 zero size 2024-04-26T04:53:06.824 INFO:tasks.workunit.client.0.smithi077.stdout:6/538: dwrite d8/f50 [0,4194304] 0 2024-04-26T04:53:06.825 INFO:tasks.workunit.client.0.smithi077.stdout:0/418: chown d0/f2 233813450 1 2024-04-26T04:53:06.826 INFO:tasks.workunit.client.0.smithi077.stdout:7/512: truncate da/d15/d42/d8c/f9b 3359531 0 2024-04-26T04:53:06.826 INFO:tasks.workunit.client.0.smithi077.stdout:3/387: chown d0/d9/f79 17 1 2024-04-26T04:53:06.827 INFO:tasks.workunit.client.0.smithi077.stdout:6/539: write d8/d8f/f9f [687688,47427] 0 2024-04-26T04:53:06.827 INFO:tasks.workunit.client.0.smithi077.stdout:8/525: write d4/df/d22/f68 [1929323,100424] 0 2024-04-26T04:53:06.829 INFO:tasks.workunit.client.0.smithi077.stdout:0/419: mkdir d0/d47/d93/d94 0 2024-04-26T04:53:06.831 INFO:tasks.workunit.client.0.smithi077.stdout:8/526: read d4/df/f15 [8111281,123750] 0 2024-04-26T04:53:06.832 INFO:tasks.workunit.client.0.smithi077.stdout:7/513: mknod da/d13/d22/d5d/cab 0 2024-04-26T04:53:06.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/420: write d0/d19/d1e/d59/f6e [3269580,45463] 0 2024-04-26T04:53:06.837 INFO:tasks.workunit.client.0.smithi077.stdout:7/514: dread - da/d13/d22/d34/f92 zero size 2024-04-26T04:53:06.838 INFO:tasks.workunit.client.0.smithi077.stdout:0/421: fdatasync d0/f9 0 2024-04-26T04:53:06.841 INFO:tasks.workunit.client.0.smithi077.stdout:8/527: creat d4/df/d6f/da3/fb1 x:0 0 0 2024-04-26T04:53:06.843 INFO:tasks.workunit.client.0.smithi077.stdout:6/540: dwrite d8/d52/d53/d5e/fa5 [0,4194304] 0 2024-04-26T04:53:06.845 INFO:tasks.workunit.client.0.smithi077.stdout:8/528: truncate d4/f42 1738222 0 2024-04-26T04:53:06.846 INFO:tasks.workunit.client.0.smithi077.stdout:0/422: creat d0/d2c/d4e/f95 x:0 0 0 2024-04-26T04:53:06.846 INFO:tasks.workunit.client.0.smithi077.stdout:3/388: dwrite d0/d9/d32/f21 [0,4194304] 0 2024-04-26T04:53:06.850 INFO:tasks.workunit.client.0.smithi077.stdout:3/389: write d0/d9/d32/d29/f71 [326006,45007] 0 2024-04-26T04:53:06.854 INFO:tasks.workunit.client.0.smithi077.stdout:0/423: unlink d0/d19/d1e/f26 0 2024-04-26T04:53:06.854 INFO:tasks.workunit.client.0.smithi077.stdout:6/541: symlink d8/d52/d53/d5e/da6/lb7 0 2024-04-26T04:53:06.855 INFO:tasks.workunit.client.0.smithi077.stdout:3/390: dread d0/d9/d32/f21 [0,4194304] 0 2024-04-26T04:53:06.855 INFO:tasks.workunit.client.0.smithi077.stdout:8/529: unlink d4/df/d22/d2d/d2b/c58 0 2024-04-26T04:53:06.856 INFO:tasks.workunit.client.0.smithi077.stdout:3/391: chown d0/d7/d1f/d4d/c75 12 1 2024-04-26T04:53:06.857 INFO:tasks.workunit.client.0.smithi077.stdout:0/424: mkdir d0/d19/d1e/d59/d96 0 2024-04-26T04:53:06.860 INFO:tasks.workunit.client.0.smithi077.stdout:8/530: chown d4/c20 3 1 2024-04-26T04:53:06.863 INFO:tasks.workunit.client.0.smithi077.stdout:0/425: mkdir d0/d47/d64/d97 0 2024-04-26T04:53:06.864 INFO:tasks.workunit.client.1.smithi078.stdout:8/875: write d0/d19/d52/d35/d44/fe5 [1130766,126423] 0 2024-04-26T04:53:06.864 INFO:tasks.workunit.client.1.smithi078.stdout:5/864: symlink d7/d18/d32/d37/d8f/dc5/l122 0 2024-04-26T04:53:06.864 INFO:tasks.workunit.client.1.smithi078.stdout:2/894: chown d8/d11/d18/d137/d34/dd3/de4/d8e/dec/l10a 11187008 1 2024-04-26T04:53:06.865 INFO:tasks.workunit.client.1.smithi078.stdout:3/904: fdatasync d16/d3c/d81/f91 0 2024-04-26T04:53:06.867 INFO:tasks.workunit.client.0.smithi077.stdout:8/531: unlink d4/df/d22/d80/d86/fa1 0 2024-04-26T04:53:06.867 INFO:tasks.workunit.client.0.smithi077.stdout:3/392: dwrite d0/d7/d15/f3f [0,4194304] 0 2024-04-26T04:53:06.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.866+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15540f8f20 con 0x7f154006b7a0 2024-04-26T04:53:06.868 INFO:tasks.workunit.client.1.smithi078.stdout:6/953: write d2/d1a/d38/dd5/fde [302476,85399] 0 2024-04-26T04:53:06.869 INFO:tasks.workunit.client.1.smithi078.stdout:1/944: dwrite d7/d1e/d9e/d92/fe5 [0,4194304] 0 2024-04-26T04:53:06.869 INFO:tasks.workunit.client.0.smithi077.stdout:0/426: unlink d0/d2c/l37 0 2024-04-26T04:53:06.870 INFO:tasks.workunit.client.1.smithi078.stdout:3/905: write d16/d26/d2e/d5f/d49/dad/db0/fef [3914231,62835] 0 2024-04-26T04:53:06.870 INFO:tasks.workunit.client.1.smithi078.stdout:0/903: truncate d2/d9/d78/d84/f8e 3140464 0 2024-04-26T04:53:06.872 INFO:tasks.workunit.client.0.smithi077.stdout:7/515: dwrite da/d6b/f93 [0,4194304] 0 2024-04-26T04:53:06.873 INFO:tasks.workunit.client.1.smithi078.stdout:9/824: symlink dc/df/d4b/d22/d2a/d92/lf7 0 2024-04-26T04:53:06.875 INFO:tasks.workunit.client.1.smithi078.stdout:9/825: truncate dc/df/d4b/dae/ff5 584414 0 2024-04-26T04:53:06.876 INFO:tasks.workunit.client.0.smithi077.stdout:0/427: rmdir d0/d28 39 2024-04-26T04:53:06.876 INFO:tasks.workunit.client.1.smithi078.stdout:9/826: dread dc/f60 [0,4194304] 0 2024-04-26T04:53:06.878 INFO:tasks.workunit.client.0.smithi077.stdout:3/393: dread d0/f19 [0,4194304] 0 2024-04-26T04:53:06.882 INFO:tasks.workunit.client.0.smithi077.stdout:7/516: creat da/d13/d22/d34/fac x:0 0 0 2024-04-26T04:53:06.882 INFO:tasks.workunit.client.1.smithi078.stdout:2/895: read d8/d11/d18/d5d/f8a [3959402,55335] 0 2024-04-26T04:53:06.883 INFO:tasks.workunit.client.0.smithi077.stdout:7/517: chown da/d13/d22/d5d/c91 462628350 1 2024-04-26T04:53:06.884 INFO:tasks.workunit.client.1.smithi078.stdout:5/865: symlink d7/d18/l123 0 2024-04-26T04:53:06.884 INFO:tasks.workunit.client.1.smithi078.stdout:2/896: truncate d8/d11/f132 747972 0 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.884+0000 7f1549ffb700 1 -- 172.21.15.77:0/3689703552 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f15540f8f20 con 0x7f154006b7a0 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (7m) 28s ago 8m 25.5M - 0.25.0 c8568f914cd2 662f44992626 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (8m) 28s ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (7m) 31s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:53:06.886 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (7m) 28s ago 8m 85.2M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (3m) 28s ago 3m 177M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (3m) 28s ago 3m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (3m) 31s ago 3m 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (3m) 31s ago 3m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (9m) 28s ago 9m 344M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (53s) 31s ago 7m 490M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 70ff04585016 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (9m) 28s ago 10m 49.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (7m) 31s ago 7m 38.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 starting - - - - 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (7m) 31s ago 7m 16.4M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (6m) 28s ago 6m 78.1M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:53:06.887 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (6m) 28s ago 6m 85.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:53:06.888 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (5m) 28s ago 5m 76.9M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:53:06.888 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (5m) 31s ago 5m 99.3M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:53:06.888 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (5m) 31s ago 5m 89.1M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:53:06.888 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (4m) 31s ago 4m 85.9M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:53:06.888 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (6m) 28s ago 8m 58.3M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:53:06.888 INFO:tasks.workunit.client.0.smithi077.stdout:0/428: rename d0/d19/d1e/f89 to d0/d19/f98 0 2024-04-26T04:53:06.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 msgr2=0x7f154006dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 0x7f154006dc50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1550004ba0 tx=0x7f155000b040 comp rx=0 tx=0).stop 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 msgr2=0x7f155406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1544009fd0 tx=0x7f1544009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 shutdown_connections 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f154006b7a0 0x7f154006dc50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1554066e90 0x7f155406e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 --2- 172.21.15.77:0/3689703552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15540fff20 0x7f155406e8c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 >> 172.21.15.77:0/3689703552 conn(0x7f15540f7430 msgr2=0x7f15540f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 shutdown_connections 2024-04-26T04:53:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:06.888+0000 7f155b589700 1 -- 172.21.15.77:0/3689703552 wait complete. 2024-04-26T04:53:06.894 INFO:tasks.workunit.client.1.smithi078.stdout:1/945: unlink d7/d1e/d9e/d92/fe5 0 2024-04-26T04:53:06.894 INFO:tasks.workunit.client.1.smithi078.stdout:1/946: fdatasync d7/ff7 0 2024-04-26T04:53:06.895 INFO:tasks.workunit.client.0.smithi077.stdout:7/518: rename da/d13/l4d to da/d6b/lad 0 2024-04-26T04:53:06.896 INFO:tasks.workunit.client.0.smithi077.stdout:7/519: stat da/d6b/d63/f9e 0 2024-04-26T04:53:06.897 INFO:tasks.workunit.client.1.smithi078.stdout:8/876: mkdir d0/d19/d25/d11a 0 2024-04-26T04:53:06.898 INFO:tasks.workunit.client.1.smithi078.stdout:4/886: sync 2024-04-26T04:53:06.899 INFO:tasks.workunit.client.1.smithi078.stdout:1/947: dwrite d7/d1e/d9e/dd/df0/ff5 [0,4194304] 0 2024-04-26T04:53:06.904 INFO:tasks.workunit.client.1.smithi078.stdout:1/948: truncate d7/ddd/f11a 114726 0 2024-04-26T04:53:06.905 INFO:tasks.workunit.client.0.smithi077.stdout:8/532: dread d4/d17/d35/d5b/f63 [0,4194304] 0 2024-04-26T04:53:06.906 INFO:tasks.workunit.client.1.smithi078.stdout:8/877: dwrite d0/d7a/d83/df3/f107 [0,4194304] 0 2024-04-26T04:53:06.907 INFO:tasks.workunit.client.0.smithi077.stdout:0/429: rename d0/d19/d1e/f34 to d0/d19/d1e/d59/d96/f99 0 2024-04-26T04:53:06.909 INFO:tasks.workunit.client.0.smithi077.stdout:7/520: symlink da/d13/d22/d7d/lae 0 2024-04-26T04:53:06.911 INFO:tasks.workunit.client.1.smithi078.stdout:9/827: creat dc/df/d4b/dae/ff8 x:0 0 0 2024-04-26T04:53:06.913 INFO:tasks.workunit.client.1.smithi078.stdout:2/897: creat d8/d11/d18/d137/d34/d70/def/f13b x:0 0 0 2024-04-26T04:53:06.914 INFO:tasks.workunit.client.0.smithi077.stdout:8/533: creat d4/d87/fb2 x:0 0 0 2024-04-26T04:53:06.918 INFO:tasks.workunit.client.1.smithi078.stdout:6/954: dwrite d2/d1a/d38/dd5/ffa [0,4194304] 0 2024-04-26T04:53:06.919 INFO:tasks.workunit.client.0.smithi077.stdout:3/394: dwrite d0/d7/d15/f20 [0,4194304] 0 2024-04-26T04:53:06.926 INFO:tasks.workunit.client.0.smithi077.stdout:8/534: mknod d4/d87/cb3 0 2024-04-26T04:53:06.928 INFO:tasks.workunit.client.1.smithi078.stdout:6/955: dwrite d2/de/d27/da9/da/d36/db6/f12c [0,4194304] 0 2024-04-26T04:53:06.929 INFO:tasks.workunit.client.1.smithi078.stdout:6/956: write d2/de/d27/f64 [1180979,41331] 0 2024-04-26T04:53:06.933 INFO:tasks.workunit.client.1.smithi078.stdout:0/904: symlink d2/d9/d60/d6a/l132 0 2024-04-26T04:53:06.936 INFO:tasks.workunit.client.0.smithi077.stdout:0/430: link d0/f2 d0/d47/d93/f9a 0 2024-04-26T04:53:06.937 INFO:tasks.workunit.client.0.smithi077.stdout:0/431: readlink d0/d47/l8b 0 2024-04-26T04:53:06.941 INFO:tasks.workunit.client.1.smithi078.stdout:1/949: rename d7/d1e/d9e/d4c to d7/d1e/d9e/db/d49/dba/dc5/d134 0 2024-04-26T04:53:06.943 INFO:tasks.workunit.client.1.smithi078.stdout:8/878: unlink d0/d19/d25/d32/d8a/ccb 0 2024-04-26T04:53:06.945 INFO:tasks.workunit.client.1.smithi078.stdout:3/906: creat d16/d26/f136 x:0 0 0 2024-04-26T04:53:06.948 INFO:tasks.workunit.client.1.smithi078.stdout:6/957: symlink d2/de/d27/d58/d5b/l135 0 2024-04-26T04:53:06.948 INFO:tasks.workunit.client.1.smithi078.stdout:0/905: symlink d2/d9/d31/d4a/d11c/l133 0 2024-04-26T04:53:06.950 INFO:tasks.workunit.client.1.smithi078.stdout:4/887: link d0/d1d/d46/d100/f132 d0/d1d/db4/f13a 0 2024-04-26T04:53:06.951 INFO:tasks.workunit.client.1.smithi078.stdout:8/879: mknod d0/d19/d25/d32/d8a/c11b 0 2024-04-26T04:53:06.952 INFO:tasks.workunit.client.1.smithi078.stdout:3/907: mknod d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/c137 0 2024-04-26T04:53:06.952 INFO:tasks.workunit.client.1.smithi078.stdout:6/958: symlink d2/de/d27/da9/da/d36/db9/d97/l136 0 2024-04-26T04:53:06.956 INFO:tasks.workunit.client.1.smithi078.stdout:8/880: rmdir d0/d19/d52/d35 39 2024-04-26T04:53:06.956 INFO:tasks.workunit.client.1.smithi078.stdout:3/908: rmdir d16 39 2024-04-26T04:53:06.958 INFO:tasks.workunit.client.1.smithi078.stdout:6/959: creat d2/de/d27/da9/d3a/d69/daa/dba/dc4/f137 x:0 0 0 2024-04-26T04:53:06.959 INFO:tasks.workunit.client.1.smithi078.stdout:7/853: sync 2024-04-26T04:53:06.959 INFO:tasks.workunit.client.1.smithi078.stdout:4/888: dwrite d0/d1d/db4/f13a [0,4194304] 0 2024-04-26T04:53:06.960 INFO:tasks.workunit.client.1.smithi078.stdout:1/950: rename d7/d1e/d9e/l96 to d7/d1e/d9e/dd/d20/l135 0 2024-04-26T04:53:06.962 INFO:tasks.workunit.client.1.smithi078.stdout:7/854: write d9/da/d11/fe7 [148209,97538] 0 2024-04-26T04:53:06.962 INFO:tasks.workunit.client.1.smithi078.stdout:3/909: mkdir d16/d26/d2e/d5f/d49/d138 0 2024-04-26T04:53:06.962 INFO:tasks.workunit.client.0.smithi077.stdout:3/395: unlink d0/d9/d32/d2a/f6b 0 2024-04-26T04:53:06.964 INFO:tasks.workunit.client.0.smithi077.stdout:0/432: read d0/d19/f98 [539530,122778] 0 2024-04-26T04:53:06.965 INFO:tasks.workunit.client.0.smithi077.stdout:0/433: stat d0/c23 0 2024-04-26T04:53:06.966 INFO:tasks.workunit.client.1.smithi078.stdout:6/960: dwrite d2/d1a/d5c/f11a [0,4194304] 0 2024-04-26T04:53:06.971 INFO:tasks.workunit.client.1.smithi078.stdout:4/889: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/dba/fcb zero size 2024-04-26T04:53:06.974 INFO:tasks.workunit.client.1.smithi078.stdout:7/855: mknod d9/da/d11/d13/d50/d8a/c11a 0 2024-04-26T04:53:06.981 INFO:tasks.workunit.client.1.smithi078.stdout:4/890: mknod d0/d1d/de1/d20/d31/c13b 0 2024-04-26T04:53:06.982 INFO:tasks.workunit.client.1.smithi078.stdout:4/891: truncate d0/de/d22/d27/f130 238998 0 2024-04-26T04:53:06.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/856: mkdir d9/da/d33/d4b/d4c/d5b/dc3/d11b 0 2024-04-26T04:53:06.984 INFO:tasks.workunit.client.1.smithi078.stdout:7/857: stat d9/da/d33/d4b/d4c/d5b/d83/de6/cf7 0 2024-04-26T04:53:06.984 INFO:tasks.workunit.client.1.smithi078.stdout:9/828: dread dc/df/d4b/d22/dbb/fe4 [0,4194304] 0 2024-04-26T04:53:06.986 INFO:tasks.workunit.client.0.smithi077.stdout:7/521: truncate da/d13/d22/d5d/f60 93826 0 2024-04-26T04:53:06.994 INFO:tasks.workunit.client.1.smithi078.stdout:5/866: sync 2024-04-26T04:53:06.996 INFO:tasks.workunit.client.1.smithi078.stdout:7/858: mknod d9/da/d11/d13/d1a/d52/d5a/d28/d6d/c11c 0 2024-04-26T04:53:06.996 INFO:tasks.workunit.client.1.smithi078.stdout:2/898: dwrite d8/f55 [0,4194304] 0 2024-04-26T04:53:06.997 INFO:tasks.workunit.client.1.smithi078.stdout:4/892: rmdir d0/d1d/de1/d20/d31/d47/db2/d113 0 2024-04-26T04:53:06.998 INFO:tasks.workunit.client.1.smithi078.stdout:7/859: fdatasync d9/da/d20/f9b 0 2024-04-26T04:53:07.000 INFO:tasks.workunit.client.1.smithi078.stdout:9/829: dread dc/df/d27/d81/da8/fac [0,4194304] 0 2024-04-26T04:53:07.000 INFO:tasks.workunit.client.1.smithi078.stdout:7/860: fsync d9/da/d11/d13/d1a/d52/d5a/d41/f51 0 2024-04-26T04:53:07.002 INFO:tasks.workunit.client.1.smithi078.stdout:5/867: mknod d7/d18/d32/c124 0 2024-04-26T04:53:07.006 INFO:tasks.workunit.client.1.smithi078.stdout:9/830: dwrite dc/df/d4b/d22/d2a/d92/ddb/fee [0,4194304] 0 2024-04-26T04:53:07.008 INFO:tasks.workunit.client.1.smithi078.stdout:2/899: creat d8/d11/d18/d137/d34/dd6/f13c x:0 0 0 2024-04-26T04:53:07.008 INFO:tasks.workunit.client.1.smithi078.stdout:2/900: read - d8/ff2 zero size 2024-04-26T04:53:07.010 INFO:tasks.workunit.client.1.smithi078.stdout:4/893: creat d0/de/d22/d71/dae/f13c x:0 0 0 2024-04-26T04:53:07.011 INFO:tasks.workunit.client.1.smithi078.stdout:6/961: dread d2/de/d27/da9/f8 [0,4194304] 0 2024-04-26T04:53:07.013 INFO:tasks.workunit.client.1.smithi078.stdout:6/962: write d2/de/d27/da9/f109 [2953970,72085] 0 2024-04-26T04:53:07.015 INFO:tasks.workunit.client.1.smithi078.stdout:5/868: truncate d7/d18/d1c/f92 984967 0 2024-04-26T04:53:07.016 INFO:tasks.workunit.client.1.smithi078.stdout:0/906: dwrite d2/d1e/d5a/da3/fdd [4194304,4194304] 0 2024-04-26T04:53:07.017 INFO:tasks.workunit.client.1.smithi078.stdout:8/881: write d0/d19/d25/d32/d8a/f57 [580013,79058] 0 2024-04-26T04:53:07.021 INFO:tasks.workunit.client.1.smithi078.stdout:9/831: unlink dc/df/f1d 0 2024-04-26T04:53:07.023 INFO:tasks.workunit.client.1.smithi078.stdout:3/910: write d16/f20 [1238041,37983] 0 2024-04-26T04:53:07.028 INFO:tasks.workunit.client.1.smithi078.stdout:1/951: dwrite d7/d1e/d9e/db/d49/dba/fc8 [0,4194304] 0 2024-04-26T04:53:07.029 INFO:tasks.workunit.client.1.smithi078.stdout:4/894: creat d0/d1d/f13d x:0 0 0 2024-04-26T04:53:07.031 INFO:tasks.workunit.client.0.smithi077.stdout:8/535: write d4/d17/d35/f70 [762119,43295] 0 2024-04-26T04:53:07.032 INFO:tasks.workunit.client.0.smithi077.stdout:8/536: chown d4/f42 2002051 1 2024-04-26T04:53:07.038 INFO:tasks.workunit.client.1.smithi078.stdout:5/869: mkdir d7/d21/d2f/db6/d125 0 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 -- 172.21.15.77:0/3688642038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 msgr2=0x7fce000fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 --2- 172.21.15.77:0/3688642038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce000fd3d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004940 tx=0x7fcdf0009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 -- 172.21.15.77:0/3688642038 shutdown_connections 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 --2- 172.21.15.77:0/3688642038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce000fd3d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 --2- 172.21.15.77:0/3688642038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 0x7fce000fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 -- 172.21.15.77:0/3688642038 >> 172.21.15.77:0/3688642038 conn(0x7fce000f7430 msgr2=0x7fce000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.041+0000 7fce07bcb700 1 -- 172.21.15.77:0/3688642038 shutdown_connections 2024-04-26T04:53:07.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.042+0000 7fce07bcb700 1 -- 172.21.15.77:0/3688642038 wait complete. 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.042+0000 7fce07bcb700 1 Processor -- start 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.042+0000 7fce07bcb700 1 -- start start 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce07bcb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 0x7fce0010b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce07bcb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce07bcb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0010c310 con 0x7fce000fcf60 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce07bcb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0010cb10 con 0x7fce000fbcb0 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58348/0 (socket says 172.21.15.77:58348) 2024-04-26T04:53:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 -- 172.21.15.77:0/2686618373 learned_addr learned my addr 172.21.15.77:0/2686618373 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:07.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05967700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 0x7fce0010b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:07.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 -- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 msgr2=0x7fce0010b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 0x7fce0010b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.043+0000 7fce05166700 1 -- 172.21.15.77:0/2686618373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc005040 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.044+0000 7fce05166700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0011a20 tx=0x7fcdf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.044+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf00092e0 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.044+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0005040 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.044+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf001a420 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.044+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf0009440 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.045+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0010d140 con 0x7fce000fcf60 2024-04-26T04:53:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.045+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0005f5c0 con 0x7fce000fcf60 2024-04-26T04:53:07.048 INFO:tasks.workunit.client.1.smithi078.stdout:9/832: chown dc/df/d27/l78 7996790 1 2024-04-26T04:53:07.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.046+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fcdf001a590 con 0x7fce000fcf60 2024-04-26T04:53:07.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.047+0000 7fcdf67fc700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 0x7fcdec06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.047+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fcdf008da30 con 0x7fce000fcf60 2024-04-26T04:53:07.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.047+0000 7fce05967700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 0x7fcdec06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:07.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.048+0000 7fce05967700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 0x7fcdec06dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:07.052 INFO:tasks.workunit.client.1.smithi078.stdout:3/911: rename d16/d26/d2e/d5f/d49/d77/dba to d16/d9b/d139 0 2024-04-26T04:53:07.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.053+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fcdf005b2a0 con 0x7fce000fcf60 2024-04-26T04:53:07.057 INFO:tasks.workunit.client.1.smithi078.stdout:0/907: dread d2/d9/d31/d4a/f9b [0,4194304] 0 2024-04-26T04:53:07.057 INFO:tasks.workunit.client.1.smithi078.stdout:1/952: rmdir d7/d1e/d9e/dd/d20 39 2024-04-26T04:53:07.063 INFO:tasks.workunit.client.0.smithi077.stdout:0/434: creat d0/d47/d93/d94/f9b x:0 0 0 2024-04-26T04:53:07.063 INFO:tasks.workunit.client.1.smithi078.stdout:5/870: dread d7/d21/d2f/fe0 [0,4194304] 0 2024-04-26T04:53:07.064 INFO:tasks.workunit.client.1.smithi078.stdout:9/833: symlink dc/df/d27/dbc/lf9 0 2024-04-26T04:53:07.064 INFO:tasks.workunit.client.1.smithi078.stdout:9/834: chown dc/df/d4b/d22/d2d/d98/fb4 866060 1 2024-04-26T04:53:07.065 INFO:tasks.workunit.client.1.smithi078.stdout:9/835: dread dc/f4e [0,4194304] 0 2024-04-26T04:53:07.066 INFO:tasks.workunit.client.1.smithi078.stdout:2/901: rename d8/d11/d18/dfe/f136 to d8/d11/d18/d137/d34/dd3/de4/d8e/d91/f13d 0 2024-04-26T04:53:07.067 INFO:tasks.workunit.client.1.smithi078.stdout:2/902: write d8/d11/d18/dbb/f101 [602985,81508] 0 2024-04-26T04:53:07.070 INFO:tasks.workunit.client.0.smithi077.stdout:7/522: dwrite da/d13/d22/d34/f51 [0,4194304] 0 2024-04-26T04:53:07.071 INFO:tasks.workunit.client.0.smithi077.stdout:8/537: rmdir d4/df 39 2024-04-26T04:53:07.075 INFO:tasks.workunit.client.0.smithi077.stdout:3/396: creat d0/f7a x:0 0 0 2024-04-26T04:53:07.077 INFO:tasks.workunit.client.1.smithi078.stdout:1/953: unlink d7/d1e/d9e/dfc/l122 0 2024-04-26T04:53:07.077 INFO:tasks.workunit.client.0.smithi077.stdout:0/435: chown d0/c17 13365571 1 2024-04-26T04:53:07.080 INFO:tasks.workunit.client.0.smithi077.stdout:7/523: creat da/d15/d1a/d39/faf x:0 0 0 2024-04-26T04:53:07.086 INFO:tasks.workunit.client.1.smithi078.stdout:6/963: write d2/d1a/d38/dd5/ff3 [239955,2650] 0 2024-04-26T04:53:07.089 INFO:tasks.workunit.client.1.smithi078.stdout:0/908: rename d2/d39/lbd to d2/d1e/d5a/dda/l134 0 2024-04-26T04:53:07.090 INFO:tasks.workunit.client.0.smithi077.stdout:7/524: mkdir da/d6b/d63/d64/db0 0 2024-04-26T04:53:07.090 INFO:tasks.workunit.client.1.smithi078.stdout:7/861: sync 2024-04-26T04:53:07.091 INFO:tasks.workunit.client.1.smithi078.stdout:9/836: truncate dc/df/d4b/d22/dbb/fe4 1635352 0 2024-04-26T04:53:07.092 INFO:tasks.workunit.client.0.smithi077.stdout:8/538: creat d4/df/d6f/fb4 x:0 0 0 2024-04-26T04:53:07.093 INFO:tasks.workunit.client.1.smithi078.stdout:2/903: creat d8/d11/d18/dbb/f13e x:0 0 0 2024-04-26T04:53:07.094 INFO:tasks.workunit.client.1.smithi078.stdout:2/904: truncate d8/d11/d18/d137/fe3 1041620 0 2024-04-26T04:53:07.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/525: dread da/d15/d42/d8c/f9b [0,4194304] 0 2024-04-26T04:53:07.095 INFO:tasks.workunit.client.0.smithi077.stdout:3/397: mknod d0/d7/d1f/d76/c7b 0 2024-04-26T04:53:07.096 INFO:tasks.workunit.client.1.smithi078.stdout:3/912: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d13a 0 2024-04-26T04:53:07.098 INFO:tasks.workunit.client.0.smithi077.stdout:0/436: chown d0/d2c/d4e/c1c 58544380 1 2024-04-26T04:53:07.098 INFO:tasks.workunit.client.1.smithi078.stdout:3/913: read d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/fc5 [3162988,64369] 0 2024-04-26T04:53:07.101 INFO:tasks.workunit.client.1.smithi078.stdout:6/964: dread d2/de/d27/da9/d3a/d69/dee/f10f [0,4194304] 0 2024-04-26T04:53:07.102 INFO:tasks.workunit.client.0.smithi077.stdout:7/526: dwrite da/f36 [4194304,4194304] 0 2024-04-26T04:53:07.106 INFO:tasks.workunit.client.1.smithi078.stdout:4/895: write d0/de/ff7 [64760,88642] 0 2024-04-26T04:53:07.107 INFO:tasks.workunit.client.1.smithi078.stdout:4/896: dread - d0/d1d/de1/d20/d31/f6f zero size 2024-04-26T04:53:07.108 INFO:tasks.workunit.client.1.smithi078.stdout:4/897: fsync d0/de/d22/d71/dae/f13c 0 2024-04-26T04:53:07.109 INFO:tasks.workunit.client.0.smithi077.stdout:8/539: creat d4/d69/d73/fb5 x:0 0 0 2024-04-26T04:53:07.111 INFO:tasks.workunit.client.0.smithi077.stdout:3/398: stat d0/c45 0 2024-04-26T04:53:07.112 INFO:tasks.workunit.client.1.smithi078.stdout:8/882: dwrite d0/d19/d52/d35/d44/f4d [0,4194304] 0 2024-04-26T04:53:07.116 INFO:tasks.workunit.client.1.smithi078.stdout:7/862: write d9/da/d11/d13/d50/d8a/fea [360597,117880] 0 2024-04-26T04:53:07.121 INFO:tasks.workunit.client.1.smithi078.stdout:7/863: dwrite d9/da/d33/d4b/d4c/fe5 [0,4194304] 0 2024-04-26T04:53:07.125 INFO:tasks.workunit.client.0.smithi077.stdout:7/527: rename da/d13/d22/d7d/d80/c8e to da/d6b/d63/d64/db0/cb1 0 2024-04-26T04:53:07.129 INFO:tasks.workunit.client.0.smithi077.stdout:8/540: chown d4/df/d6f/c9f 15 1 2024-04-26T04:53:07.130 INFO:tasks.workunit.client.0.smithi077.stdout:8/541: write d4/fe [2371471,51412] 0 2024-04-26T04:53:07.131 INFO:tasks.workunit.client.1.smithi078.stdout:3/914: symlink d16/d26/d2e/d5f/d49/dcc/de2/l13b 0 2024-04-26T04:53:07.133 INFO:tasks.workunit.client.0.smithi077.stdout:0/437: unlink d0/d2c/l6a 0 2024-04-26T04:53:07.137 INFO:tasks.workunit.client.0.smithi077.stdout:0/438: dread d0/d19/d1e/d59/f6e [0,4194304] 0 2024-04-26T04:53:07.139 INFO:tasks.workunit.client.0.smithi077.stdout:7/528: creat da/d13/d22/d5d/fb2 x:0 0 0 2024-04-26T04:53:07.140 INFO:tasks.workunit.client.0.smithi077.stdout:7/529: dread - da/d13/fa1 zero size 2024-04-26T04:53:07.141 INFO:tasks.workunit.client.1.smithi078.stdout:4/898: unlink d0/d1d/de1/f107 0 2024-04-26T04:53:07.145 INFO:tasks.workunit.client.1.smithi078.stdout:0/909: symlink d2/d123/l135 0 2024-04-26T04:53:07.147 INFO:tasks.workunit.client.1.smithi078.stdout:4/899: read d0/d1d/de1/d20/d31/d47/db2/d44/f68 [2087036,100217] 0 2024-04-26T04:53:07.147 INFO:tasks.workunit.client.0.smithi077.stdout:0/439: mknod d0/d19/d1e/d82/c9c 0 2024-04-26T04:53:07.148 INFO:tasks.workunit.client.1.smithi078.stdout:4/900: truncate d0/d1d/de1/d20/d31/d47/db2/ff9 240299 0 2024-04-26T04:53:07.149 INFO:tasks.workunit.client.0.smithi077.stdout:7/530: creat da/d6b/fb3 x:0 0 0 2024-04-26T04:53:07.150 INFO:tasks.workunit.client.0.smithi077.stdout:7/531: write da/d15/f2f [1920136,26441] 0 2024-04-26T04:53:07.150 INFO:tasks.workunit.client.1.smithi078.stdout:4/901: dread d0/de/d22/d27/de7/f10e [0,4194304] 0 2024-04-26T04:53:07.151 INFO:tasks.workunit.client.0.smithi077.stdout:7/532: stat da/d13/c9d 0 2024-04-26T04:53:07.151 INFO:tasks.workunit.client.0.smithi077.stdout:7/533: readlink da/l4c 0 2024-04-26T04:53:07.152 INFO:tasks.workunit.client.1.smithi078.stdout:1/954: sync 2024-04-26T04:53:07.152 INFO:tasks.workunit.client.1.smithi078.stdout:1/955: chown d7/d1e/d9e/db/d37/f65 126 1 2024-04-26T04:53:07.153 INFO:tasks.workunit.client.1.smithi078.stdout:1/956: stat d7/d1e/d9e/db/d49/dba/d87/fc1 0 2024-04-26T04:53:07.153 INFO:tasks.workunit.client.1.smithi078.stdout:2/905: truncate d8/d11/d18/d5d/d61/f76 1328729 0 2024-04-26T04:53:07.154 INFO:tasks.workunit.client.1.smithi078.stdout:2/906: chown d8/d11/d18/d5d/d8b/db0/d10d/d110 1978578513 1 2024-04-26T04:53:07.155 INFO:tasks.workunit.client.1.smithi078.stdout:2/907: read - d8/d11/d18/d137/d34/d70/f103 zero size 2024-04-26T04:53:07.156 INFO:tasks.workunit.client.1.smithi078.stdout:2/908: chown d8/d11/d6e/f80 3164 1 2024-04-26T04:53:07.159 INFO:tasks.workunit.client.1.smithi078.stdout:3/915: rmdir d16/d26/d2e/d5f/d49 39 2024-04-26T04:53:07.159 INFO:tasks.workunit.client.1.smithi078.stdout:7/864: write d9/da/d11/d13/d1a/d52/d5a/d28/d6d/f6e [971094,116967] 0 2024-04-26T04:53:07.160 INFO:tasks.workunit.client.0.smithi077.stdout:8/542: dread d4/df/d22/d2d/d2b/f7b [0,4194304] 0 2024-04-26T04:53:07.161 INFO:tasks.workunit.client.1.smithi078.stdout:7/865: write d9/da/d33/d4b/d4c/f55 [3344048,11363] 0 2024-04-26T04:53:07.161 INFO:tasks.workunit.client.1.smithi078.stdout:6/965: fsync d2/de/d27/da9/da/d36/db9/f57 0 2024-04-26T04:53:07.162 INFO:tasks.workunit.client.0.smithi077.stdout:7/534: getdents da/d15/d1a/d39/d96 0 2024-04-26T04:53:07.163 INFO:tasks.workunit.client.1.smithi078.stdout:5/871: getdents d7/d48/d57/d65/d46/de7 0 2024-04-26T04:53:07.164 INFO:tasks.workunit.client.0.smithi077.stdout:8/543: fdatasync d4/df/f16 0 2024-04-26T04:53:07.166 INFO:tasks.workunit.client.0.smithi077.stdout:0/440: creat d0/d19/d1e/f9d x:0 0 0 2024-04-26T04:53:07.167 INFO:tasks.workunit.client.1.smithi078.stdout:7/866: dwrite f4 [4194304,4194304] 0 2024-04-26T04:53:07.167 INFO:tasks.workunit.client.1.smithi078.stdout:8/883: symlink d0/l11c 0 2024-04-26T04:53:07.170 INFO:tasks.workunit.client.1.smithi078.stdout:9/837: creat dc/df/d4b/d22/d2a/ffa x:0 0 0 2024-04-26T04:53:07.171 INFO:tasks.workunit.client.0.smithi077.stdout:8/544: creat d4/d17/fb6 x:0 0 0 2024-04-26T04:53:07.172 INFO:tasks.workunit.client.0.smithi077.stdout:0/441: truncate d0/d19/d1e/d2f/f3b 41221 0 2024-04-26T04:53:07.173 INFO:tasks.workunit.client.1.smithi078.stdout:8/884: dwrite d0/d19/d25/d32/d8a/d92/da4/fc6 [0,4194304] 0 2024-04-26T04:53:07.174 INFO:tasks.workunit.client.0.smithi077.stdout:7/535: symlink da/d15/d1a/d39/d96/lb4 0 2024-04-26T04:53:07.175 INFO:tasks.workunit.client.0.smithi077.stdout:7/536: chown da/d13/d22/d5d 93827 1 2024-04-26T04:53:07.175 INFO:tasks.workunit.client.1.smithi078.stdout:1/957: creat d7/d1e/d9e/db/d37/d58/d5c/dc0/f136 x:0 0 0 2024-04-26T04:53:07.177 INFO:tasks.workunit.client.1.smithi078.stdout:2/909: rmdir d8/d11/d18/d5d/d8b 39 2024-04-26T04:53:07.178 INFO:tasks.workunit.client.1.smithi078.stdout:1/958: write d7/f125 [574360,116832] 0 2024-04-26T04:53:07.180 INFO:tasks.workunit.client.0.smithi077.stdout:0/442: chown d0/ca 1169 1 2024-04-26T04:53:07.182 INFO:tasks.workunit.client.0.smithi077.stdout:0/443: write d0/d19/d69/f84 [100766,4558] 0 2024-04-26T04:53:07.183 INFO:tasks.workunit.client.1.smithi078.stdout:8/885: dwrite d0/d19/d52/d35/d44/d6b/d104/f7f [0,4194304] 0 2024-04-26T04:53:07.183 INFO:tasks.workunit.client.0.smithi077.stdout:0/444: write d0/d19/d1e/f9d [246910,84359] 0 2024-04-26T04:53:07.184 INFO:tasks.workunit.client.1.smithi078.stdout:5/872: creat d7/d48/d57/d65/d7a/f126 x:0 0 0 2024-04-26T04:53:07.186 INFO:tasks.workunit.client.0.smithi077.stdout:7/537: creat da/d15/d1a/fb5 x:0 0 0 2024-04-26T04:53:07.189 INFO:tasks.workunit.client.0.smithi077.stdout:8/545: getdents d4/df/d22/d80/d86 0 2024-04-26T04:53:07.189 INFO:tasks.workunit.client.1.smithi078.stdout:0/910: rename d2/d9/d18/f45 to d2/d9/d18/db7/d92/f136 0 2024-04-26T04:53:07.192 INFO:tasks.workunit.client.0.smithi077.stdout:0/445: unlink d0/d47/d64/l70 0 2024-04-26T04:53:07.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/838: creat dc/df/d17/ffb x:0 0 0 2024-04-26T04:53:07.195 INFO:tasks.workunit.client.1.smithi078.stdout:9/839: stat dc/df/d27/l9a 0 2024-04-26T04:53:07.196 INFO:tasks.workunit.client.0.smithi077.stdout:8/546: symlink d4/df/d6f/da3/lb7 0 2024-04-26T04:53:07.196 INFO:tasks.workunit.client.1.smithi078.stdout:9/840: write dc/df/d27/d81/f91 [2573094,35508] 0 2024-04-26T04:53:07.197 INFO:tasks.workunit.client.1.smithi078.stdout:4/902: fsync d0/d1d/de1/d20/f23 0 2024-04-26T04:53:07.199 INFO:tasks.workunit.client.0.smithi077.stdout:8/547: mkdir d4/df/d6f/db8 0 2024-04-26T04:53:07.201 INFO:tasks.workunit.client.1.smithi078.stdout:2/910: unlink d8/d11/f45 0 2024-04-26T04:53:07.201 INFO:tasks.workunit.client.1.smithi078.stdout:2/911: stat d8/c17 0 2024-04-26T04:53:07.202 INFO:tasks.workunit.client.1.smithi078.stdout:2/912: chown d8/fc6 207030 1 2024-04-26T04:53:07.202 INFO:tasks.workunit.client.0.smithi077.stdout:0/446: fsync d0/d2c/f88 0 2024-04-26T04:53:07.203 INFO:tasks.workunit.client.1.smithi078.stdout:2/913: write d8/d11/d18/f108 [583505,76683] 0 2024-04-26T04:53:07.203 INFO:tasks.workunit.client.0.smithi077.stdout:0/447: write d0/d47/d64/f80 [206868,128023] 0 2024-04-26T04:53:07.204 INFO:tasks.workunit.client.1.smithi078.stdout:2/914: write d8/d11/d18/d5d/d61/f134 [4838325,102282] 0 2024-04-26T04:53:07.206 INFO:tasks.workunit.client.1.smithi078.stdout:2/915: truncate d8/d11/f74 4905050 0 2024-04-26T04:53:07.207 INFO:tasks.workunit.client.1.smithi078.stdout:2/916: fdatasync d8/d11/d6e/ff4 0 2024-04-26T04:53:07.210 INFO:tasks.workunit.client.1.smithi078.stdout:2/917: dread d8/d11/d18/dbb/f101 [0,4194304] 0 2024-04-26T04:53:07.211 INFO:tasks.workunit.client.1.smithi078.stdout:2/918: readlink d8/d11/l65 0 2024-04-26T04:53:07.211 INFO:tasks.workunit.client.1.smithi078.stdout:8/886: mknod d0/d19/d52/d35/d44/d6b/d6c/db4/c11d 0 2024-04-26T04:53:07.213 INFO:tasks.workunit.client.0.smithi077.stdout:0/448: mkdir d0/d47/d64/d9e 0 2024-04-26T04:53:07.218 INFO:tasks.workunit.client.1.smithi078.stdout:7/867: write d9/da/d11/d13/d50/d8a/f9c [67014,29420] 0 2024-04-26T04:53:07.222 INFO:tasks.workunit.client.0.smithi077.stdout:0/449: mkdir d0/d47/d68/d9f 0 2024-04-26T04:53:07.226 INFO:tasks.workunit.client.1.smithi078.stdout:9/841: rmdir dc/df/d4b/d22/d2a/d92 39 2024-04-26T04:53:07.227 INFO:tasks.workunit.client.0.smithi077.stdout:0/450: symlink d0/d19/d1e/d81/la0 0 2024-04-26T04:53:07.228 INFO:tasks.workunit.client.0.smithi077.stdout:7/538: write da/d13/d22/d5d/f60 [600148,73193] 0 2024-04-26T04:53:07.232 INFO:tasks.workunit.client.0.smithi077.stdout:0/451: unlink d0/d47/d64/f5c 0 2024-04-26T04:53:07.232 INFO:tasks.workunit.client.1.smithi078.stdout:7/868: dread d9/f6a [0,4194304] 0 2024-04-26T04:53:07.233 INFO:tasks.workunit.client.0.smithi077.stdout:0/452: chown d0/d2c/d4e/f95 311 1 2024-04-26T04:53:07.233 INFO:tasks.workunit.client.1.smithi078.stdout:1/959: write d7/fc [4170099,32182] 0 2024-04-26T04:53:07.234 INFO:tasks.workunit.client.0.smithi077.stdout:8/548: write d4/df/f32 [1637785,20283] 0 2024-04-26T04:53:07.235 INFO:tasks.workunit.client.1.smithi078.stdout:7/869: chown d9/da/d11/d13/d1a/d52/d5a/d28/d6d/ffc 3219650 1 2024-04-26T04:53:07.236 INFO:tasks.workunit.client.1.smithi078.stdout:3/916: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/d13c 0 2024-04-26T04:53:07.237 INFO:tasks.workunit.client.0.smithi077.stdout:0/453: mknod d0/d19/d1e/d81/ca1 0 2024-04-26T04:53:07.238 INFO:tasks.workunit.client.1.smithi078.stdout:2/919: mknod d8/d53/c13f 0 2024-04-26T04:53:07.239 INFO:tasks.workunit.client.1.smithi078.stdout:1/960: dwrite d7/d1e/d9e/db/fe9 [0,4194304] 0 2024-04-26T04:53:07.240 INFO:tasks.workunit.client.0.smithi077.stdout:7/539: mknod da/d15/d1a/cb6 0 2024-04-26T04:53:07.241 INFO:tasks.workunit.client.1.smithi078.stdout:1/961: write d7/d1e/d9e/db/d37/f88 [4312031,110336] 0 2024-04-26T04:53:07.241 INFO:tasks.workunit.client.1.smithi078.stdout:6/966: link d2/de/d27/da9/d2a/dbf/d5f/laf d2/de/d27/da9/d2a/dbf/d5f/d11c/l138 0 2024-04-26T04:53:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:06 smithi077 ceph-mon[26311]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:07.244 INFO:tasks.workunit.client.0.smithi077.stdout:8/549: unlink d4/d17/fb6 0 2024-04-26T04:53:07.245 INFO:tasks.workunit.client.0.smithi077.stdout:8/550: write d4/fe [3896551,8817] 0 2024-04-26T04:53:07.247 INFO:tasks.workunit.client.1.smithi078.stdout:5/873: dwrite d7/da/d1e/d91/f102 [0,4194304] 0 2024-04-26T04:53:07.247 INFO:tasks.workunit.client.0.smithi077.stdout:0/454: mkdir d0/d19/d69/da2 0 2024-04-26T04:53:07.250 INFO:tasks.workunit.client.1.smithi078.stdout:0/911: dwrite d2/d1e/f9e [0,4194304] 0 2024-04-26T04:53:07.250 INFO:tasks.workunit.client.0.smithi077.stdout:7/540: symlink da/d13/d22/lb7 0 2024-04-26T04:53:07.253 INFO:tasks.workunit.client.0.smithi077.stdout:8/551: mknod d4/df/d18/d56/cb9 0 2024-04-26T04:53:07.256 INFO:tasks.workunit.client.1.smithi078.stdout:7/870: creat d9/da/d11/d13/d50/f11d x:0 0 0 2024-04-26T04:53:07.257 INFO:tasks.workunit.client.1.smithi078.stdout:7/871: write d9/da/d11/fe7 [1383485,105910] 0 2024-04-26T04:53:07.257 INFO:tasks.workunit.client.0.smithi077.stdout:7/541: symlink da/d13/d22/d5d/d8d/lb8 0 2024-04-26T04:53:07.258 INFO:tasks.workunit.client.1.smithi078.stdout:7/872: read - d9/da/d11/d13/d1a/d65/f114 zero size 2024-04-26T04:53:07.261 INFO:tasks.workunit.client.1.smithi078.stdout:3/917: creat d16/d26/d2e/d5f/d49/d77/f13d x:0 0 0 2024-04-26T04:53:07.262 INFO:tasks.workunit.client.1.smithi078.stdout:7/873: chown d9/da/d33/d4b/d4c/d6c/fb6 1318 1 2024-04-26T04:53:07.262 INFO:tasks.workunit.client.1.smithi078.stdout:3/918: write d16/d26/d2e/d5f/d49/d77/fc6 [8840119,125157] 0 2024-04-26T04:53:07.263 INFO:tasks.workunit.client.1.smithi078.stdout:6/967: rmdir d2/de/d27/da9/da/d36/db9/d97/d105 39 2024-04-26T04:53:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:06 smithi078 ceph-mon[33746]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:07.308 INFO:tasks.workunit.client.1.smithi078.stdout:8/887: sync 2024-04-26T04:53:07.316 INFO:tasks.workunit.client.0.smithi077.stdout:8/552: link d4/df/d22/d75/l9d d4/d17/d35/lba 0 2024-04-26T04:53:07.318 INFO:tasks.workunit.client.1.smithi078.stdout:0/912: rename d2/d1e/f61 to d2/d9/d60/dd5/f137 0 2024-04-26T04:53:07.321 INFO:tasks.workunit.client.0.smithi077.stdout:8/553: dwrite d4/d87/fb2 [0,4194304] 0 2024-04-26T04:53:07.325 INFO:tasks.workunit.client.0.smithi077.stdout:8/554: dread - d4/d17/d35/d5b/f7f zero size 2024-04-26T04:53:07.325 INFO:tasks.workunit.client.1.smithi078.stdout:5/874: sync 2024-04-26T04:53:07.328 INFO:tasks.workunit.client.0.smithi077.stdout:8/555: dread - d4/df/d22/f62 zero size 2024-04-26T04:53:07.330 INFO:tasks.workunit.client.0.smithi077.stdout:8/556: write d4/df/d6f/fb4 [138320,20217] 0 2024-04-26T04:53:07.330 INFO:tasks.workunit.client.1.smithi078.stdout:3/919: mknod d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130/c13e 0 2024-04-26T04:53:07.332 INFO:tasks.workunit.client.1.smithi078.stdout:9/842: dwrite dc/df/d27/f9b [0,4194304] 0 2024-04-26T04:53:07.334 INFO:tasks.workunit.client.1.smithi078.stdout:8/888: mknod d0/d7a/dbe/c11e 0 2024-04-26T04:53:07.334 INFO:tasks.workunit.client.1.smithi078.stdout:3/920: read d16/d9b/ffc [1416060,80976] 0 2024-04-26T04:53:07.336 INFO:tasks.workunit.client.1.smithi078.stdout:7/874: rename d9/lf2 to d9/da/d33/d4b/d4c/d5b/d83/de6/l11e 0 2024-04-26T04:53:07.338 INFO:tasks.workunit.client.1.smithi078.stdout:2/920: write d8/d11/d18/d137/d34/dd3/de4/d52/fca [2237525,29302] 0 2024-04-26T04:53:07.338 INFO:tasks.workunit.client.1.smithi078.stdout:2/921: chown d8/d53/fdf 234 1 2024-04-26T04:53:07.339 INFO:tasks.workunit.client.0.smithi077.stdout:8/557: mknod d4/d17/d35/cbb 0 2024-04-26T04:53:07.339 INFO:tasks.workunit.client.1.smithi078.stdout:2/922: dread - d8/d53/f11a zero size 2024-04-26T04:53:07.340 INFO:tasks.workunit.client.0.smithi077.stdout:8/558: chown d4/d17/d35 240107 1 2024-04-26T04:53:07.341 INFO:tasks.workunit.client.1.smithi078.stdout:2/923: write d8/d11/d18/d137/d34/f3d [2004753,22153] 0 2024-04-26T04:53:07.343 INFO:tasks.workunit.client.1.smithi078.stdout:5/875: creat d7/d11/dee/d111/f127 x:0 0 0 2024-04-26T04:53:07.344 INFO:tasks.workunit.client.1.smithi078.stdout:1/962: creat d7/d1e/d9e/db/d49/dba/f137 x:0 0 0 2024-04-26T04:53:07.347 INFO:tasks.workunit.client.0.smithi077.stdout:7/542: dwrite da/d15/d42/f77 [0,4194304] 0 2024-04-26T04:53:07.349 INFO:tasks.workunit.client.1.smithi078.stdout:9/843: truncate dc/df/d27/d81/f66 4039084 0 2024-04-26T04:53:07.351 INFO:tasks.workunit.client.1.smithi078.stdout:4/903: getdents d0/de/d22 0 2024-04-26T04:53:07.351 INFO:tasks.workunit.client.1.smithi078.stdout:7/875: dwrite d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 [0,4194304] 0 2024-04-26T04:53:07.355 INFO:tasks.workunit.client.0.smithi077.stdout:7/543: symlink da/d13/d22/lb9 0 2024-04-26T04:53:07.359 INFO:tasks.workunit.client.0.smithi077.stdout:7/544: mknod da/d15/d1a/d39/cba 0 2024-04-26T04:53:07.359 INFO:tasks.workunit.client.1.smithi078.stdout:8/889: dread d0/d19/d52/d35/d44/d6b/d6c/fe9 [0,4194304] 0 2024-04-26T04:53:07.360 INFO:tasks.workunit.client.1.smithi078.stdout:5/876: stat d7/d18/d32/d37/d79/c7b 0 2024-04-26T04:53:07.361 INFO:tasks.workunit.client.1.smithi078.stdout:8/890: dread - d0/d19/d52/d35/d73/fe8 zero size 2024-04-26T04:53:07.361 INFO:tasks.workunit.client.1.smithi078.stdout:5/877: fdatasync d7/d48/d57/d65/d46/de7/fec 0 2024-04-26T04:53:07.362 INFO:tasks.workunit.client.1.smithi078.stdout:1/963: mknod d7/d1e/d9e/db/d49/c138 0 2024-04-26T04:53:07.363 INFO:tasks.workunit.client.1.smithi078.stdout:1/964: fdatasync d7/ff7 0 2024-04-26T04:53:07.367 INFO:tasks.workunit.client.1.smithi078.stdout:6/968: dread d2/de/d27/da9/d2a/d4d/f60 [0,4194304] 0 2024-04-26T04:53:07.371 INFO:tasks.workunit.client.1.smithi078.stdout:2/924: unlink d8/d11/d14/f5f 0 2024-04-26T04:53:07.374 INFO:tasks.workunit.client.1.smithi078.stdout:5/878: creat d7/d48/da9/d10f/f128 x:0 0 0 2024-04-26T04:53:07.375 INFO:tasks.workunit.client.1.smithi078.stdout:1/965: chown d7/d1e/d9e/f3e 109423085 1 2024-04-26T04:53:07.376 INFO:tasks.workunit.client.1.smithi078.stdout:3/921: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/f13f x:0 0 0 2024-04-26T04:53:07.378 INFO:tasks.workunit.client.1.smithi078.stdout:6/969: readlink d2/de/d27/da9/d3a/d69/le3 0 2024-04-26T04:53:07.380 INFO:tasks.workunit.client.0.smithi077.stdout:7/545: dread da/d15/d1a/d39/f7a [0,4194304] 0 2024-04-26T04:53:07.380 INFO:tasks.workunit.client.0.smithi077.stdout:7/546: stat da/f68 0 2024-04-26T04:53:07.381 INFO:tasks.workunit.client.0.smithi077.stdout:8/559: dwrite d4/d69/fa2 [0,4194304] 0 2024-04-26T04:53:07.381 INFO:tasks.workunit.client.1.smithi078.stdout:8/891: dread d0/d19/d52/d35/f8f [0,4194304] 0 2024-04-26T04:53:07.383 INFO:tasks.workunit.client.1.smithi078.stdout:5/879: dwrite d7/d18/d32/d37/d79/da7/fea [0,4194304] 0 2024-04-26T04:53:07.384 INFO:tasks.workunit.client.1.smithi078.stdout:9/844: creat dc/df/d37/ffc x:0 0 0 2024-04-26T04:53:07.386 INFO:tasks.workunit.client.0.smithi077.stdout:7/547: fdatasync da/d15/d1a/d39/f7a 0 2024-04-26T04:53:07.386 INFO:tasks.workunit.client.1.smithi078.stdout:1/966: write d7/d1e/d9e/db/d49/dba/dc5/d134/d3d/f91 [394428,117457] 0 2024-04-26T04:53:07.388 INFO:tasks.workunit.client.0.smithi077.stdout:8/560: truncate d4/df/d18/f4c 660098 0 2024-04-26T04:53:07.391 INFO:tasks.workunit.client.1.smithi078.stdout:4/904: write d0/d1d/de1/d20/d31/f6b [309818,68035] 0 2024-04-26T04:53:07.391 INFO:tasks.workunit.client.1.smithi078.stdout:2/925: mkdir d8/d11/d18/d5d/d8b/db0/d140 0 2024-04-26T04:53:07.393 INFO:tasks.workunit.client.1.smithi078.stdout:3/922: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f58 [0,4194304] 0 2024-04-26T04:53:07.395 INFO:tasks.workunit.client.1.smithi078.stdout:8/892: chown d0/d19/d52/de7 7 1 2024-04-26T04:53:07.397 INFO:tasks.workunit.client.1.smithi078.stdout:5/880: creat d7/d18/d32/f129 x:0 0 0 2024-04-26T04:53:07.397 INFO:tasks.workunit.client.1.smithi078.stdout:8/893: readlink d0/d7a/d83/df3/l110 0 2024-04-26T04:53:07.400 INFO:tasks.workunit.client.1.smithi078.stdout:7/876: dwrite d9/da/d11/d13/d1a/d2d/f3f [0,4194304] 0 2024-04-26T04:53:07.403 INFO:tasks.workunit.client.1.smithi078.stdout:0/913: sync 2024-04-26T04:53:07.403 INFO:tasks.workunit.client.1.smithi078.stdout:9/845: dread dc/df/d27/f5c [0,4194304] 0 2024-04-26T04:53:07.404 INFO:tasks.workunit.client.1.smithi078.stdout:4/905: dread - d0/d1d/db4/f11c zero size 2024-04-26T04:53:07.407 INFO:tasks.workunit.client.1.smithi078.stdout:2/926: creat d8/d11/d18/d5d/d61/df5/d131/f141 x:0 0 0 2024-04-26T04:53:07.408 INFO:tasks.workunit.client.1.smithi078.stdout:5/881: fdatasync d7/d18/f40 0 2024-04-26T04:53:07.409 INFO:tasks.workunit.client.0.smithi077.stdout:7/548: getdents da/d13/d22/d5d 0 2024-04-26T04:53:07.409 INFO:tasks.workunit.client.1.smithi078.stdout:2/927: truncate d8/d11/d18/d137/d34/f126 568383 0 2024-04-26T04:53:07.412 INFO:tasks.workunit.client.0.smithi077.stdout:7/549: write da/d13/d22/f46 [592242,73550] 0 2024-04-26T04:53:07.413 INFO:tasks.workunit.client.0.smithi077.stdout:7/550: stat da/d15/d1a/d39/faf 0 2024-04-26T04:53:07.413 INFO:tasks.workunit.client.1.smithi078.stdout:5/882: dwrite f5 [0,4194304] 0 2024-04-26T04:53:07.416 INFO:tasks.workunit.client.1.smithi078.stdout:3/923: rename d16/d9b/fa7 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/f140 0 2024-04-26T04:53:07.419 INFO:tasks.workunit.client.1.smithi078.stdout:1/967: symlink d7/d1e/d9e/dd/d20/dca/d115/l139 0 2024-04-26T04:53:07.420 INFO:tasks.workunit.client.1.smithi078.stdout:9/846: fdatasync dc/f3d 0 2024-04-26T04:53:07.421 INFO:tasks.workunit.client.0.smithi077.stdout:8/561: dread d4/d17/d35/f52 [0,4194304] 0 2024-04-26T04:53:07.421 INFO:tasks.workunit.client.0.smithi077.stdout:7/551: dwrite da/d13/d22/f58 [0,4194304] 0 2024-04-26T04:53:07.422 INFO:tasks.workunit.client.1.smithi078.stdout:4/906: symlink d0/de/d22/d27/d7b/l13e 0 2024-04-26T04:53:07.422 INFO:tasks.workunit.client.1.smithi078.stdout:3/924: fsync d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/f8c 0 2024-04-26T04:53:07.423 INFO:tasks.workunit.client.1.smithi078.stdout:6/970: dread d2/de/d27/da9/f6b [0,4194304] 0 2024-04-26T04:53:07.424 INFO:tasks.workunit.client.1.smithi078.stdout:5/883: fdatasync d7/d18/d32/f8b 0 2024-04-26T04:53:07.424 INFO:tasks.workunit.client.1.smithi078.stdout:7/877: dread d9/f6a [0,4194304] 0 2024-04-26T04:53:07.426 INFO:tasks.workunit.client.1.smithi078.stdout:7/878: read d9/da/d33/f56 [45918,91493] 0 2024-04-26T04:53:07.426 INFO:tasks.workunit.client.1.smithi078.stdout:2/928: rename d8/l30 to d8/d11/d18/dfe/l142 0 2024-04-26T04:53:07.427 INFO:tasks.workunit.client.1.smithi078.stdout:2/929: dread - d8/d11/d18/dbb/f13e zero size 2024-04-26T04:53:07.427 INFO:tasks.workunit.client.0.smithi077.stdout:7/552: chown da/d6b/d63/c6c 8313 1 2024-04-26T04:53:07.428 INFO:tasks.workunit.client.0.smithi077.stdout:7/553: chown da/d13/d22/l76 154 1 2024-04-26T04:53:07.429 INFO:tasks.workunit.client.1.smithi078.stdout:2/930: fsync d8/d11/d18/dbb/f13e 0 2024-04-26T04:53:07.430 INFO:tasks.workunit.client.1.smithi078.stdout:9/847: dread - dc/df/d8a/fea zero size 2024-04-26T04:53:07.431 INFO:tasks.workunit.client.0.smithi077.stdout:8/562: mkdir d4/df/d22/d2d/d2b/d33/dbc 0 2024-04-26T04:53:07.438 INFO:tasks.workunit.client.1.smithi078.stdout:2/931: read d8/f4e [5740216,130258] 0 2024-04-26T04:53:07.455 INFO:tasks.workunit.client.1.smithi078.stdout:3/925: dread d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/f3d [0,4194304] 0 2024-04-26T04:53:07.460 INFO:tasks.workunit.client.1.smithi078.stdout:3/926: dread fd [0,4194304] 0 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.487+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce00103630 con 0x7fce000fcf60 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.487+0000 7fcdf67fc700 1 -- 172.21.15.77:0/2686618373 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fcdf001a840 con 0x7fce000fcf60 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:53:07.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:53:07.490 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:53:07.491 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:53:07.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 msgr2=0x7fcdec06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 0x7fcdec06dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009fd0 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:07.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 msgr2=0x7fce0010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0011a20 tx=0x7fcdf0009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 shutdown_connections 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fcdec06b790 0x7fcdec06dc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce000fbcb0 0x7fce0010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 --2- 172.21.15.77:0/2686618373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce000fcf60 0x7fce0010bd10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 >> 172.21.15.77:0/2686618373 conn(0x7fce000f7430 msgr2=0x7fce000f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 shutdown_connections 2024-04-26T04:53:07.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.489+0000 7fce07bcb700 1 -- 172.21.15.77:0/2686618373 wait complete. 2024-04-26T04:53:07.538 INFO:tasks.workunit.client.1.smithi078.stdout:5/884: mkdir d7/d11/dee/d12a 0 2024-04-26T04:53:07.540 INFO:tasks.workunit.client.0.smithi077.stdout:8/563: dread d4/df/d22/f50 [0,4194304] 0 2024-04-26T04:53:07.543 INFO:tasks.workunit.client.0.smithi077.stdout:8/564: creat d4/df/d22/d80/d86/fbd x:0 0 0 2024-04-26T04:53:07.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/894: write d0/d19/d52/d5c/fcd [737729,118004] 0 2024-04-26T04:53:07.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/895: write d0/d19/d25/d32/d8a/d92/fcf [1009246,2043] 0 2024-04-26T04:53:07.547 INFO:tasks.workunit.client.1.smithi078.stdout:1/968: mknod d7/d1e/d9e/db/d49/dba/c13a 0 2024-04-26T04:53:07.549 INFO:tasks.workunit.client.1.smithi078.stdout:4/907: mknod d0/de/d22/d27/c13f 0 2024-04-26T04:53:07.553 INFO:tasks.workunit.client.0.smithi077.stdout:8/565: unlink d4/c6 0 2024-04-26T04:53:07.554 INFO:tasks.workunit.client.1.smithi078.stdout:2/932: truncate d8/d11/d18/f98 398874 0 2024-04-26T04:53:07.556 INFO:tasks.workunit.client.1.smithi078.stdout:3/927: creat d16/d26/d2e/d5f/d49/f141 x:0 0 0 2024-04-26T04:53:07.557 INFO:tasks.workunit.client.1.smithi078.stdout:3/928: chown d16/d26/d2e/d5f/d49 15024544 1 2024-04-26T04:53:07.558 INFO:tasks.workunit.client.0.smithi077.stdout:8/566: fsync d4/df/d6f/f7e 0 2024-04-26T04:53:07.558 INFO:tasks.workunit.client.0.smithi077.stdout:8/567: stat d4/d87/f9b 0 2024-04-26T04:53:07.559 INFO:tasks.workunit.client.1.smithi078.stdout:5/885: mknod d7/da/d1e/d91/c12b 0 2024-04-26T04:53:07.563 INFO:tasks.workunit.client.0.smithi077.stdout:8/568: creat d4/d87/fbe x:0 0 0 2024-04-26T04:53:07.563 INFO:tasks.workunit.client.1.smithi078.stdout:0/914: dwrite d2/d9/d31/d4a/fb9 [0,4194304] 0 2024-04-26T04:53:07.564 INFO:tasks.workunit.client.1.smithi078.stdout:7/879: getdents d9/da/d33/d4b/d4c/d5b/d83/dee 0 2024-04-26T04:53:07.566 INFO:tasks.workunit.client.0.smithi077.stdout:7/554: dwrite da/d6b/f93 [0,4194304] 0 2024-04-26T04:53:07.569 INFO:tasks.workunit.client.1.smithi078.stdout:1/969: mkdir d7/d2e/d10d/d13b 0 2024-04-26T04:53:07.570 INFO:tasks.workunit.client.1.smithi078.stdout:6/971: dwrite d2/de/d27/da9/d3a/d42/f59 [0,4194304] 0 2024-04-26T04:53:07.575 INFO:tasks.workunit.client.0.smithi077.stdout:7/555: mknod da/d6b/d63/d64/db0/cbb 0 2024-04-26T04:53:07.576 INFO:tasks.workunit.client.1.smithi078.stdout:4/908: creat d0/d1d/d46/f140 x:0 0 0 2024-04-26T04:53:07.577 INFO:tasks.workunit.client.1.smithi078.stdout:4/909: stat d0/de/d22/d71/dae/cd0 0 2024-04-26T04:53:07.577 INFO:tasks.workunit.client.1.smithi078.stdout:4/910: fsync d0/de/ff7 0 2024-04-26T04:53:07.579 INFO:tasks.workunit.client.1.smithi078.stdout:4/911: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/de0/f116 zero size 2024-04-26T04:53:07.580 INFO:tasks.workunit.client.1.smithi078.stdout:4/912: write d0/d1d/d46/f140 [934936,18437] 0 2024-04-26T04:53:07.581 INFO:tasks.workunit.client.0.smithi077.stdout:7/556: fsync da/d13/d22/d7d/d80/f8b 0 2024-04-26T04:53:07.582 INFO:tasks.workunit.client.1.smithi078.stdout:3/929: mkdir d16/d26/d2e/d5f/d49/dcc/de2/d142 0 2024-04-26T04:53:07.584 INFO:tasks.workunit.client.1.smithi078.stdout:5/886: mknod d7/d18/d32/d37/d8f/c12c 0 2024-04-26T04:53:07.585 INFO:tasks.workunit.client.0.smithi077.stdout:7/557: write da/d15/d1a/f79 [3817760,40988] 0 2024-04-26T04:53:07.586 INFO:tasks.workunit.client.1.smithi078.stdout:8/896: mknod d0/d19/d25/d11a/c11f 0 2024-04-26T04:53:07.587 INFO:tasks.workunit.client.0.smithi077.stdout:7/558: write da/d15/f2f [909479,62670] 0 2024-04-26T04:53:07.589 INFO:tasks.workunit.client.1.smithi078.stdout:1/970: mknod d7/d1e/d9e/db/d37/d58/d5c/dc0/c13c 0 2024-04-26T04:53:07.590 INFO:tasks.workunit.client.1.smithi078.stdout:6/972: mknod d2/d52/c139 0 2024-04-26T04:53:07.592 INFO:tasks.workunit.client.1.smithi078.stdout:8/897: dwrite d0/d19/d52/d35/ff1 [0,4194304] 0 2024-04-26T04:53:07.593 INFO:tasks.workunit.client.0.smithi077.stdout:7/559: truncate da/d13/d22/d7d/d80/f8b 200874 0 2024-04-26T04:53:07.594 INFO:tasks.workunit.client.1.smithi078.stdout:1/971: truncate d7/d1e/d9e/db/d49/dba/d87/db5/f112 78435 0 2024-04-26T04:53:07.595 INFO:tasks.workunit.client.1.smithi078.stdout:2/933: mknod d8/c143 0 2024-04-26T04:53:07.598 INFO:tasks.workunit.client.1.smithi078.stdout:5/887: read - d7/d21/d2f/fc2 zero size 2024-04-26T04:53:07.599 INFO:tasks.workunit.client.1.smithi078.stdout:5/888: write d7/da/d8a/fb7 [4793901,42906] 0 2024-04-26T04:53:07.603 INFO:tasks.workunit.client.1.smithi078.stdout:9/848: getdents dc/df/d4b/d22 0 2024-04-26T04:53:07.604 INFO:tasks.workunit.client.1.smithi078.stdout:6/973: symlink d2/de/l13a 0 2024-04-26T04:53:07.612 INFO:tasks.workunit.client.1.smithi078.stdout:5/889: rename d7/d18/d1c/f7d to d7/d48/d57/d65/d7a/d9c/d104/f12d 0 2024-04-26T04:53:07.612 INFO:tasks.workunit.client.0.smithi077.stdout:8/569: dwrite d4/df/f46 [0,4194304] 0 2024-04-26T04:53:07.617 INFO:tasks.workunit.client.1.smithi078.stdout:6/974: creat d2/de/d27/d58/d10c/f13b x:0 0 0 2024-04-26T04:53:07.618 INFO:tasks.workunit.client.0.smithi077.stdout:8/570: mknod d4/df/d6f/cbf 0 2024-04-26T04:53:07.618 INFO:tasks.workunit.client.0.smithi077.stdout:7/560: dread da/d15/faa [0,4194304] 0 2024-04-26T04:53:07.618 INFO:tasks.workunit.client.1.smithi078.stdout:5/890: dwrite d7/d11/dee/d111/f127 [0,4194304] 0 2024-04-26T04:53:07.619 INFO:tasks.workunit.client.0.smithi077.stdout:8/571: dread - d4/d17/d35/d5b/fa5 zero size 2024-04-26T04:53:07.619 INFO:tasks.workunit.client.1.smithi078.stdout:0/915: write d2/d9/d18/db7/d92/fd3 [708579,17160] 0 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 -- 172.21.15.77:0/1209262960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 msgr2=0x7f1648100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 --2- 172.21.15.77:0/1209262960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648100150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f163c004940 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 -- 172.21.15.77:0/1209262960 shutdown_connections 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 --2- 172.21.15.77:0/1209262960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648100150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 --2- 172.21.15.77:0/1209262960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648066ed0 0x7f16480fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 -- 172.21.15.77:0/1209262960 >> 172.21.15.77:0/1209262960 conn(0x7f16480f7430 msgr2=0x7f16480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:07.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 -- 172.21.15.77:0/1209262960 shutdown_connections 2024-04-26T04:53:07.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.620+0000 7f164dc14700 1 -- 172.21.15.77:0/1209262960 wait complete. 2024-04-26T04:53:07.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 Processor -- start 2024-04-26T04:53:07.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 -- start start 2024-04-26T04:53:07.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648066ed0 0x7f1648113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648114a60 con 0x7f16480fdcd0 2024-04-26T04:53:07.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.621+0000 7f164dc14700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648114bd0 con 0x7f1648066ed0 2024-04-26T04:53:07.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:07.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45838/0 (socket says 172.21.15.77:45838) 2024-04-26T04:53:07.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 -- 172.21.15.77:0/2620106838 learned_addr learned my addr 172.21.15.77:0/2620106838 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:07.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 -- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648066ed0 msgr2=0x7f1648113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648066ed0 0x7f1648113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 -- 172.21.15.77:0/2620106838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c005040 con 0x7f16480fdcd0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.622+0000 7f1646ffd700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b3b0 tx=0x7f163c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.623+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c00e040 con 0x7f16480fdcd0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.623+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f163c004290 con 0x7f16480fdcd0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.623+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c01c630 con 0x7f16480fdcd0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.623+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16481195d0 con 0x7f16480fdcd0 2024-04-26T04:53:07.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.623+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1648119b90 con 0x7f16480fdcd0 2024-04-26T04:53:07.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.624+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164804c710 con 0x7f16480fdcd0 2024-04-26T04:53:07.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.625+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f163c01c790 con 0x7f16480fdcd0 2024-04-26T04:53:07.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.625+0000 7f162ffff700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 0x7f163006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:07.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.625+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f163c08b7b0 con 0x7f16480fdcd0 2024-04-26T04:53:07.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.626+0000 7f16477fe700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 0x7f163006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:07.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.627+0000 7f16477fe700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 0x7f163006dc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1648066c10 tx=0x7f1638007cc0 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:07.630 INFO:tasks.workunit.client.1.smithi078.stdout:4/913: write d0/d1d/de1/d20/d31/f6f [420941,2674] 0 2024-04-26T04:53:07.630 INFO:tasks.workunit.client.1.smithi078.stdout:4/914: chown d0/de/d22/d71/ff0 14 1 2024-04-26T04:53:07.631 INFO:tasks.workunit.client.1.smithi078.stdout:3/930: write d16/f17 [2239049,80460] 0 2024-04-26T04:53:07.633 INFO:tasks.workunit.client.1.smithi078.stdout:7/880: truncate d9/da/d11/d13/d1a/d52/d5a/d41/f51 435480 0 2024-04-26T04:53:07.633 INFO:tasks.workunit.client.1.smithi078.stdout:9/849: mknod dc/df/d37/cfd 0 2024-04-26T04:53:07.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:07.633+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f163c013420 con 0x7f16480fdcd0 2024-04-26T04:53:07.637 INFO:tasks.workunit.client.1.smithi078.stdout:1/972: write d7/d2e/f33 [1718048,61244] 0 2024-04-26T04:53:07.638 INFO:tasks.workunit.client.1.smithi078.stdout:1/973: stat d7/d1e/d9e/db/d49/dba/d87/d119/l130 0 2024-04-26T04:53:07.640 INFO:tasks.workunit.client.1.smithi078.stdout:5/891: mknod d7/d18/d32/d84/c12e 0 2024-04-26T04:53:07.641 INFO:tasks.workunit.client.1.smithi078.stdout:5/892: write d7/d18/d1c/d8e/f103 [788558,50442] 0 2024-04-26T04:53:07.641 INFO:tasks.workunit.client.1.smithi078.stdout:0/916: mknod d2/d9/d31/d4a/d11c/c138 0 2024-04-26T04:53:07.641 INFO:tasks.workunit.client.0.smithi077.stdout:7/561: link da/d13/f19 da/d6b/d63/d64/fbc 0 2024-04-26T04:53:07.643 INFO:tasks.workunit.client.1.smithi078.stdout:0/917: write d2/d9/d18/db7/fd1 [6926357,113672] 0 2024-04-26T04:53:07.643 INFO:tasks.workunit.client.0.smithi077.stdout:8/572: getdents d4/d17/d90 0 2024-04-26T04:53:07.647 INFO:tasks.workunit.client.1.smithi078.stdout:2/934: write d8/d11/d14/d5b/fd0 [380450,63999] 0 2024-04-26T04:53:07.648 INFO:tasks.workunit.client.1.smithi078.stdout:5/893: dwrite d7/d48/d57/d65/d46/de7/fd8 [0,4194304] 0 2024-04-26T04:53:07.649 INFO:tasks.workunit.client.0.smithi077.stdout:8/573: dwrite d4/d69/fa2 [4194304,4194304] 0 2024-04-26T04:53:07.656 INFO:tasks.workunit.client.1.smithi078.stdout:2/935: dwrite d8/f2b [0,4194304] 0 2024-04-26T04:53:07.660 INFO:tasks.workunit.client.1.smithi078.stdout:0/918: dread d2/d1e/d5a/da3/fdd [0,4194304] 0 2024-04-26T04:53:07.661 INFO:tasks.workunit.client.1.smithi078.stdout:0/919: write d2/d9/d31/f50 [4971573,128209] 0 2024-04-26T04:53:07.662 INFO:tasks.workunit.client.1.smithi078.stdout:0/920: chown d2/d9/d18/d7d 2 1 2024-04-26T04:53:07.662 INFO:tasks.workunit.client.1.smithi078.stdout:0/921: chown d2/f1d 232548297 1 2024-04-26T04:53:07.673 INFO:tasks.workunit.client.1.smithi078.stdout:2/936: dread d8/d11/d18/d137/f49 [0,4194304] 0 2024-04-26T04:53:07.674 INFO:tasks.workunit.client.1.smithi078.stdout:2/937: fsync d8/d11/d14/fab 0 2024-04-26T04:53:07.679 INFO:tasks.workunit.client.1.smithi078.stdout:5/894: dread d7/da/d1e/d91/ff6 [0,4194304] 0 2024-04-26T04:53:07.713 INFO:tasks.workunit.client.0.smithi077.stdout:1/456: sync 2024-04-26T04:53:07.729 INFO:tasks.workunit.client.0.smithi077.stdout:8/574: dread d4/df/d22/f43 [0,4194304] 0 2024-04-26T04:53:07.748 INFO:tasks.workunit.client.0.smithi077.stdout:8/575: dwrite d4/d17/d35/d5b/f7f [0,4194304] 0 2024-04-26T04:53:07.751 INFO:tasks.workunit.client.0.smithi077.stdout:8/576: truncate d4/d17/d35/d5b/f8b 857516 0 2024-04-26T04:53:07.752 INFO:tasks.workunit.client.1.smithi078.stdout:8/898: link d0/d19/d25/d32/d8a/d92/da4/fed d0/d19/d52/d35/d44/d6b/f120 0 2024-04-26T04:53:07.755 INFO:tasks.workunit.client.1.smithi078.stdout:4/915: fsync d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/f8d 0 2024-04-26T04:53:07.758 INFO:tasks.workunit.client.1.smithi078.stdout:7/881: mknod d9/da/d11/d13/d50/d8a/c11f 0 2024-04-26T04:53:07.761 INFO:tasks.workunit.client.0.smithi077.stdout:8/577: unlink d4/df/d22/d2d/d2b/f9c 0 2024-04-26T04:53:07.762 INFO:tasks.workunit.client.0.smithi077.stdout:8/578: read d4/d17/d90/fab [319,8103] 0 2024-04-26T04:53:07.762 INFO:tasks.workunit.client.1.smithi078.stdout:9/850: symlink dc/df/d4b/d22/d2a/d57/lfe 0 2024-04-26T04:53:07.763 INFO:tasks.workunit.client.0.smithi077.stdout:8/579: write d4/df/d22/d80/d86/fbd [791631,32434] 0 2024-04-26T04:53:07.764 INFO:tasks.workunit.client.1.smithi078.stdout:6/975: mknod d2/de/d27/da9/da/d36/db9/d97/d105/c13c 0 2024-04-26T04:53:07.765 INFO:tasks.workunit.client.1.smithi078.stdout:1/974: creat d7/d2e/d10d/f13d x:0 0 0 2024-04-26T04:53:07.765 INFO:tasks.workunit.client.1.smithi078.stdout:1/975: readlink d7/d1e/d9e/db/d47/l69 0 2024-04-26T04:53:07.766 INFO:tasks.workunit.client.1.smithi078.stdout:1/976: dread - d7/d2e/d10d/f13d zero size 2024-04-26T04:53:07.767 INFO:tasks.workunit.client.0.smithi077.stdout:8/580: mkdir d4/df/d18/d56/dc0 0 2024-04-26T04:53:07.771 INFO:tasks.workunit.client.1.smithi078.stdout:1/977: dwrite d7/d1e/d9e/db/d37/d58/fef [0,4194304] 0 2024-04-26T04:53:07.775 INFO:tasks.workunit.client.1.smithi078.stdout:2/938: creat d8/d11/d18/d5d/d8b/db0/d10d/d110/f144 x:0 0 0 2024-04-26T04:53:07.776 INFO:tasks.workunit.client.1.smithi078.stdout:5/895: rmdir d7/d48/d57/d65/d7a/d9c/d104/d96/df0 39 2024-04-26T04:53:07.777 INFO:tasks.workunit.client.1.smithi078.stdout:8/899: rmdir d0/d19/d52/de7 39 2024-04-26T04:53:07.779 INFO:tasks.workunit.client.1.smithi078.stdout:4/916: mknod d0/d1d/de1/d20/d31/d47/db2/c141 0 2024-04-26T04:53:07.780 INFO:tasks.workunit.client.1.smithi078.stdout:3/931: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d13a/f143 x:0 0 0 2024-04-26T04:53:07.785 INFO:tasks.workunit.client.1.smithi078.stdout:1/978: mkdir d7/d1e/d9e/db/d49/dba/dc5/d13e 0 2024-04-26T04:53:07.786 INFO:tasks.workunit.client.1.smithi078.stdout:6/976: dwrite d2/de/d27/da9/d3a/d69/dee/f10f [0,4194304] 0 2024-04-26T04:53:07.787 INFO:tasks.workunit.client.1.smithi078.stdout:0/922: fsync d2/d9/d18/d7d/da9/ff0 0 2024-04-26T04:53:07.788 INFO:tasks.workunit.client.1.smithi078.stdout:1/979: chown d7/d1e/d9e/db/d49/dba/dc5/d134/d3d/f91 4237 1 2024-04-26T04:53:07.788 INFO:tasks.workunit.client.1.smithi078.stdout:2/939: rmdir d8/d11/d18/d5d/d8b 39 2024-04-26T04:53:07.789 INFO:tasks.workunit.client.1.smithi078.stdout:5/896: mknod d7/d21/d2f/d90/c12f 0 2024-04-26T04:53:07.789 INFO:tasks.workunit.client.0.smithi077.stdout:8/581: write d4/df/d22/f50 [748687,11000] 0 2024-04-26T04:53:07.792 INFO:tasks.workunit.client.1.smithi078.stdout:3/932: mknod d16/d26/d2e/d5f/d49/dcc/c144 0 2024-04-26T04:53:07.794 INFO:tasks.workunit.client.1.smithi078.stdout:4/917: rename d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/dab/cc9 to d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/c142 0 2024-04-26T04:53:07.795 INFO:tasks.workunit.client.1.smithi078.stdout:1/980: dwrite d7/d1e/d9e/dd/d20/f108 [0,4194304] 0 2024-04-26T04:53:07.802 INFO:tasks.workunit.client.1.smithi078.stdout:8/900: symlink d0/d19/d52/d35/d44/d6b/d104/d58/l121 0 2024-04-26T04:53:07.804 INFO:tasks.workunit.client.0.smithi077.stdout:8/582: creat d4/df/d22/d2d/fc1 x:0 0 0 2024-04-26T04:53:07.806 INFO:tasks.workunit.client.1.smithi078.stdout:2/940: truncate d8/d11/f31 5001449 0 2024-04-26T04:53:07.806 INFO:tasks.workunit.client.1.smithi078.stdout:9/851: dwrite dc/df/d17/f88 [0,4194304] 0 2024-04-26T04:53:07.806 INFO:tasks.workunit.client.1.smithi078.stdout:0/923: dread d2/d9/d18/db7/d92/fca [0,4194304] 0 2024-04-26T04:53:07.807 INFO:tasks.workunit.client.1.smithi078.stdout:5/897: mkdir d7/d21/d2f/d90/d130 0 2024-04-26T04:53:07.809 INFO:tasks.workunit.client.1.smithi078.stdout:7/882: creat d9/da/d11/f120 x:0 0 0 2024-04-26T04:53:07.812 INFO:tasks.workunit.client.1.smithi078.stdout:4/918: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fd9 zero size 2024-04-26T04:53:07.812 INFO:tasks.workunit.client.0.smithi077.stdout:8/583: dread d4/df/d22/d2d/d2b/f5d [0,4194304] 0 2024-04-26T04:53:07.813 INFO:tasks.workunit.client.1.smithi078.stdout:4/919: dread - d0/de/d22/deb/d104/d105/f123 zero size 2024-04-26T04:53:07.815 INFO:tasks.workunit.client.1.smithi078.stdout:6/977: mkdir d2/de/d27/da9/d13d 0 2024-04-26T04:53:07.815 INFO:tasks.workunit.client.1.smithi078.stdout:6/978: readlink d2/de/l6e 0 2024-04-26T04:53:07.817 INFO:tasks.workunit.client.1.smithi078.stdout:6/979: chown d2/de/d27/da9/d2a/dbf/df6/f131 3909545 1 2024-04-26T04:53:07.817 INFO:tasks.workunit.client.0.smithi077.stdout:8/584: rename d4/d17/d35/c55 to d4/df/d22/d2d/d2b/d33/cc2 0 2024-04-26T04:53:07.821 INFO:tasks.workunit.client.0.smithi077.stdout:8/585: rename d4/d17/c5c to d4/d17/d35/d5b/d8d/cc3 0 2024-04-26T04:53:07.822 INFO:tasks.workunit.client.0.smithi077.stdout:8/586: stat d4/df/d18/d56 0 2024-04-26T04:53:07.823 INFO:tasks.workunit.client.1.smithi078.stdout:1/981: creat d7/d1e/d9e/db/d49/dba/dc5/d13e/f13f x:0 0 0 2024-04-26T04:53:07.824 INFO:tasks.workunit.client.1.smithi078.stdout:1/982: fsync d7/d1e/d9e/dd/fd5 0 2024-04-26T04:53:07.824 INFO:tasks.workunit.client.1.smithi078.stdout:4/920: symlink d0/de/d22/d27/d7b/l143 0 2024-04-26T04:53:07.825 INFO:tasks.workunit.client.1.smithi078.stdout:6/980: symlink d2/d1a/d5c/l13e 0 2024-04-26T04:53:07.825 INFO:tasks.workunit.client.1.smithi078.stdout:1/983: chown d7/d1e/d9e/db/d49/dba/d87/d119/c132 210 1 2024-04-26T04:53:07.826 INFO:tasks.workunit.client.0.smithi077.stdout:8/587: symlink d4/df/d6f/da3/lc4 0 2024-04-26T04:53:07.827 INFO:tasks.workunit.client.1.smithi078.stdout:6/981: write d2/de/d27/d58/d5b/f126 [339582,30738] 0 2024-04-26T04:53:07.827 INFO:tasks.workunit.client.1.smithi078.stdout:8/901: mkdir d0/d19/d52/d5c/d122 0 2024-04-26T04:53:07.828 INFO:tasks.workunit.client.1.smithi078.stdout:9/852: dread dc/df/f9f [0,4194304] 0 2024-04-26T04:53:07.829 INFO:tasks.workunit.client.1.smithi078.stdout:8/902: truncate d0/d19/d52/d35/d73/fe8 782640 0 2024-04-26T04:53:07.829 INFO:tasks.workunit.client.1.smithi078.stdout:0/924: rename d2/d9/d18/ce to d2/d9/d60/c139 0 2024-04-26T04:53:07.831 INFO:tasks.workunit.client.1.smithi078.stdout:5/898: mkdir d7/d48/d57/d65/d70/dca/d131 0 2024-04-26T04:53:07.832 INFO:tasks.workunit.client.1.smithi078.stdout:5/899: dread - d7/d48/d57/d65/d46/de7/fdf zero size 2024-04-26T04:53:07.833 INFO:tasks.workunit.client.0.smithi077.stdout:8/588: dwrite d4/d87/fbe [0,4194304] 0 2024-04-26T04:53:07.835 INFO:tasks.workunit.client.0.smithi077.stdout:8/589: write d4/df/d22/d2d/d2b/f7b [1125463,102438] 0 2024-04-26T04:53:07.835 INFO:tasks.workunit.client.1.smithi078.stdout:2/941: truncate d8/d11/d18/d5d/d61/f62 1014503 0 2024-04-26T04:53:07.841 INFO:tasks.workunit.client.1.smithi078.stdout:3/933: write d16/d26/d2e/f9e [710664,93264] 0 2024-04-26T04:53:07.842 INFO:tasks.workunit.client.1.smithi078.stdout:1/984: symlink d7/d1e/d9e/dd/df0/l140 0 2024-04-26T04:53:07.848 INFO:tasks.workunit.client.1.smithi078.stdout:6/982: rename d2/de/d27/d58/d5b/l135 to d2/d1a/d38/dd5/d116/l13f 0 2024-04-26T04:53:07.849 INFO:tasks.workunit.client.1.smithi078.stdout:0/925: mkdir d2/d39/d96/d13a 0 2024-04-26T04:53:07.849 INFO:tasks.workunit.client.1.smithi078.stdout:5/900: mknod d7/d48/d57/d65/d7a/d9c/c132 0 2024-04-26T04:53:07.850 INFO:tasks.workunit.client.1.smithi078.stdout:2/942: readlink d8/d11/d18/d5d/d61/l64 0 2024-04-26T04:53:07.853 INFO:tasks.workunit.client.1.smithi078.stdout:4/921: link d0/d1d/de1/d20/f4f d0/d1d/f144 0 2024-04-26T04:53:07.856 INFO:tasks.workunit.client.1.smithi078.stdout:1/985: creat d7/d1e/d9e/db/d49/dba/dc5/f141 x:0 0 0 2024-04-26T04:53:07.857 INFO:tasks.workunit.client.1.smithi078.stdout:1/986: chown d7/d1e/d9e/db/d37/faa 42261837 1 2024-04-26T04:53:07.857 INFO:tasks.workunit.client.1.smithi078.stdout:3/934: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/f145 x:0 0 0 2024-04-26T04:53:07.861 INFO:tasks.workunit.client.1.smithi078.stdout:7/883: getdents d9/da/d11/d13/d1a 0 2024-04-26T04:53:07.867 INFO:tasks.workunit.client.0.smithi077.stdout:8/590: dwrite d4/df/d22/d2d/d2b/d33/f39 [0,4194304] 0 2024-04-26T04:53:07.867 INFO:tasks.workunit.client.1.smithi078.stdout:8/903: dwrite d0/f23 [0,4194304] 0 2024-04-26T04:53:07.871 INFO:tasks.workunit.client.1.smithi078.stdout:3/935: chown d16/d26/d2e/d5f/dbb/ffd 808285 1 2024-04-26T04:53:07.872 INFO:tasks.workunit.client.1.smithi078.stdout:1/987: chown d7/d1e/d9e/db/d49/dba/dc5/d134/d3d/f120 462607 1 2024-04-26T04:53:07.879 INFO:tasks.workunit.client.1.smithi078.stdout:0/926: unlink d2/d9/d31/c58 0 2024-04-26T04:53:07.884 INFO:tasks.workunit.client.1.smithi078.stdout:7/884: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/d121 0 2024-04-26T04:53:07.884 INFO:tasks.workunit.client.1.smithi078.stdout:4/922: dwrite d0/ffb [0,4194304] 0 2024-04-26T04:53:07.886 INFO:tasks.workunit.client.1.smithi078.stdout:7/885: dread - d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/ff1 zero size 2024-04-26T04:53:07.888 INFO:tasks.workunit.client.1.smithi078.stdout:5/901: truncate d7/d18/d32/d37/d79/da7/fea 2305600 0 2024-04-26T04:53:07.888 INFO:tasks.workunit.client.1.smithi078.stdout:7/886: chown d9/da/d33/fff 89 1 2024-04-26T04:53:07.891 INFO:tasks.workunit.client.0.smithi077.stdout:8/591: truncate d4/df/d22/d2d/d2b/f4d 3783381 0 2024-04-26T04:53:07.893 INFO:tasks.workunit.client.1.smithi078.stdout:9/853: getdents dc/df/d27 0 2024-04-26T04:53:07.893 INFO:tasks.workunit.client.1.smithi078.stdout:8/904: creat d0/d19/d25/d32/d8a/f123 x:0 0 0 2024-04-26T04:53:07.894 INFO:tasks.workunit.client.0.smithi077.stdout:8/592: creat d4/d17/fc5 x:0 0 0 2024-04-26T04:53:07.896 INFO:tasks.workunit.client.0.smithi077.stdout:8/593: dread - d4/df/d22/d80/f8e zero size 2024-04-26T04:53:07.897 INFO:tasks.workunit.client.1.smithi078.stdout:1/988: rmdir d7/d1e/d9e/db/d49/dba/d87/d119 39 2024-04-26T04:53:07.898 INFO:tasks.workunit.client.1.smithi078.stdout:6/983: creat d2/de/d27/da9/da/f140 x:0 0 0 2024-04-26T04:53:07.901 INFO:tasks.workunit.client.0.smithi077.stdout:8/594: readlink d4/d17/l23 0 2024-04-26T04:53:07.901 INFO:tasks.workunit.client.1.smithi078.stdout:8/905: dwrite d0/d19/d25/d32/d8a/f123 [0,4194304] 0 2024-04-26T04:53:07.906 INFO:tasks.workunit.client.1.smithi078.stdout:4/923: mkdir d0/d1d/de1/d20/d31/d145 0 2024-04-26T04:53:07.908 INFO:tasks.workunit.client.0.smithi077.stdout:8/595: write d4/d17/d35/f52 [5109373,125091] 0 2024-04-26T04:53:07.915 INFO:tasks.workunit.client.0.smithi077.stdout:5/470: sync 2024-04-26T04:53:07.915 INFO:tasks.workunit.client.0.smithi077.stdout:4/480: sync 2024-04-26T04:53:07.915 INFO:tasks.workunit.client.1.smithi078.stdout:3/936: dwrite d16/d26/f29 [0,4194304] 0 2024-04-26T04:53:07.915 INFO:tasks.workunit.client.1.smithi078.stdout:7/887: read d9/da/d11/fad [2073865,41101] 0 2024-04-26T04:53:07.916 INFO:tasks.workunit.client.1.smithi078.stdout:3/937: write d16/d9b/f125 [183000,97319] 0 2024-04-26T04:53:07.917 INFO:tasks.workunit.client.1.smithi078.stdout:9/854: creat dc/df/d4b/d22/d2a/d57/fff x:0 0 0 2024-04-26T04:53:07.917 INFO:tasks.workunit.client.1.smithi078.stdout:3/938: write d16/d26/f29 [3995310,7788] 0 2024-04-26T04:53:07.918 INFO:tasks.workunit.client.0.smithi077.stdout:6/542: sync 2024-04-26T04:53:07.919 INFO:tasks.workunit.client.0.smithi077.stdout:5/471: readlink d1/d42/da4/l51 0 2024-04-26T04:53:07.920 INFO:tasks.workunit.client.1.smithi078.stdout:3/939: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/fe6 [567243,54844] 0 2024-04-26T04:53:07.922 INFO:tasks.workunit.client.1.smithi078.stdout:3/940: read d16/f57 [4135779,42935] 0 2024-04-26T04:53:07.929 INFO:tasks.workunit.client.0.smithi077.stdout:5/472: dread d1/d12/d45/f5e [0,4194304] 0 2024-04-26T04:53:07.930 INFO:tasks.workunit.client.0.smithi077.stdout:5/473: stat d1/d12/d28/f2e 0 2024-04-26T04:53:07.931 INFO:tasks.workunit.client.0.smithi077.stdout:6/543: mknod d8/d2a/cb8 0 2024-04-26T04:53:07.931 INFO:tasks.workunit.client.1.smithi078.stdout:2/943: link d8/d11/d18/d137/d34/dd3/dd9/ffa d8/d11/d14/f145 0 2024-04-26T04:53:07.932 INFO:tasks.workunit.client.1.smithi078.stdout:2/944: chown d8/d11/d18/d137/l138 1 1 2024-04-26T04:53:07.937 INFO:tasks.workunit.client.1.smithi078.stdout:5/902: creat d7/d21/d2f/db6/d125/f133 x:0 0 0 2024-04-26T04:53:07.937 INFO:tasks.workunit.client.0.smithi077.stdout:4/481: getdents d4/d17/d18/d43/d53/d56 0 2024-04-26T04:53:07.937 INFO:tasks.workunit.client.1.smithi078.stdout:8/906: write d0/d19/d52/d35/d44/d6b/f99 [781984,72189] 0 2024-04-26T04:53:07.940 INFO:tasks.workunit.client.1.smithi078.stdout:1/989: dwrite d7/d1e/d9e/db/d37/fa9 [0,4194304] 0 2024-04-26T04:53:07.941 INFO:tasks.workunit.client.0.smithi077.stdout:5/474: write d1/d8/fc [2733979,5124] 0 2024-04-26T04:53:07.941 INFO:tasks.workunit.client.0.smithi077.stdout:5/475: fsync d1/d30/d35/f6c 0 2024-04-26T04:53:07.944 INFO:tasks.workunit.client.1.smithi078.stdout:0/927: dwrite d2/d9/d31/d4a/f5d [0,4194304] 0 2024-04-26T04:53:07.947 INFO:tasks.workunit.client.0.smithi077.stdout:5/476: rmdir d1/d12/d87 39 2024-04-26T04:53:07.948 INFO:tasks.workunit.client.1.smithi078.stdout:1/990: write d7/d1e/d9e/dd/d20/f108 [38567,102035] 0 2024-04-26T04:53:07.948 INFO:tasks.workunit.client.0.smithi077.stdout:5/477: write d1/d42/da4/f34 [353189,107527] 0 2024-04-26T04:53:07.950 INFO:tasks.workunit.client.0.smithi077.stdout:6/544: link d8/ca d8/d8f/cb9 0 2024-04-26T04:53:07.951 INFO:tasks.workunit.client.1.smithi078.stdout:3/941: mkdir d16/d146 0 2024-04-26T04:53:07.951 INFO:tasks.workunit.client.0.smithi077.stdout:5/478: dread - d1/d8/d72/f99 zero size 2024-04-26T04:53:07.951 INFO:tasks.workunit.client.0.smithi077.stdout:6/545: dread - d8/d52/da0/f98 zero size 2024-04-26T04:53:07.952 INFO:tasks.workunit.client.1.smithi078.stdout:4/924: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 [0,4194304] 0 2024-04-26T04:53:07.955 INFO:tasks.workunit.client.1.smithi078.stdout:8/907: dwrite d0/d7a/d83/fd7 [0,4194304] 0 2024-04-26T04:53:07.955 INFO:tasks.workunit.client.0.smithi077.stdout:4/482: creat d4/de/d15/d1f/d35/d5c/d8b/fa5 x:0 0 0 2024-04-26T04:53:07.955 INFO:tasks.workunit.client.1.smithi078.stdout:4/925: dread d0/de/d22/d27/d56/fe3 [0,4194304] 0 2024-04-26T04:53:07.956 INFO:tasks.workunit.client.0.smithi077.stdout:4/483: chown d4/d17/f98 19836042 1 2024-04-26T04:53:07.958 INFO:tasks.workunit.client.0.smithi077.stdout:5/479: creat d1/d42/faf x:0 0 0 2024-04-26T04:53:07.959 INFO:tasks.workunit.client.0.smithi077.stdout:5/480: dread - d1/fa3 zero size 2024-04-26T04:53:07.959 INFO:tasks.workunit.client.0.smithi077.stdout:6/546: mknod d8/d1c/cba 0 2024-04-26T04:53:07.962 INFO:tasks.workunit.client.0.smithi077.stdout:4/484: creat d4/de/fa6 x:0 0 0 2024-04-26T04:53:07.962 INFO:tasks.workunit.client.1.smithi078.stdout:4/926: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/fb1 [0,4194304] 0 2024-04-26T04:53:07.963 INFO:tasks.workunit.client.0.smithi077.stdout:4/485: stat d4/d17/f98 0 2024-04-26T04:53:07.963 INFO:tasks.workunit.client.1.smithi078.stdout:4/927: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/dba/fcb zero size 2024-04-26T04:53:07.965 INFO:tasks.workunit.client.0.smithi077.stdout:5/481: mknod d1/d8d/cb0 0 2024-04-26T04:53:07.967 INFO:tasks.workunit.client.1.smithi078.stdout:2/945: creat d8/d11/d18/d137/d34/dd6/f146 x:0 0 0 2024-04-26T04:53:07.968 INFO:tasks.workunit.client.0.smithi077.stdout:6/547: unlink d8/f49 0 2024-04-26T04:53:07.972 INFO:tasks.workunit.client.1.smithi078.stdout:5/903: creat d7/d48/d57/d65/d7a/d9c/d104/f134 x:0 0 0 2024-04-26T04:53:07.973 INFO:tasks.workunit.client.1.smithi078.stdout:5/904: write d7/d21/f42 [1034671,89576] 0 2024-04-26T04:53:07.977 INFO:tasks.workunit.client.0.smithi077.stdout:6/548: fdatasync d8/f88 0 2024-04-26T04:53:07.978 INFO:tasks.workunit.client.1.smithi078.stdout:7/888: dread d9/da/d33/f70 [0,4194304] 0 2024-04-26T04:53:07.980 INFO:tasks.workunit.client.1.smithi078.stdout:0/928: symlink d2/d9/d31/l13b 0 2024-04-26T04:53:07.982 INFO:tasks.workunit.client.1.smithi078.stdout:7/889: dread d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc2 [0,4194304] 0 2024-04-26T04:53:07.983 INFO:tasks.workunit.client.1.smithi078.stdout:7/890: chown d9/da/d11/d13/d50/d8a/f9c 27505 1 2024-04-26T04:53:07.984 INFO:tasks.workunit.client.1.smithi078.stdout:4/928: creat d0/d1d/de1/d20/d31/d47/db2/f146 x:0 0 0 2024-04-26T04:53:07.992 INFO:tasks.workunit.client.0.smithi077.stdout:4/486: dread d4/d17/d18/d43/d53/d56/f2d [0,4194304] 0 2024-04-26T04:53:07.995 INFO:tasks.workunit.client.1.smithi078.stdout:6/984: getdents d2/de/d27/da9/da/d36/db9/d97/d105 0 2024-04-26T04:53:07.997 INFO:tasks.workunit.client.1.smithi078.stdout:5/905: dread d7/d21/f4e [0,4194304] 0 2024-04-26T04:53:07.998 INFO:tasks.workunit.client.1.smithi078.stdout:7/891: creat d9/da/d11/d13/d50/f122 x:0 0 0 2024-04-26T04:53:07.998 INFO:tasks.workunit.client.1.smithi078.stdout:7/892: chown d9/l49 14 1 2024-04-26T04:53:08.002 INFO:tasks.workunit.client.1.smithi078.stdout:2/946: mknod d8/d11/d18/d137/d34/c147 0 2024-04-26T04:53:08.003 INFO:tasks.workunit.client.1.smithi078.stdout:2/947: write d8/d11/d18/d137/d34/d70/def/f121 [99018,119518] 0 2024-04-26T04:53:08.006 INFO:tasks.workunit.client.0.smithi077.stdout:5/482: write d1/d30/d33/f76 [317079,3560] 0 2024-04-26T04:53:08.009 INFO:tasks.workunit.client.1.smithi078.stdout:9/855: write dc/df/d4b/d22/d2a/d92/fa2 [986956,57877] 0 2024-04-26T04:53:08.010 INFO:tasks.workunit.client.1.smithi078.stdout:2/948: dwrite d8/d11/d18/d137/d34/f126 [0,4194304] 0 2024-04-26T04:53:08.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.009+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1648119ff0 con 0x7f16480fdcd0 2024-04-26T04:53:08.011 INFO:tasks.workunit.client.1.smithi078.stdout:3/942: stat d16/d26/d2e/d74/fc8 0 2024-04-26T04:53:08.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.009+0000 7f162ffff700 1 -- 172.21.15.77:0/2620106838 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f163c058cf0 con 0x7f16480fdcd0 2024-04-26T04:53:08.011 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:53:08.011 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:53:08.012 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:53:08.013 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:08.014 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:08.015 INFO:tasks.workunit.client.1.smithi078.stdout:8/908: creat d0/d19/d52/d5c/f124 x:0 0 0 2024-04-26T04:53:08.015 INFO:tasks.workunit.client.1.smithi078.stdout:8/909: read d0/d19/d52/d35/d44/d6b/d104/d58/fe3 [2144055,116870] 0 2024-04-26T04:53:08.015 INFO:tasks.workunit.client.1.smithi078.stdout:6/985: fdatasync d2/de/d27/da9/da/d36/db9/d97/fd2 0 2024-04-26T04:53:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 msgr2=0x7f163006dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 0x7f163006dc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1648066c10 tx=0x7f1638007cc0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 msgr2=0x7f1648114460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b3b0 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:08.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 shutdown_connections 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f163006b7a0 0x7f163006dc50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648066ed0 0x7f1648113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 --2- 172.21.15.77:0/2620106838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16480fdcd0 0x7f1648114460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.013+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 >> 172.21.15.77:0/2620106838 conn(0x7f16480f7430 msgr2=0x7f16480f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.014+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 shutdown_connections 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.014+0000 7f164dc14700 1 -- 172.21.15.77:0/2620106838 wait complete. 2024-04-26T04:53:08.017 INFO:tasks.workunit.client.1.smithi078.stdout:6/986: readlink d2/de/d27/da9/d2a/dbf/d5f/laf 0 2024-04-26T04:53:08.017 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:53:08.019 INFO:tasks.workunit.client.1.smithi078.stdout:4/929: symlink d0/d1d/de1/l147 0 2024-04-26T04:53:08.021 INFO:tasks.workunit.client.1.smithi078.stdout:7/893: dread - d9/da/d11/d13/d1a/d52/d5a/fa1 zero size 2024-04-26T04:53:08.025 INFO:tasks.workunit.client.1.smithi078.stdout:1/991: getdents d7/d1e/d9e/db/d49/d93 0 2024-04-26T04:53:08.028 INFO:tasks.workunit.client.1.smithi078.stdout:9/856: symlink dc/d55/de2/l100 0 2024-04-26T04:53:08.029 INFO:tasks.workunit.client.1.smithi078.stdout:9/857: stat dc/df/d89/lb2 0 2024-04-26T04:53:08.030 INFO:tasks.workunit.client.1.smithi078.stdout:2/949: symlink d8/d11/d18/d137/d34/dd3/dd9/dfc/l148 0 2024-04-26T04:53:08.032 INFO:tasks.workunit.client.1.smithi078.stdout:3/943: creat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/f147 x:0 0 0 2024-04-26T04:53:08.036 INFO:tasks.workunit.client.0.smithi077.stdout:6/549: creat d8/d1c/db5/fbb x:0 0 0 2024-04-26T04:53:08.038 INFO:tasks.workunit.client.1.smithi078.stdout:8/910: rename d0/d19/d52/d35/d44/d6b/d6c/l9a to d0/d19/d52/d35/d44/d6b/d104/d58/d112/l125 0 2024-04-26T04:53:08.038 INFO:tasks.workunit.client.0.smithi077.stdout:4/487: getdents d4/d17/d18/d43/d51 0 2024-04-26T04:53:08.040 INFO:tasks.workunit.client.1.smithi078.stdout:5/906: mknod d7/d21/c135 0 2024-04-26T04:53:08.042 INFO:tasks.workunit.client.1.smithi078.stdout:0/929: dwrite d2/d1e/f2a [4194304,4194304] 0 2024-04-26T04:53:08.043 INFO:tasks.workunit.client.0.smithi077.stdout:5/483: unlink d1/l3d 0 2024-04-26T04:53:08.043 INFO:tasks.workunit.client.1.smithi078.stdout:0/930: fsync d2/d1e/f9e 0 2024-04-26T04:53:08.044 INFO:tasks.workunit.client.1.smithi078.stdout:6/987: mknod d2/de/d27/da9/d2a/dbf/d5f/c141 0 2024-04-26T04:53:08.045 INFO:tasks.workunit.client.0.smithi077.stdout:6/550: symlink d8/d52/d53/lbc 0 2024-04-26T04:53:08.046 INFO:tasks.workunit.client.1.smithi078.stdout:4/930: mkdir d0/de/d22/d27/de7/d148 0 2024-04-26T04:53:08.048 INFO:tasks.workunit.client.0.smithi077.stdout:4/488: truncate d4/de/f47 545650 0 2024-04-26T04:53:08.049 INFO:tasks.workunit.client.1.smithi078.stdout:7/894: fsync d9/da/d33/fb0 0 2024-04-26T04:53:08.052 INFO:tasks.workunit.client.1.smithi078.stdout:4/931: dwrite d0/d1d/de1/d20/d31/d47/db2/d44/d55/f5d [0,4194304] 0 2024-04-26T04:53:08.053 INFO:tasks.workunit.client.0.smithi077.stdout:6/551: symlink d8/d52/d53/d5e/da6/lbd 0 2024-04-26T04:53:08.058 INFO:tasks.workunit.client.0.smithi077.stdout:5/484: unlink d1/d42/da4/d25/d31/d3c/f4e 0 2024-04-26T04:53:08.059 INFO:tasks.workunit.client.0.smithi077.stdout:5/485: dread - d1/d42/faf zero size 2024-04-26T04:53:08.063 INFO:tasks.workunit.client.1.smithi078.stdout:8/911: rename d0/d19/d25/d32/d8a/d92/da4/lfb to d0/d19/d52/d35/d44/d6b/d104/d58/d112/l126 0 2024-04-26T04:53:08.064 INFO:tasks.workunit.client.0.smithi077.stdout:5/486: symlink d1/d30/d35/lb1 0 2024-04-26T04:53:08.065 INFO:tasks.workunit.client.1.smithi078.stdout:5/907: creat d7/d48/d57/d65/d7a/d9c/d104/f136 x:0 0 0 2024-04-26T04:53:08.066 INFO:tasks.workunit.client.0.smithi077.stdout:6/552: creat d8/d52/da0/dad/fbe x:0 0 0 2024-04-26T04:53:08.067 INFO:tasks.workunit.client.0.smithi077.stdout:4/489: creat d4/d17/d18/fa7 x:0 0 0 2024-04-26T04:53:08.068 INFO:tasks.workunit.client.1.smithi078.stdout:0/931: truncate d2/d9/d60/d6a/fc4 1270191 0 2024-04-26T04:53:08.068 INFO:tasks.workunit.client.1.smithi078.stdout:0/932: readlink d2/d9/d60/d7f/l94 0 2024-04-26T04:53:08.069 INFO:tasks.workunit.client.1.smithi078.stdout:6/988: creat d2/de/d27/da9/d2a/dbf/d5f/d11c/f142 x:0 0 0 2024-04-26T04:53:08.069 INFO:tasks.workunit.client.0.smithi077.stdout:6/553: symlink d8/d1c/lbf 0 2024-04-26T04:53:08.072 INFO:tasks.workunit.client.0.smithi077.stdout:9/485: sync 2024-04-26T04:53:08.072 INFO:tasks.workunit.client.0.smithi077.stdout:3/399: sync 2024-04-26T04:53:08.072 INFO:tasks.workunit.client.0.smithi077.stdout:2/492: sync 2024-04-26T04:53:08.075 INFO:tasks.workunit.client.1.smithi078.stdout:7/895: read d9/da/f15 [99363,115536] 0 2024-04-26T04:53:08.077 INFO:tasks.workunit.client.0.smithi077.stdout:4/490: fdatasync d4/de/d15/d1f/f32 0 2024-04-26T04:53:08.083 INFO:tasks.workunit.client.0.smithi077.stdout:3/400: mknod d0/d7/d1f/d76/c7c 0 2024-04-26T04:53:08.083 INFO:tasks.workunit.client.1.smithi078.stdout:8/912: creat d0/d19/d52/d35/d44/d6b/f127 x:0 0 0 2024-04-26T04:53:08.086 INFO:tasks.workunit.client.0.smithi077.stdout:2/493: creat d9/dc/d1d/d1e/d77/fa2 x:0 0 0 2024-04-26T04:53:08.087 INFO:tasks.workunit.client.1.smithi078.stdout:5/908: truncate d7/d48/d57/fab 3101152 0 2024-04-26T04:53:08.092 INFO:tasks.workunit.client.1.smithi078.stdout:9/858: write dc/df/d4b/f86 [2835294,84546] 0 2024-04-26T04:53:08.094 INFO:tasks.workunit.client.0.smithi077.stdout:5/487: unlink d1/d12/d87/c8a 0 2024-04-26T04:53:08.096 INFO:tasks.workunit.client.0.smithi077.stdout:6/554: mknod d8/d52/d53/da7/cc0 0 2024-04-26T04:53:08.098 INFO:tasks.workunit.client.1.smithi078.stdout:1/992: creat d7/d1e/d9e/db/f142 x:0 0 0 2024-04-26T04:53:08.099 INFO:tasks.workunit.client.1.smithi078.stdout:1/993: chown d7/d2e/d10d/d13b 99 1 2024-04-26T04:53:08.099 INFO:tasks.workunit.client.1.smithi078.stdout:1/994: read d7/f125 [71392,22318] 0 2024-04-26T04:53:08.100 INFO:tasks.workunit.client.1.smithi078.stdout:4/932: symlink d0/d1d/db4/d134/l149 0 2024-04-26T04:53:08.104 INFO:tasks.workunit.client.0.smithi077.stdout:3/401: rename d0/d7/d1f/c68 to d0/d9/d32/d29/c7d 0 2024-04-26T04:53:08.106 INFO:tasks.workunit.client.1.smithi078.stdout:7/896: creat d9/da/d11/d13/d1a/d65/d117/f123 x:0 0 0 2024-04-26T04:53:08.109 INFO:tasks.workunit.client.1.smithi078.stdout:3/944: mkdir d16/d3c/d81/da5/d148 0 2024-04-26T04:53:08.115 INFO:tasks.workunit.client.1.smithi078.stdout:5/909: creat d7/d18/d32/d37/d8f/dc5/f137 x:0 0 0 2024-04-26T04:53:08.118 INFO:tasks.workunit.client.0.smithi077.stdout:5/488: mknod d1/d12/d28/d38/cb2 0 2024-04-26T04:53:08.119 INFO:tasks.workunit.client.0.smithi077.stdout:5/489: dread - d1/d12/d28/f9f zero size 2024-04-26T04:53:08.120 INFO:tasks.workunit.client.1.smithi078.stdout:9/859: mknod dc/df/d4b/d22/dbb/c101 0 2024-04-26T04:53:08.120 INFO:tasks.workunit.client.0.smithi077.stdout:5/490: readlink d1/d42/da4/l73 0 2024-04-26T04:53:08.120 INFO:tasks.workunit.client.1.smithi078.stdout:0/933: dwrite d2/d9/d31/d4a/f9b [0,4194304] 0 2024-04-26T04:53:08.128 INFO:tasks.workunit.client.1.smithi078.stdout:1/995: fdatasync d7/d1e/d9e/db/d49/d93/f12a 0 2024-04-26T04:53:08.132 INFO:tasks.workunit.client.1.smithi078.stdout:4/933: symlink d0/de/d22/d71/dae/l14a 0 2024-04-26T04:53:08.133 INFO:tasks.workunit.client.0.smithi077.stdout:5/491: symlink d1/d30/d33/d66/lb3 0 2024-04-26T04:53:08.141 INFO:tasks.workunit.client.0.smithi077.stdout:2/494: creat d9/dc/d1d/d34/d39/fa3 x:0 0 0 2024-04-26T04:53:08.141 INFO:tasks.workunit.client.0.smithi077.stdout:6/555: dread d8/f17 [0,4194304] 0 2024-04-26T04:53:08.142 INFO:tasks.workunit.client.0.smithi077.stdout:9/486: link d9/dd/d1b/d2b/d32/l63 d9/dd/d1b/d2b/d32/laa 0 2024-04-26T04:53:08.143 INFO:tasks.workunit.client.1.smithi078.stdout:8/913: mkdir d0/d19/d25/d32/d8a/d92/d128 0 2024-04-26T04:53:08.145 INFO:tasks.workunit.client.0.smithi077.stdout:5/492: unlink d1/d12/c1e 0 2024-04-26T04:53:08.146 INFO:tasks.workunit.client.0.smithi077.stdout:5/493: truncate d1/d12/d28/f9f 40703 0 2024-04-26T04:53:08.147 INFO:tasks.workunit.client.0.smithi077.stdout:5/494: chown d1/d42/d62/f9e 29 1 2024-04-26T04:53:08.148 INFO:tasks.workunit.client.1.smithi078.stdout:9/860: fdatasync dc/df/d4b/d22/d2a/d57/d69/f96 0 2024-04-26T04:53:08.151 INFO:tasks.workunit.client.1.smithi078.stdout:0/934: creat d2/d9/d18/d7d/da9/dab/f13c x:0 0 0 2024-04-26T04:53:08.151 INFO:tasks.workunit.client.1.smithi078.stdout:0/935: readlink d2/d1e/d5a/l115 0 2024-04-26T04:53:08.156 INFO:tasks.workunit.client.1.smithi078.stdout:7/897: dwrite d9/da/d11/d13/d1a/d52/d5a/d28/f100 [0,4194304] 0 2024-04-26T04:53:08.158 INFO:tasks.workunit.client.0.smithi077.stdout:6/556: mknod d8/d55/cc1 0 2024-04-26T04:53:08.158 INFO:tasks.workunit.client.1.smithi078.stdout:5/910: write d7/d48/d57/d65/d7a/d9c/d104/fbb [988889,88277] 0 2024-04-26T04:53:08.160 INFO:tasks.workunit.client.0.smithi077.stdout:3/402: dwrite d0/fb [8388608,4194304] 0 2024-04-26T04:53:08.163 INFO:tasks.workunit.client.0.smithi077.stdout:3/403: stat d0/d7/d15/f20 0 2024-04-26T04:53:08.164 INFO:tasks.workunit.client.0.smithi077.stdout:2/495: dwrite d9/dc/d1d/d34/d39/d63/d79/f1c [0,4194304] 0 2024-04-26T04:53:08.164 INFO:tasks.workunit.client.0.smithi077.stdout:3/404: readlink d0/d9/d32/l73 0 2024-04-26T04:53:08.166 INFO:tasks.workunit.client.0.smithi077.stdout:5/495: write d1/d12/f2b [2950167,6176] 0 2024-04-26T04:53:08.171 INFO:tasks.workunit.client.0.smithi077.stdout:9/487: creat d9/dd/d1b/fab x:0 0 0 2024-04-26T04:53:08.172 INFO:tasks.workunit.client.0.smithi077.stdout:6/557: symlink d8/d55/lc2 0 2024-04-26T04:53:08.173 INFO:tasks.workunit.client.0.smithi077.stdout:6/558: chown d8/d55/lc2 0 1 2024-04-26T04:53:08.176 INFO:tasks.workunit.client.1.smithi078.stdout:4/934: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/fa4 [0,4194304] 0 2024-04-26T04:53:08.177 INFO:tasks.workunit.client.0.smithi077.stdout:9/488: dread d9/d43/f56 [0,4194304] 0 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 -- 172.21.15.77:0/523375850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 msgr2=0x7fb1900ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 --2- 172.21.15.77:0/523375850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb1900ff9b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb180004940 tx=0x7fb180009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 -- 172.21.15.77:0/523375850 shutdown_connections 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 --2- 172.21.15.77:0/523375850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb190102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 --2- 172.21.15.77:0/523375850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb1900ff9b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.177+0000 7fb194ad0700 1 -- 172.21.15.77:0/523375850 >> 172.21.15.77:0/523375850 conn(0x7fb1900f7420 msgr2=0x7fb1900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:08.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:07 smithi077 ceph-mon[26311]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:08.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:07 smithi077 ceph-mon[26311]: pgmap v23: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 64 MiB/s rd, 200 MiB/s wr, 386 op/s 2024-04-26T04:53:08.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:07 smithi077 ceph-mon[26311]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:08.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:07 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2686618373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:08.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.178+0000 7fb194ad0700 1 -- 172.21.15.77:0/523375850 shutdown_connections 2024-04-26T04:53:08.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.178+0000 7fb194ad0700 1 -- 172.21.15.77:0/523375850 wait complete. 2024-04-26T04:53:08.181 INFO:tasks.workunit.client.0.smithi077.stdout:6/559: rmdir d8/d1c 39 2024-04-26T04:53:08.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 Processor -- start 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 -- start start 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb19010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19010e4d0 con 0x7fb190066f70 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb194ad0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19010e640 con 0x7fb1900ffef0 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb18e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb18e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45848/0 (socket says 172.21.15.77:45848) 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb18e59c700 1 -- 172.21.15.77:0/3580309214 learned_addr learned my addr 172.21.15.77:0/3580309214 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.179+0000 7fb18dd9b700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb19010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.180+0000 7fb18dd9b700 1 -- 172.21.15.77:0/3580309214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 msgr2=0x7fb19010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.180+0000 7fb18dd9b700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.180+0000 7fb18dd9b700 1 -- 172.21.15.77:0/3580309214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180005040 con 0x7fb1900ffef0 2024-04-26T04:53:08.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.180+0000 7fb18e59c700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:53:08.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.180+0000 7fb18dd9b700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb19010ded0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb184009fd0 tx=0x7fb18400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:08.183 INFO:tasks.workunit.client.1.smithi078.stdout:6/989: getdents d2/de/d27/da9/da/d36 0 2024-04-26T04:53:08.184 INFO:tasks.workunit.client.1.smithi078.stdout:9/861: dwrite dc/df/d4b/dae/fc1 [0,4194304] 0 2024-04-26T04:53:08.184 INFO:tasks.workunit.client.0.smithi077.stdout:5/496: dread - d1/d30/d35/f57 zero size 2024-04-26T04:53:08.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.181+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb184004e90 con 0x7fb1900ffef0 2024-04-26T04:53:08.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.181+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb184004e90 con 0x7fb1900ffef0 2024-04-26T04:53:08.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.181+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb18400c7d0 con 0x7fb1900ffef0 2024-04-26T04:53:08.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.181+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb190112ff0 con 0x7fb1900ffef0 2024-04-26T04:53:08.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.181+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb190113500 con 0x7fb1900ffef0 2024-04-26T04:53:08.185 INFO:tasks.workunit.client.0.smithi077.stdout:2/496: dwrite d9/d15/d91/f96 [0,4194304] 0 2024-04-26T04:53:08.185 INFO:tasks.workunit.client.1.smithi078.stdout:7/898: rename d9/da/d20/leb to d9/da/d11/d13/d102/l124 0 2024-04-26T04:53:08.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.183+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19004c710 con 0x7fb1900ffef0 2024-04-26T04:53:08.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.184+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fb184007cc0 con 0x7fb1900ffef0 2024-04-26T04:53:08.186 INFO:tasks.workunit.client.0.smithi077.stdout:2/497: chown f5 745900022 1 2024-04-26T04:53:08.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.185+0000 7fb17effd700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 0x7fb17806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.186+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb184089c80 con 0x7fb1900ffef0 2024-04-26T04:53:08.188 INFO:tasks.workunit.client.0.smithi077.stdout:6/560: dread d8/f3f [0,4194304] 0 2024-04-26T04:53:08.188 INFO:tasks.workunit.client.0.smithi077.stdout:9/489: mknod d9/cac 0 2024-04-26T04:53:08.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.187+0000 7fb18e59c700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 0x7fb17806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.187+0000 7fb18e59c700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 0x7fb17806dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b0d0 tx=0x7fb180009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:08.190 INFO:tasks.workunit.client.0.smithi077.stdout:5/497: fdatasync d1/d30/d35/f6c 0 2024-04-26T04:53:08.191 INFO:tasks.workunit.client.1.smithi078.stdout:3/945: creat d16/d26/d2e/d5f/f149 x:0 0 0 2024-04-26T04:53:08.192 INFO:tasks.workunit.client.0.smithi077.stdout:2/498: symlink d9/dc/d1d/d34/d39/d63/la4 0 2024-04-26T04:53:08.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.194+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb184050100 con 0x7fb1900ffef0 2024-04-26T04:53:08.198 INFO:tasks.workunit.client.0.smithi077.stdout:6/561: fsync d8/d52/d53/f65 0 2024-04-26T04:53:08.200 INFO:tasks.workunit.client.0.smithi077.stdout:2/499: dwrite d9/f12 [4194304,4194304] 0 2024-04-26T04:53:08.202 INFO:tasks.workunit.client.0.smithi077.stdout:5/498: fdatasync d1/d12/d28/d38/f7c 0 2024-04-26T04:53:08.202 INFO:tasks.workunit.client.1.smithi078.stdout:0/936: fsync d2/f2e 0 2024-04-26T04:53:08.203 INFO:tasks.workunit.client.0.smithi077.stdout:3/405: dread d0/d7/f34 [0,4194304] 0 2024-04-26T04:53:08.204 INFO:tasks.workunit.client.1.smithi078.stdout:9/862: symlink dc/df/d4b/d22/l102 0 2024-04-26T04:53:08.205 INFO:tasks.workunit.client.0.smithi077.stdout:9/490: mkdir d9/d3a/d79/d95/dad 0 2024-04-26T04:53:08.208 INFO:tasks.workunit.client.0.smithi077.stdout:6/562: creat d8/d52/d53/da7/fc3 x:0 0 0 2024-04-26T04:53:08.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/996: link d7/d1e/d9e/dd/d20/fb3 d7/d1e/d9e/dd/d20/d12e/f143 0 2024-04-26T04:53:08.211 INFO:tasks.workunit.client.1.smithi078.stdout:1/997: chown d7/d1e/f54 1728 1 2024-04-26T04:53:08.212 INFO:tasks.workunit.client.1.smithi078.stdout:3/946: dread - d16/d26/d2e/d74/fc8 zero size 2024-04-26T04:53:08.212 INFO:tasks.workunit.client.1.smithi078.stdout:1/998: chown d7/d1e/d9e/db/d49/dba/d87/db5/f112 608633272 1 2024-04-26T04:53:08.213 INFO:tasks.workunit.client.0.smithi077.stdout:3/406: dwrite d0/d7/d1f/d4d/f63 [0,4194304] 0 2024-04-26T04:53:08.214 INFO:tasks.workunit.client.0.smithi077.stdout:9/491: dread d9/dd/d1b/d2b/d32/d36/fa7 [0,4194304] 0 2024-04-26T04:53:08.217 INFO:tasks.workunit.client.0.smithi077.stdout:9/492: truncate d9/dd/d1b/fab 286566 0 2024-04-26T04:53:08.218 INFO:tasks.workunit.client.0.smithi077.stdout:9/493: dread - d9/d3a/f89 zero size 2024-04-26T04:53:08.220 INFO:tasks.workunit.client.0.smithi077.stdout:2/500: mknod d9/dc/d6e/ca5 0 2024-04-26T04:53:08.229 INFO:tasks.workunit.client.1.smithi078.stdout:3/947: write d16/d26/d2e/d5f/f11c [900021,95560] 0 2024-04-26T04:53:08.232 INFO:tasks.workunit.client.1.smithi078.stdout:7/899: dread d9/da/d11/d13/d1a/d52/d5a/d28/ffe [4194304,4194304] 0 2024-04-26T04:53:08.232 INFO:tasks.workunit.client.1.smithi078.stdout:1/999: mknod d7/d2e/d10d/c144 0 2024-04-26T04:53:08.232 INFO:tasks.workunit.client.1.smithi078.stdout:7/900: dread - d9/da/d11/d13/d50/f122 zero size 2024-04-26T04:53:08.233 INFO:tasks.workunit.client.1.smithi078.stdout:8/914: link d0/d19/d25/d32/fba d0/d19/d52/f129 0 2024-04-26T04:53:08.233 INFO:tasks.workunit.client.1.smithi078.stdout:7/901: dread - d9/da/d11/d13/d1a/d52/d5a/fa1 zero size 2024-04-26T04:53:08.234 INFO:tasks.workunit.client.1.smithi078.stdout:8/915: chown d0/d19/d25/d32/d8a/d92 1566656 1 2024-04-26T04:53:08.234 INFO:tasks.workunit.client.0.smithi077.stdout:2/501: rmdir d9/dc/d1d/d34/d39/d63/d79 39 2024-04-26T04:53:08.235 INFO:tasks.workunit.client.0.smithi077.stdout:2/502: dread - d9/dc/d1d/d1e/d77/fa2 zero size 2024-04-26T04:53:08.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/937: symlink d2/d9/d78/l13d 0 2024-04-26T04:53:08.238 INFO:tasks.workunit.client.1.smithi078.stdout:3/948: dread - d16/d26/d2e/d5f/d49/dad/db0/fdd zero size 2024-04-26T04:53:08.238 INFO:tasks.workunit.client.1.smithi078.stdout:3/949: readlink d16/d26/d2e/l44 0 2024-04-26T04:53:08.239 INFO:tasks.workunit.client.1.smithi078.stdout:4/935: dwrite d0/de/d22/d27/d56/d5a/fb6 [0,4194304] 0 2024-04-26T04:53:08.240 INFO:tasks.workunit.client.0.smithi077.stdout:2/503: mkdir d9/dc/d5f/d8e/da6 0 2024-04-26T04:53:08.242 INFO:tasks.workunit.client.1.smithi078.stdout:7/902: rename d9/da/d11/d13/d1a/d52/d5a/d28/f73 to d9/da/d11/d13/d1a/f125 0 2024-04-26T04:53:08.243 INFO:tasks.workunit.client.0.smithi077.stdout:5/499: write d1/d42/d62/f6f [2994199,79689] 0 2024-04-26T04:53:08.246 INFO:tasks.workunit.client.1.smithi078.stdout:2/950: sync 2024-04-26T04:53:08.246 INFO:tasks.workunit.client.1.smithi078.stdout:4/936: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/df6/f101 [0,4194304] 0 2024-04-26T04:53:08.249 INFO:tasks.workunit.client.0.smithi077.stdout:7/562: sync 2024-04-26T04:53:08.249 INFO:tasks.workunit.client.0.smithi077.stdout:1/457: sync 2024-04-26T04:53:08.249 INFO:tasks.workunit.client.0.smithi077.stdout:0/455: sync 2024-04-26T04:53:08.251 INFO:tasks.workunit.client.1.smithi078.stdout:9/863: write dc/df/d4b/d22/d2a/d57/d69/db5/fb8 [591433,116853] 0 2024-04-26T04:53:08.253 INFO:tasks.workunit.client.0.smithi077.stdout:6/563: dwrite d8/d52/da0/f7d [0,4194304] 0 2024-04-26T04:53:08.255 INFO:tasks.workunit.client.0.smithi077.stdout:5/500: creat d1/d8d/da7/fb4 x:0 0 0 2024-04-26T04:53:08.257 INFO:tasks.workunit.client.1.smithi078.stdout:6/990: truncate d2/d1a/d38/dd5/ffd 20869 0 2024-04-26T04:53:08.258 INFO:tasks.workunit.client.1.smithi078.stdout:9/864: dwrite dc/df/d37/fd5 [4194304,4194304] 0 2024-04-26T04:53:08.259 INFO:tasks.workunit.client.0.smithi077.stdout:9/494: write d9/dd/d1b/d2b/f2f [449179,113260] 0 2024-04-26T04:53:08.259 INFO:tasks.workunit.client.1.smithi078.stdout:6/991: chown d2/de/d27/da9/d3a/d69/daa/fc6 84288 1 2024-04-26T04:53:08.262 INFO:tasks.workunit.client.0.smithi077.stdout:3/407: dwrite d0/d9/d32/d29/f31 [4194304,4194304] 0 2024-04-26T04:53:08.266 INFO:tasks.workunit.client.0.smithi077.stdout:7/563: mkdir da/d13/d22/d5d/dbd 0 2024-04-26T04:53:08.266 INFO:tasks.workunit.client.0.smithi077.stdout:2/504: write d9/dc/f5c [5179597,60019] 0 2024-04-26T04:53:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:07 smithi078 ceph-mon[33746]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:07 smithi078 ceph-mon[33746]: pgmap v23: 65 pgs: 65 active+clean; 2.5 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 64 MiB/s rd, 200 MiB/s wr, 386 op/s 2024-04-26T04:53:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:07 smithi078 ceph-mon[33746]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:07 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2686618373' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:08.270 INFO:tasks.workunit.client.0.smithi077.stdout:1/458: truncate d9/f1d 756825 0 2024-04-26T04:53:08.270 INFO:tasks.workunit.client.1.smithi078.stdout:4/937: mknod d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/c14b 0 2024-04-26T04:53:08.271 INFO:tasks.workunit.client.1.smithi078.stdout:8/916: link d0/d19/d52/f129 d0/d19/d25/d32/d8a/d92/da4/f12a 0 2024-04-26T04:53:08.273 INFO:tasks.workunit.client.1.smithi078.stdout:4/938: write d0/de/d22/d27/d7b/f82 [4100059,89336] 0 2024-04-26T04:53:08.273 INFO:tasks.workunit.client.0.smithi077.stdout:5/501: unlink d1/fa3 0 2024-04-26T04:53:08.274 INFO:tasks.workunit.client.0.smithi077.stdout:5/502: read d1/d12/d45/f5e [288401,9753] 0 2024-04-26T04:53:08.275 INFO:tasks.workunit.client.1.smithi078.stdout:4/939: read - d0/de/d22/d27/d7b/f117 zero size 2024-04-26T04:53:08.276 INFO:tasks.workunit.client.1.smithi078.stdout:8/917: truncate d0/d19/d52/d5c/df0/ffe 1025771 0 2024-04-26T04:53:08.276 INFO:tasks.workunit.client.0.smithi077.stdout:3/408: creat d0/d7/d1f/d76/f7e x:0 0 0 2024-04-26T04:53:08.276 INFO:tasks.workunit.client.0.smithi077.stdout:3/409: stat d0 0 2024-04-26T04:53:08.277 INFO:tasks.workunit.client.0.smithi077.stdout:3/410: chown d0/d9/c57 611 1 2024-04-26T04:53:08.278 INFO:tasks.workunit.client.1.smithi078.stdout:2/951: dread d8/d11/d18/d137/d34/dd3/de4/f6a [0,4194304] 0 2024-04-26T04:53:08.280 INFO:tasks.workunit.client.0.smithi077.stdout:0/456: unlink d0/d19/d1e/d59/d5a/l8a 0 2024-04-26T04:53:08.282 INFO:tasks.workunit.client.1.smithi078.stdout:0/938: creat d2/d9/f13e x:0 0 0 2024-04-26T04:53:08.282 INFO:tasks.workunit.client.0.smithi077.stdout:7/564: chown da/d6b/c7b 2522176 1 2024-04-26T04:53:08.283 INFO:tasks.workunit.client.1.smithi078.stdout:3/950: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/f40 [0,4194304] 0 2024-04-26T04:53:08.289 INFO:tasks.workunit.client.0.smithi077.stdout:6/564: symlink d8/d1c/db5/lc4 0 2024-04-26T04:53:08.291 INFO:tasks.workunit.client.0.smithi077.stdout:5/503: creat d1/d12/d28/d38/fb5 x:0 0 0 2024-04-26T04:53:08.295 INFO:tasks.workunit.client.1.smithi078.stdout:4/940: unlink d0/d1d/d46/d100/fd1 0 2024-04-26T04:53:08.296 INFO:tasks.workunit.client.0.smithi077.stdout:7/565: symlink da/d13/d22/d7d/d80/lbe 0 2024-04-26T04:53:08.301 INFO:tasks.workunit.client.0.smithi077.stdout:1/459: rename d9/db/l5d to d9/db/dd/d68/l7f 0 2024-04-26T04:53:08.302 INFO:tasks.workunit.client.1.smithi078.stdout:5/911: sync 2024-04-26T04:53:08.304 INFO:tasks.workunit.client.0.smithi077.stdout:6/565: unlink l7 0 2024-04-26T04:53:08.306 INFO:tasks.workunit.client.1.smithi078.stdout:9/865: dread dc/df/d4b/f1c [0,4194304] 0 2024-04-26T04:53:08.308 INFO:tasks.workunit.client.0.smithi077.stdout:5/504: fdatasync d1/d42/d62/f89 0 2024-04-26T04:53:08.309 INFO:tasks.workunit.client.1.smithi078.stdout:2/952: dread d8/d11/d18/d137/d34/dd3/de4/fea [0,4194304] 0 2024-04-26T04:53:08.313 INFO:tasks.workunit.client.1.smithi078.stdout:0/939: dread d2/d1e/d5a/da3/f113 [0,4194304] 0 2024-04-26T04:53:08.313 INFO:tasks.workunit.client.1.smithi078.stdout:4/941: unlink d0/d1d/de1/dd8/f102 0 2024-04-26T04:53:08.314 INFO:tasks.workunit.client.0.smithi077.stdout:7/566: creat da/d15/d1a/d39/d96/fbf x:0 0 0 2024-04-26T04:53:08.316 INFO:tasks.workunit.client.0.smithi077.stdout:2/505: link d9/dc/d1d/d34/f69 d9/d15/d44/d4e/fa7 0 2024-04-26T04:53:08.318 INFO:tasks.workunit.client.1.smithi078.stdout:6/992: dwrite d2/de/fac [0,4194304] 0 2024-04-26T04:53:08.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/566: creat d8/d1c/db5/fc5 x:0 0 0 2024-04-26T04:53:08.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/567: chown d8/d52/d53/c81 1 1 2024-04-26T04:53:08.322 INFO:tasks.workunit.client.1.smithi078.stdout:3/951: truncate d16/d7e/ffe 76940 0 2024-04-26T04:53:08.323 INFO:tasks.workunit.client.0.smithi077.stdout:3/411: write d0/d7/d1f/f51 [52529,130793] 0 2024-04-26T04:53:08.325 INFO:tasks.workunit.client.1.smithi078.stdout:6/993: dread d2/de/d27/da9/d2a/dbf/d5f/f9f [0,4194304] 0 2024-04-26T04:53:08.327 INFO:tasks.workunit.client.1.smithi078.stdout:7/903: mknod d9/da/d11/d13/d1a/d52/d5a/d41/d68/d121/c126 0 2024-04-26T04:53:08.330 INFO:tasks.workunit.client.1.smithi078.stdout:2/953: rename d8/d11/d18/d137/d34/dd3/dd9/dfc/l148 to d8/d11/d18/d137/d34/dd3/l149 0 2024-04-26T04:53:08.331 INFO:tasks.workunit.client.1.smithi078.stdout:2/954: chown d8/d53/f11a 15 1 2024-04-26T04:53:08.333 INFO:tasks.workunit.client.0.smithi077.stdout:7/567: creat da/d13/d22/d5d/d9f/fc0 x:0 0 0 2024-04-26T04:53:08.334 INFO:tasks.workunit.client.1.smithi078.stdout:0/940: rmdir d2/d9/d60/d7f 39 2024-04-26T04:53:08.338 INFO:tasks.workunit.client.0.smithi077.stdout:2/506: rename d9/f66 to d9/d15/d44/fa8 0 2024-04-26T04:53:08.339 INFO:tasks.workunit.client.1.smithi078.stdout:4/942: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/de0/f14c x:0 0 0 2024-04-26T04:53:08.340 INFO:tasks.workunit.client.0.smithi077.stdout:6/568: read - d8/d1c/f8a zero size 2024-04-26T04:53:08.341 INFO:tasks.workunit.client.1.smithi078.stdout:8/918: creat d0/d19/d52/d35/d44/d6b/d104/f12b x:0 0 0 2024-04-26T04:53:08.343 INFO:tasks.workunit.client.1.smithi078.stdout:8/919: read d0/d19/d52/d35/d44/d6b/d104/d58/fb7 [1178576,93995] 0 2024-04-26T04:53:08.344 INFO:tasks.workunit.client.1.smithi078.stdout:8/920: chown d0/d19/l22 2 1 2024-04-26T04:53:08.347 INFO:tasks.workunit.client.0.smithi077.stdout:0/457: dwrite d0/d28/f8e [0,4194304] 0 2024-04-26T04:53:08.348 INFO:tasks.workunit.client.0.smithi077.stdout:0/458: chown d0/d19/d1e/d2f 65078 1 2024-04-26T04:53:08.354 INFO:tasks.workunit.client.0.smithi077.stdout:2/507: fsync d9/dc/d1d/d1e/f5e 0 2024-04-26T04:53:08.359 INFO:tasks.workunit.client.1.smithi078.stdout:2/955: fdatasync d8/d53/f71 0 2024-04-26T04:53:08.359 INFO:tasks.workunit.client.0.smithi077.stdout:5/505: dwrite d1/d30/f53 [0,4194304] 0 2024-04-26T04:53:08.361 INFO:tasks.workunit.client.0.smithi077.stdout:6/569: rmdir d8/d55 39 2024-04-26T04:53:08.366 INFO:tasks.workunit.client.0.smithi077.stdout:0/459: mkdir d0/d47/d93/da3 0 2024-04-26T04:53:08.371 INFO:tasks.workunit.client.1.smithi078.stdout:8/921: rename d0/d19/d52/d35/d44/d6b/d104/d58/d5b/dee/f118 to d0/d7a/d83/f12c 0 2024-04-26T04:53:08.372 INFO:tasks.workunit.client.1.smithi078.stdout:8/922: chown d0/d19/d52/d5c/d122 244 1 2024-04-26T04:53:08.372 INFO:tasks.workunit.client.0.smithi077.stdout:1/460: dread d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:53:08.373 INFO:tasks.workunit.client.0.smithi077.stdout:1/461: chown d9/d53/d6e/c48 54 1 2024-04-26T04:53:08.373 INFO:tasks.workunit.client.1.smithi078.stdout:5/912: rmdir d7/d18/d32/d37/d79/da7/d113 0 2024-04-26T04:53:08.375 INFO:tasks.workunit.client.0.smithi077.stdout:5/506: mknod d1/d30/d33/d66/cb6 0 2024-04-26T04:53:08.376 INFO:tasks.workunit.client.0.smithi077.stdout:5/507: read d1/fa5 [61865,16548] 0 2024-04-26T04:53:08.377 INFO:tasks.workunit.client.1.smithi078.stdout:3/952: creat d16/d9b/d139/f14a x:0 0 0 2024-04-26T04:53:08.377 INFO:tasks.workunit.client.0.smithi077.stdout:5/508: dread d1/d12/d28/f9f [0,4194304] 0 2024-04-26T04:53:08.377 INFO:tasks.workunit.client.0.smithi077.stdout:7/568: write da/fc [5095672,57793] 0 2024-04-26T04:53:08.380 INFO:tasks.workunit.client.0.smithi077.stdout:7/569: truncate da/d15/d42/d8c/f9b 4287104 0 2024-04-26T04:53:08.381 INFO:tasks.workunit.client.1.smithi078.stdout:9/866: rename dc/df/d4b/d22/d2a/d92 to dc/df/d4b/d22/d2a/d103 0 2024-04-26T04:53:08.384 INFO:tasks.workunit.client.1.smithi078.stdout:6/994: fdatasync d2/de/d27/da9/d2a/d4d/fa7 0 2024-04-26T04:53:08.385 INFO:tasks.workunit.client.0.smithi077.stdout:0/460: creat d0/d19/d1e/d2f/fa4 x:0 0 0 2024-04-26T04:53:08.386 INFO:tasks.workunit.client.0.smithi077.stdout:0/461: readlink d0/l6 0 2024-04-26T04:53:08.388 INFO:tasks.workunit.client.0.smithi077.stdout:1/462: symlink d9/d53/d6e/l80 0 2024-04-26T04:53:08.388 INFO:tasks.workunit.client.1.smithi078.stdout:4/943: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/fa6 [0,4194304] 0 2024-04-26T04:53:08.388 INFO:tasks.workunit.client.1.smithi078.stdout:4/944: chown d0/d1d/db4/d134/l149 1 1 2024-04-26T04:53:08.389 INFO:tasks.workunit.client.1.smithi078.stdout:4/945: write d0/d1d/de1/d20/d31/d47/db2/ff9 [493752,126364] 0 2024-04-26T04:53:08.390 INFO:tasks.workunit.client.1.smithi078.stdout:2/956: mkdir d8/d11/d18/dfe/d14a 0 2024-04-26T04:53:08.394 INFO:tasks.workunit.client.0.smithi077.stdout:6/570: write d8/d1c/f28 [2969720,92793] 0 2024-04-26T04:53:08.397 INFO:tasks.workunit.client.1.smithi078.stdout:2/957: dwrite d8/d11/d18/d137/d34/dd3/de4/d8e/f112 [0,4194304] 0 2024-04-26T04:53:08.397 INFO:tasks.workunit.client.0.smithi077.stdout:3/412: truncate d0/f38 524766 0 2024-04-26T04:53:08.398 INFO:tasks.workunit.client.0.smithi077.stdout:3/413: read d0/d9/f30 [7702049,59061] 0 2024-04-26T04:53:08.401 INFO:tasks.workunit.client.0.smithi077.stdout:7/570: creat da/d6b/d63/d64/fc1 x:0 0 0 2024-04-26T04:53:08.402 INFO:tasks.workunit.client.1.smithi078.stdout:8/923: unlink d0/d19/c41 0 2024-04-26T04:53:08.403 INFO:tasks.workunit.client.1.smithi078.stdout:7/904: sync 2024-04-26T04:53:08.404 INFO:tasks.workunit.client.1.smithi078.stdout:0/941: write d2/d9/d18/db7/d92/fad [1131617,78139] 0 2024-04-26T04:53:08.408 INFO:tasks.workunit.client.0.smithi077.stdout:2/508: rename d9/fd to d9/dc/d1d/d34/fa9 0 2024-04-26T04:53:08.408 INFO:tasks.workunit.client.1.smithi078.stdout:9/867: symlink dc/df/d4b/d22/d2d/l104 0 2024-04-26T04:53:08.409 INFO:tasks.workunit.client.1.smithi078.stdout:8/924: dwrite d0/f116 [0,4194304] 0 2024-04-26T04:53:08.411 INFO:tasks.workunit.client.1.smithi078.stdout:6/995: mknod d2/de/d27/de8/c143 0 2024-04-26T04:53:08.413 INFO:tasks.workunit.client.1.smithi078.stdout:9/868: truncate dc/df/d4b/d22/d2d/d98/fbd 346588 0 2024-04-26T04:53:08.413 INFO:tasks.workunit.client.1.smithi078.stdout:9/869: chown dc 12250539 1 2024-04-26T04:53:08.415 INFO:tasks.workunit.client.1.smithi078.stdout:4/946: mknod d0/de/d22/d27/d7b/c14d 0 2024-04-26T04:53:08.415 INFO:tasks.workunit.client.0.smithi077.stdout:2/509: dwrite d9/dc/f41 [0,4194304] 0 2024-04-26T04:53:08.417 INFO:tasks.workunit.client.1.smithi078.stdout:8/925: dwrite d0/fae [0,4194304] 0 2024-04-26T04:53:08.418 INFO:tasks.workunit.client.1.smithi078.stdout:4/947: fsync d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f90 0 2024-04-26T04:53:08.424 INFO:tasks.workunit.client.0.smithi077.stdout:5/509: rename d1/d12/d28/c80 to d1/d30/d33/d66/cb7 0 2024-04-26T04:53:08.429 INFO:tasks.workunit.client.1.smithi078.stdout:0/942: unlink d2/d9/d31/d4a/fb9 0 2024-04-26T04:53:08.433 INFO:tasks.workunit.client.0.smithi077.stdout:3/414: mknod d0/d9/c7f 0 2024-04-26T04:53:08.435 INFO:tasks.workunit.client.1.smithi078.stdout:7/905: symlink d9/da/d33/d4b/d4c/d5b/dc3/l127 0 2024-04-26T04:53:08.436 INFO:tasks.workunit.client.0.smithi077.stdout:7/571: mknod da/d13/d22/d5d/dbd/cc2 0 2024-04-26T04:53:08.439 INFO:tasks.workunit.client.0.smithi077.stdout:3/415: dwrite d0/d7/d15/f22 [0,4194304] 0 2024-04-26T04:53:08.441 INFO:tasks.workunit.client.0.smithi077.stdout:3/416: dread - d0/d9/d32/d2a/f4e zero size 2024-04-26T04:53:08.442 INFO:tasks.workunit.client.0.smithi077.stdout:0/462: rmdir d0/d47/d93/da3 0 2024-04-26T04:53:08.442 INFO:tasks.workunit.client.0.smithi077.stdout:6/571: write d8/ff [6362201,110604] 0 2024-04-26T04:53:08.443 INFO:tasks.workunit.client.1.smithi078.stdout:9/870: dread dc/df/d4b/d22/d2a/d57/fbf [0,4194304] 0 2024-04-26T04:53:08.445 INFO:tasks.workunit.client.0.smithi077.stdout:3/417: dread d0/fb [8388608,4194304] 0 2024-04-26T04:53:08.445 INFO:tasks.workunit.client.1.smithi078.stdout:6/996: fsync d2/de/f33 0 2024-04-26T04:53:08.453 INFO:tasks.workunit.client.0.smithi077.stdout:7/572: fsync da/d13/d22/d7d/f85 0 2024-04-26T04:53:08.455 INFO:tasks.workunit.client.0.smithi077.stdout:1/463: dwrite d9/f21 [0,4194304] 0 2024-04-26T04:53:08.457 INFO:tasks.workunit.client.0.smithi077.stdout:2/510: mknod d9/dc/d1d/d34/caa 0 2024-04-26T04:53:08.458 INFO:tasks.workunit.client.1.smithi078.stdout:8/926: mkdir d0/d19/d52/d35/d44/d6b/d6c/db4/d12d 0 2024-04-26T04:53:08.459 INFO:tasks.workunit.client.1.smithi078.stdout:0/943: dread d2/d9/d31/fe0 [4194304,4194304] 0 2024-04-26T04:53:08.465 INFO:tasks.workunit.client.1.smithi078.stdout:8/927: dwrite d0/d19/d52/d35/d44/d6b/d104/f12b [0,4194304] 0 2024-04-26T04:53:08.466 INFO:tasks.workunit.client.1.smithi078.stdout:5/913: rename d7/d48/d57/d65/d7a/d9c to d7/d21/d138 0 2024-04-26T04:53:08.468 INFO:tasks.workunit.client.0.smithi077.stdout:0/463: creat d0/d47/d68/fa5 x:0 0 0 2024-04-26T04:53:08.470 INFO:tasks.workunit.client.0.smithi077.stdout:3/418: write d0/d7/f34 [1975708,33420] 0 2024-04-26T04:53:08.471 INFO:tasks.workunit.client.0.smithi077.stdout:3/419: readlink d0/d7/d1f/l55 0 2024-04-26T04:53:08.473 INFO:tasks.workunit.client.0.smithi077.stdout:5/510: mknod d1/d42/dac/cb8 0 2024-04-26T04:53:08.475 INFO:tasks.workunit.client.1.smithi078.stdout:3/953: truncate d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/d98/ff9 60245 0 2024-04-26T04:53:08.478 INFO:tasks.workunit.client.1.smithi078.stdout:9/871: symlink dc/df/d4b/d22/l105 0 2024-04-26T04:53:08.479 INFO:tasks.workunit.client.1.smithi078.stdout:9/872: stat dc/df/d4b/dae/ff5 0 2024-04-26T04:53:08.481 INFO:tasks.workunit.client.0.smithi077.stdout:7/573: rmdir da/d6b 39 2024-04-26T04:53:08.482 INFO:tasks.workunit.client.0.smithi077.stdout:8/596: sync 2024-04-26T04:53:08.484 INFO:tasks.workunit.client.0.smithi077.stdout:6/572: dwrite d8/f17 [0,4194304] 0 2024-04-26T04:53:08.487 INFO:tasks.workunit.client.0.smithi077.stdout:2/511: truncate d9/dc/d1d/d1e/f5e 87654 0 2024-04-26T04:53:08.487 INFO:tasks.workunit.client.0.smithi077.stdout:2/512: fdatasync d9/d15/d91/f96 0 2024-04-26T04:53:08.488 INFO:tasks.workunit.client.0.smithi077.stdout:8/597: read d4/f9 [378041,30651] 0 2024-04-26T04:53:08.493 INFO:tasks.workunit.client.0.smithi077.stdout:0/464: creat d0/d19/d69/fa6 x:0 0 0 2024-04-26T04:53:08.497 INFO:tasks.workunit.client.1.smithi078.stdout:5/914: unlink d7/da/d1e/f4c 0 2024-04-26T04:53:08.498 INFO:tasks.workunit.client.1.smithi078.stdout:5/915: write d7/da/f116 [491257,86126] 0 2024-04-26T04:53:08.504 INFO:tasks.workunit.client.0.smithi077.stdout:6/573: unlink d8/d1c/da4/fa8 0 2024-04-26T04:53:08.505 INFO:tasks.workunit.client.1.smithi078.stdout:9/873: mkdir dc/df/d4b/d22/d2a/d57/d69/d106 0 2024-04-26T04:53:08.509 INFO:tasks.workunit.client.1.smithi078.stdout:2/958: getdents d8/d11/d18/d137 0 2024-04-26T04:53:08.510 INFO:tasks.workunit.client.1.smithi078.stdout:2/959: chown d8/d11/fbe 3 1 2024-04-26T04:53:08.510 INFO:tasks.workunit.client.0.smithi077.stdout:1/464: write d9/db/f60 [1049904,30113] 0 2024-04-26T04:53:08.512 INFO:tasks.workunit.client.1.smithi078.stdout:4/948: creat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f14e x:0 0 0 2024-04-26T04:53:08.513 INFO:tasks.workunit.client.1.smithi078.stdout:4/949: readlink d0/de/d22/d71/dae/l14a 0 2024-04-26T04:53:08.513 INFO:tasks.workunit.client.0.smithi077.stdout:3/420: mknod d0/d7/c80 0 2024-04-26T04:53:08.513 INFO:tasks.workunit.client.0.smithi077.stdout:3/421: stat d0/c45 0 2024-04-26T04:53:08.514 INFO:tasks.workunit.client.1.smithi078.stdout:4/950: write d0/ffb [2119894,65500] 0 2024-04-26T04:53:08.515 INFO:tasks.workunit.client.1.smithi078.stdout:4/951: chown d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e 0 1 2024-04-26T04:53:08.517 INFO:tasks.workunit.client.0.smithi077.stdout:7/574: creat da/d6b/fc3 x:0 0 0 2024-04-26T04:53:08.517 INFO:tasks.workunit.client.0.smithi077.stdout:8/598: write d4/df/f1f [5424107,54684] 0 2024-04-26T04:53:08.518 INFO:tasks.workunit.client.0.smithi077.stdout:3/422: dread d0/d7/d15/f20 [0,4194304] 0 2024-04-26T04:53:08.521 INFO:tasks.workunit.client.0.smithi077.stdout:6/574: truncate d8/d52/d53/d5e/d6c/f84 398611 0 2024-04-26T04:53:08.522 INFO:tasks.workunit.client.1.smithi078.stdout:2/960: dread d8/d11/d14/f3b [8388608,4194304] 0 2024-04-26T04:53:08.523 INFO:tasks.workunit.client.1.smithi078.stdout:2/961: chown d8/d11/d18/d137/l138 124 1 2024-04-26T04:53:08.523 INFO:tasks.workunit.client.1.smithi078.stdout:6/997: link d2/de/d27/da9/d2a/dbf/df6/f131 d2/de/d27/da9/d2a/dbf/f144 0 2024-04-26T04:53:08.526 INFO:tasks.workunit.client.0.smithi077.stdout:0/465: mkdir d0/d19/d1e/da7 0 2024-04-26T04:53:08.527 INFO:tasks.workunit.client.0.smithi077.stdout:0/466: readlink d0/d19/d1e/d59/l74 0 2024-04-26T04:53:08.528 INFO:tasks.workunit.client.1.smithi078.stdout:3/954: rmdir d16/d146 0 2024-04-26T04:53:08.530 INFO:tasks.workunit.client.1.smithi078.stdout:4/952: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/f52 [0,4194304] 0 2024-04-26T04:53:08.530 INFO:tasks.workunit.client.1.smithi078.stdout:2/962: dwrite d8/d11/d18/d5d/d61/df5/d131/f141 [0,4194304] 0 2024-04-26T04:53:08.530 INFO:tasks.workunit.client.1.smithi078.stdout:7/906: getdents d9/da/d11/d13/d1a/d52/d5a/d28 0 2024-04-26T04:53:08.532 INFO:tasks.workunit.client.1.smithi078.stdout:7/907: write d9/da/d33/d4b/d4c/d5b/ff0 [3942074,432] 0 2024-04-26T04:53:08.533 INFO:tasks.workunit.client.0.smithi077.stdout:8/599: dread d4/d17/f6a [0,4194304] 0 2024-04-26T04:53:08.534 INFO:tasks.workunit.client.0.smithi077.stdout:7/575: mkdir da/d6b/d63/d64/dc4 0 2024-04-26T04:53:08.535 INFO:tasks.workunit.client.0.smithi077.stdout:0/467: dwrite d0/d47/d68/f86 [0,4194304] 0 2024-04-26T04:53:08.536 INFO:tasks.workunit.client.0.smithi077.stdout:7/576: chown da/d13/d22/l76 1 1 2024-04-26T04:53:08.537 INFO:tasks.workunit.client.1.smithi078.stdout:8/928: sync 2024-04-26T04:53:08.538 INFO:tasks.workunit.client.0.smithi077.stdout:8/600: truncate d4/d17/d35/d5b/fa0 809161 0 2024-04-26T04:53:08.538 INFO:tasks.workunit.client.0.smithi077.stdout:3/423: fdatasync d0/d7/d1f/f4a 0 2024-04-26T04:53:08.541 INFO:tasks.workunit.client.0.smithi077.stdout:6/575: unlink d8/d52/d53/d5e/c95 0 2024-04-26T04:53:08.543 INFO:tasks.workunit.client.0.smithi077.stdout:6/576: chown d8/d1c/l92 7 1 2024-04-26T04:53:08.544 INFO:tasks.workunit.client.0.smithi077.stdout:2/513: creat d9/d15/fab x:0 0 0 2024-04-26T04:53:08.546 INFO:tasks.workunit.client.0.smithi077.stdout:2/514: read d9/dc/d1d/d34/f84 [3432692,89675] 0 2024-04-26T04:53:08.547 INFO:tasks.workunit.client.0.smithi077.stdout:1/465: mkdir d9/db/dd/d68/d81 0 2024-04-26T04:53:08.548 INFO:tasks.workunit.client.1.smithi078.stdout:3/955: stat d16/d26/d2e/f51 0 2024-04-26T04:53:08.550 INFO:tasks.workunit.client.0.smithi077.stdout:1/466: dread d9/db/dd/d34/f51 [0,4194304] 0 2024-04-26T04:53:08.550 INFO:tasks.workunit.client.0.smithi077.stdout:5/511: getdents d1/d42/da4/d84 0 2024-04-26T04:53:08.551 INFO:tasks.workunit.client.0.smithi077.stdout:1/467: fsync d9/f73 0 2024-04-26T04:53:08.551 INFO:tasks.workunit.client.1.smithi078.stdout:8/929: dread d0/d19/d25/d32/f76 [0,4194304] 0 2024-04-26T04:53:08.554 INFO:tasks.workunit.client.1.smithi078.stdout:8/930: read d0/d19/d25/d32/d8a/d92/da4/fed [2118920,63771] 0 2024-04-26T04:53:08.554 INFO:tasks.workunit.client.0.smithi077.stdout:0/468: rename d0/d19/d69/da2 to d0/d19/d1e/d2f/da8 0 2024-04-26T04:53:08.558 INFO:tasks.workunit.client.1.smithi078.stdout:9/874: rename dc/df/d4b/d22/c5b to dc/df/c107 0 2024-04-26T04:53:08.561 INFO:tasks.workunit.client.1.smithi078.stdout:7/908: fdatasync d9/da/d11/d13/d1a/d2d/f10b 0 2024-04-26T04:53:08.562 INFO:tasks.workunit.client.1.smithi078.stdout:0/944: link d2/d9/d31/d4a/l59 d2/d9/l13f 0 2024-04-26T04:53:08.564 INFO:tasks.workunit.client.1.smithi078.stdout:6/998: dwrite d2/de/d27/da9/f8 [0,4194304] 0 2024-04-26T04:53:08.570 INFO:tasks.workunit.client.1.smithi078.stdout:3/956: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/d98/d14b 0 2024-04-26T04:53:08.573 INFO:tasks.workunit.client.1.smithi078.stdout:5/916: getdents d7/d18 0 2024-04-26T04:53:08.574 INFO:tasks.workunit.client.1.smithi078.stdout:5/917: fdatasync d7/da/fe8 0 2024-04-26T04:53:08.575 INFO:tasks.workunit.client.1.smithi078.stdout:4/953: fdatasync d0/d1d/de1/d20/d31/fb9 0 2024-04-26T04:53:08.577 INFO:tasks.workunit.client.0.smithi077.stdout:2/515: unlink d9/dc/d1d/d1e/d77/f92 0 2024-04-26T04:53:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.576+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1900f8f60 con 0x7fb17806b7f0 2024-04-26T04:53:08.578 INFO:tasks.workunit.client.1.smithi078.stdout:3/957: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/da8/f134 [0,4194304] 0 2024-04-26T04:53:08.578 INFO:tasks.workunit.client.1.smithi078.stdout:2/963: mkdir d8/d11/d18/d5d/d61/df5/d102/d14b 0 2024-04-26T04:53:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.578+0000 7fb17effd700 1 -- 172.21.15.77:0/3580309214 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb1900f8f60 con 0x7fb17806b7f0 2024-04-26T04:53:08.579 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:53:08.579 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:53:08.579 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:53:08.579 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:53:08.580 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:53:08.580 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "1/21 daemons upgraded", 2024-04-26T04:53:08.580 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "", 2024-04-26T04:53:08.580 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:53:08.580 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:53:08.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 msgr2=0x7fb17806dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 0x7fb17806dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b0d0 tx=0x7fb180009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 msgr2=0x7fb19010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb19010ded0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb184009fd0 tx=0x7fb18400b040 comp rx=0 tx=0).stop 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 shutdown_connections 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fb17806b7f0 0x7fb17806dca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb190066f70 0x7fb19010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 --2- 172.21.15.77:0/3580309214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1900ffef0 0x7fb19010ded0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 >> 172.21.15.77:0/3580309214 conn(0x7fb1900f7420 msgr2=0x7fb1900f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 shutdown_connections 2024-04-26T04:53:08.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.581+0000 7fb194ad0700 1 -- 172.21.15.77:0/3580309214 wait complete. 2024-04-26T04:53:08.584 INFO:tasks.workunit.client.1.smithi078.stdout:9/875: rename dc/df/d27/d81/f91 to dc/df/d4b/d22/d2d/f108 0 2024-04-26T04:53:08.589 INFO:tasks.workunit.client.1.smithi078.stdout:0/945: creat d2/d9/d60/dae/f140 x:0 0 0 2024-04-26T04:53:08.591 INFO:tasks.workunit.client.0.smithi077.stdout:0/469: mkdir d0/d47/d93/d94/da9 0 2024-04-26T04:53:08.591 INFO:tasks.workunit.client.1.smithi078.stdout:6/999: chown d2/de/d27/da9/d11 7376082 1 2024-04-26T04:53:08.592 INFO:tasks.workunit.client.0.smithi077.stdout:0/470: readlink d0/d19/d69/l7b 0 2024-04-26T04:53:08.593 INFO:tasks.workunit.client.0.smithi077.stdout:8/601: mkdir d4/df/dc6 0 2024-04-26T04:53:08.599 INFO:tasks.workunit.client.1.smithi078.stdout:5/918: unlink d7/d48/d57/d65/d46/ff7 0 2024-04-26T04:53:08.601 INFO:tasks.workunit.client.0.smithi077.stdout:6/577: symlink d8/d2a/lc6 0 2024-04-26T04:53:08.601 INFO:tasks.workunit.client.1.smithi078.stdout:7/909: dwrite d9/da/d11/d13/d1a/d52/d5a/d28/d6d/fc6 [0,4194304] 0 2024-04-26T04:53:08.601 INFO:tasks.workunit.client.1.smithi078.stdout:5/919: chown d7/d18/d32/d37/d79/f115 22431292 1 2024-04-26T04:53:08.604 INFO:tasks.workunit.client.1.smithi078.stdout:2/964: read d8/f3a [3517653,6307] 0 2024-04-26T04:53:08.605 INFO:tasks.workunit.client.1.smithi078.stdout:8/931: mkdir d0/d19/d52/d3c/d12e 0 2024-04-26T04:53:08.607 INFO:tasks.workunit.client.0.smithi077.stdout:2/516: read d9/d15/d56/f8c [1846102,100835] 0 2024-04-26T04:53:08.610 INFO:tasks.workunit.client.0.smithi077.stdout:3/424: dwrite d0/f3d [0,4194304] 0 2024-04-26T04:53:08.612 INFO:tasks.workunit.client.0.smithi077.stdout:1/468: creat d9/d53/d6e/d7c/f82 x:0 0 0 2024-04-26T04:53:08.612 INFO:tasks.workunit.client.0.smithi077.stdout:1/469: readlink d9/l56 0 2024-04-26T04:53:08.612 INFO:tasks.workunit.client.1.smithi078.stdout:9/876: mknod dc/df/d4b/d22/d2a/d57/d93/c109 0 2024-04-26T04:53:08.615 INFO:tasks.workunit.client.0.smithi077.stdout:8/602: mkdir d4/df/d6f/da3/dc7 0 2024-04-26T04:53:08.620 INFO:tasks.workunit.client.1.smithi078.stdout:5/920: creat d7/d48/d57/d65/d7a/f139 x:0 0 0 2024-04-26T04:53:08.622 INFO:tasks.workunit.client.1.smithi078.stdout:3/958: truncate d16/d3c/d81/fe0 580153 0 2024-04-26T04:53:08.626 INFO:tasks.workunit.client.1.smithi078.stdout:5/921: dwrite d7/d21/d138/f107 [0,4194304] 0 2024-04-26T04:53:08.627 INFO:tasks.workunit.client.0.smithi077.stdout:5/512: link d1/d30/l5c d1/d12/d87/lb9 0 2024-04-26T04:53:08.627 INFO:tasks.workunit.client.0.smithi077.stdout:5/513: dread - d1/d42/d62/f9e zero size 2024-04-26T04:53:08.629 INFO:tasks.workunit.client.1.smithi078.stdout:7/910: chown d9/da/d11/d13/d1a/d65/ca3 293785719 1 2024-04-26T04:53:08.629 INFO:tasks.workunit.client.0.smithi077.stdout:3/425: mkdir d0/d7/d1f/d81 0 2024-04-26T04:53:08.634 INFO:tasks.workunit.client.0.smithi077.stdout:0/471: dread d0/d2c/d4e/f14 [0,4194304] 0 2024-04-26T04:53:08.637 INFO:tasks.workunit.client.0.smithi077.stdout:6/578: link d8/f88 d8/d52/d53/fc7 0 2024-04-26T04:53:08.638 INFO:tasks.workunit.client.1.smithi078.stdout:4/954: rename d0/f13 to d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/f14f 0 2024-04-26T04:53:08.638 INFO:tasks.workunit.client.1.smithi078.stdout:5/922: dread - d7/d11/dee/d111/f120 zero size 2024-04-26T04:53:08.638 INFO:tasks.workunit.client.1.smithi078.stdout:8/932: dread d0/d19/d52/d35/d44/d6b/d6c/f86 [0,4194304] 0 2024-04-26T04:53:08.640 INFO:tasks.workunit.client.1.smithi078.stdout:2/965: dwrite d8/d11/d18/d5d/d8b/db0/d10d/d110/f124 [0,4194304] 0 2024-04-26T04:53:08.641 INFO:tasks.workunit.client.1.smithi078.stdout:4/955: stat d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118 0 2024-04-26T04:53:08.643 INFO:tasks.workunit.client.0.smithi077.stdout:8/603: creat d4/df/d22/fc8 x:0 0 0 2024-04-26T04:53:08.646 INFO:tasks.workunit.client.0.smithi077.stdout:2/517: dwrite d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:53:08.647 INFO:tasks.workunit.client.0.smithi077.stdout:0/472: dread d0/d19/f98 [0,4194304] 0 2024-04-26T04:53:08.647 INFO:tasks.workunit.client.0.smithi077.stdout:8/604: read - d4/d17/d90/faf zero size 2024-04-26T04:53:08.653 INFO:tasks.workunit.client.0.smithi077.stdout:0/473: write d0/d19/d69/fa6 [384952,17786] 0 2024-04-26T04:53:08.653 INFO:tasks.workunit.client.0.smithi077.stdout:1/470: dwrite d9/f42 [0,4194304] 0 2024-04-26T04:53:08.654 INFO:tasks.workunit.client.0.smithi077.stdout:5/514: symlink d1/lba 0 2024-04-26T04:53:08.656 INFO:tasks.workunit.client.0.smithi077.stdout:0/474: truncate d0/d19/d69/fa6 647937 0 2024-04-26T04:53:08.657 INFO:tasks.workunit.client.0.smithi077.stdout:0/475: chown d0/d2c/d4e/c1c 1 1 2024-04-26T04:53:08.664 INFO:tasks.workunit.client.0.smithi077.stdout:8/605: mknod d4/df/d22/d2d/d2b/d33/dbc/cc9 0 2024-04-26T04:53:08.670 INFO:tasks.workunit.client.0.smithi077.stdout:8/606: symlink d4/df/dc6/lca 0 2024-04-26T04:53:08.677 INFO:tasks.workunit.client.0.smithi077.stdout:8/607: dread d4/df/d18/d56/f72 [0,4194304] 0 2024-04-26T04:53:08.681 INFO:tasks.workunit.client.0.smithi077.stdout:5/515: dread d1/d42/da4/f34 [0,4194304] 0 2024-04-26T04:53:08.684 INFO:tasks.workunit.client.0.smithi077.stdout:5/516: chown d1/d12/d28/c2d 633 1 2024-04-26T04:53:08.702 INFO:tasks.workunit.client.1.smithi078.stdout:0/946: rename d2/f37 to d2/d9/d18/db7/d92/f141 0 2024-04-26T04:53:08.704 INFO:tasks.workunit.client.0.smithi077.stdout:6/579: dwrite d8/d1c/f25 [0,4194304] 0 2024-04-26T04:53:08.705 INFO:tasks.workunit.client.1.smithi078.stdout:8/933: write d0/d19/d25/fd6 [4395018,59167] 0 2024-04-26T04:53:08.707 INFO:tasks.workunit.client.0.smithi077.stdout:3/426: write d0/f38 [1555872,35552] 0 2024-04-26T04:53:08.712 INFO:tasks.workunit.client.1.smithi078.stdout:4/956: stat d0/d1d/de1/d20/d31/f3f 0 2024-04-26T04:53:08.714 INFO:tasks.workunit.client.1.smithi078.stdout:0/947: fdatasync d2/d9/d18/d7d/da9/dab/dc6/f9c 0 2024-04-26T04:53:08.714 INFO:tasks.workunit.client.1.smithi078.stdout:2/966: dread d8/d11/d18/d5d/d61/f62 [0,4194304] 0 2024-04-26T04:53:08.715 INFO:tasks.workunit.client.1.smithi078.stdout:8/934: mknod d0/d19/d52/d35/d73/c12f 0 2024-04-26T04:53:08.716 INFO:tasks.workunit.client.1.smithi078.stdout:9/877: sync 2024-04-26T04:53:08.718 INFO:tasks.workunit.client.0.smithi077.stdout:6/580: getdents d8 0 2024-04-26T04:53:08.719 INFO:tasks.workunit.client.0.smithi077.stdout:4/491: sync 2024-04-26T04:53:08.720 INFO:tasks.workunit.client.1.smithi078.stdout:8/935: dwrite d0/d19/d52/d5c/fb5 [0,4194304] 0 2024-04-26T04:53:08.724 INFO:tasks.workunit.client.0.smithi077.stdout:6/581: truncate d8/d2a/f2e 8058816 0 2024-04-26T04:53:08.725 INFO:tasks.workunit.client.1.smithi078.stdout:5/923: link d7/d11/fb5 d7/d18/d1c/f13a 0 2024-04-26T04:53:08.726 INFO:tasks.workunit.client.0.smithi077.stdout:1/471: dwrite d9/db/f1c [4194304,4194304] 0 2024-04-26T04:53:08.728 INFO:tasks.workunit.client.1.smithi078.stdout:2/967: rename d8/d11/d18/d137/d34/dd3/de4/d8e/d91/f128 to d8/db7/f14c 0 2024-04-26T04:53:08.729 INFO:tasks.workunit.client.0.smithi077.stdout:0/476: dwrite d0/d19/f32 [0,4194304] 0 2024-04-26T04:53:08.734 INFO:tasks.workunit.client.0.smithi077.stdout:6/582: unlink d8/d1c/db5/fbb 0 2024-04-26T04:53:08.741 INFO:tasks.workunit.client.0.smithi077.stdout:1/472: truncate d9/db/dd/f69 430995 0 2024-04-26T04:53:08.741 INFO:tasks.workunit.client.0.smithi077.stdout:8/608: dwrite d4/df/d18/d56/f72 [0,4194304] 0 2024-04-26T04:53:08.743 INFO:tasks.workunit.client.0.smithi077.stdout:0/477: creat d0/d47/d93/d94/faa x:0 0 0 2024-04-26T04:53:08.746 INFO:tasks.workunit.client.0.smithi077.stdout:6/583: mkdir d8/d55/dc8 0 2024-04-26T04:53:08.747 INFO:tasks.workunit.client.0.smithi077.stdout:4/492: link d4/d17/d18/d43/d53/d64/l70 d4/d17/d18/d43/d53/d75/la8 0 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 -- 172.21.15.77:0/314464103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 msgr2=0x7faa5c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 --2- 172.21.15.77:0/314464103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c0fd3d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa4c004940 tx=0x7faa4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 -- 172.21.15.77:0/314464103 shutdown_connections 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 --2- 172.21.15.77:0/314464103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c0fd3d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 --2- 172.21.15.77:0/314464103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 0x7faa5c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 -- 172.21.15.77:0/314464103 >> 172.21.15.77:0/314464103 conn(0x7faa5c0f7430 msgr2=0x7faa5c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 -- 172.21.15.77:0/314464103 shutdown_connections 2024-04-26T04:53:08.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.746+0000 7faa6387c700 1 -- 172.21.15.77:0/314464103 wait complete. 2024-04-26T04:53:08.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.747+0000 7faa6387c700 1 Processor -- start 2024-04-26T04:53:08.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.747+0000 7faa6387c700 1 -- start start 2024-04-26T04:53:08.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa6387c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 0x7faa5c10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa6387c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa6387c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c10e5c0 con 0x7faa5c0fcf60 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa6387c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c10edc0 con 0x7faa5c0fbcb0 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa60e17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa60e17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45862/0 (socket says 172.21.15.77:45862) 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa60e17700 1 -- 172.21.15.77:0/92809792 learned_addr learned my addr 172.21.15.77:0/92809792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:08.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.748+0000 7faa61618700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 0x7faa5c10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.749+0000 7faa60e17700 1 -- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 msgr2=0x7faa5c10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:08.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.749+0000 7faa60e17700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 0x7faa5c10da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:08.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.749+0000 7faa60e17700 1 -- 172.21.15.77:0/92809792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c005040 con 0x7faa5c0fcf60 2024-04-26T04:53:08.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.749+0000 7faa60e17700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009150 tx=0x7faa4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:08.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.750+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c00e040 con 0x7faa5c0fcf60 2024-04-26T04:53:08.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.750+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa4c004290 con 0x7faa5c0fcf60 2024-04-26T04:53:08.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.750+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c01c630 con 0x7faa5c0fcf60 2024-04-26T04:53:08.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.750+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c10f040 con 0x7faa5c0fcf60 2024-04-26T04:53:08.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.750+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c10f580 con 0x7faa5c0fcf60 2024-04-26T04:53:08.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.752+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c05f5c0 con 0x7faa5c0fcf60 2024-04-26T04:53:08.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.753+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7faa4c00bbf0 con 0x7faa5c0fcf60 2024-04-26T04:53:08.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.753+0000 7faa51ffb700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 0x7faa4806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:08.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.753+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faa4c08ae90 con 0x7faa5c0fcf60 2024-04-26T04:53:08.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.754+0000 7faa61618700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 0x7faa4806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:08.756 INFO:tasks.workunit.client.0.smithi077.stdout:0/478: creat d0/d19/d1e/d2f/da8/fab x:0 0 0 2024-04-26T04:53:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.755+0000 7faa61618700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 0x7faa4806dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa58009040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:08.759 INFO:tasks.workunit.client.0.smithi077.stdout:4/493: dwrite d4/d17/d18/d43/d53/d64/f81 [4194304,4194304] 0 2024-04-26T04:53:08.760 INFO:tasks.workunit.client.0.smithi077.stdout:6/584: dwrite d8/d2a/f34 [0,4194304] 0 2024-04-26T04:53:08.763 INFO:tasks.workunit.client.0.smithi077.stdout:6/585: read d8/fe [1831313,101506] 0 2024-04-26T04:53:08.765 INFO:tasks.workunit.client.0.smithi077.stdout:0/479: unlink d0/f4 0 2024-04-26T04:53:08.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:08.765+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7faa4c058710 con 0x7faa5c0fcf60 2024-04-26T04:53:08.768 INFO:tasks.workunit.client.0.smithi077.stdout:8/609: creat d4/d17/d35/fcb x:0 0 0 2024-04-26T04:53:08.770 INFO:tasks.workunit.client.0.smithi077.stdout:4/494: symlink d4/de/d15/d1f/la9 0 2024-04-26T04:53:08.771 INFO:tasks.workunit.client.1.smithi078.stdout:2/968: chown d8/d11/d18/d137/d34/dd3/de4/d52/fdb 248674 1 2024-04-26T04:53:08.772 INFO:tasks.workunit.client.0.smithi077.stdout:5/517: dwrite d1/d12/f2a [0,4194304] 0 2024-04-26T04:53:08.772 INFO:tasks.workunit.client.1.smithi078.stdout:2/969: dread d8/d11/d18/d137/f49 [0,4194304] 0 2024-04-26T04:53:08.773 INFO:tasks.workunit.client.1.smithi078.stdout:2/970: chown d8/d11/f48 6892899 1 2024-04-26T04:53:08.779 INFO:tasks.workunit.client.0.smithi077.stdout:4/495: chown d4/de/d15/d1f/d25/ca2 26 1 2024-04-26T04:53:08.779 INFO:tasks.workunit.client.1.smithi078.stdout:4/957: getdents d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa 0 2024-04-26T04:53:08.780 INFO:tasks.workunit.client.0.smithi077.stdout:8/610: dwrite d4/df/d22/f50 [0,4194304] 0 2024-04-26T04:53:08.781 INFO:tasks.workunit.client.0.smithi077.stdout:4/496: dread - d4/d17/d18/d43/d6e/f95 zero size 2024-04-26T04:53:08.781 INFO:tasks.workunit.client.0.smithi077.stdout:5/518: creat d1/d8f/fbb x:0 0 0 2024-04-26T04:53:08.785 INFO:tasks.workunit.client.1.smithi078.stdout:2/971: symlink d8/d11/d18/d5d/d8b/db0/d10d/l14d 0 2024-04-26T04:53:08.786 INFO:tasks.workunit.client.1.smithi078.stdout:2/972: truncate d8/f84 1836038 0 2024-04-26T04:53:08.786 INFO:tasks.workunit.client.0.smithi077.stdout:8/611: dread d4/df/d18/d56/f7c [4194304,4194304] 0 2024-04-26T04:53:08.787 INFO:tasks.workunit.client.1.smithi078.stdout:3/959: sync 2024-04-26T04:53:08.787 INFO:tasks.workunit.client.0.smithi077.stdout:3/427: write d0/d7/d1f/f4a [715877,28405] 0 2024-04-26T04:53:08.788 INFO:tasks.workunit.client.0.smithi077.stdout:8/612: fdatasync d4/df/d22/d2d/fc1 0 2024-04-26T04:53:08.788 INFO:tasks.workunit.client.0.smithi077.stdout:3/428: chown d0/d9/d32/d29/f6a 1 1 2024-04-26T04:53:08.790 INFO:tasks.workunit.client.0.smithi077.stdout:8/613: write d4/d17/d35/d5b/f8b [1646704,24068] 0 2024-04-26T04:53:08.793 INFO:tasks.workunit.client.1.smithi078.stdout:0/948: write d2/d9/f20 [4824521,32843] 0 2024-04-26T04:53:08.799 INFO:tasks.workunit.client.1.smithi078.stdout:4/958: symlink d0/l150 0 2024-04-26T04:53:08.800 INFO:tasks.workunit.client.1.smithi078.stdout:4/959: write d0/de/d22/d27/d7b/f82 [805311,81538] 0 2024-04-26T04:53:08.801 INFO:tasks.workunit.client.1.smithi078.stdout:5/924: write d7/d18/d32/d37/d8f/dc5/fcc [555775,51543] 0 2024-04-26T04:53:08.803 INFO:tasks.workunit.client.0.smithi077.stdout:3/429: dread d0/f19 [0,4194304] 0 2024-04-26T04:53:08.805 INFO:tasks.workunit.client.0.smithi077.stdout:1/473: write d9/db/dd/f11 [656414,111399] 0 2024-04-26T04:53:08.806 INFO:tasks.workunit.client.1.smithi078.stdout:5/925: dwrite d7/d21/d2f/db6/d125/f133 [0,4194304] 0 2024-04-26T04:53:08.810 INFO:tasks.workunit.client.0.smithi077.stdout:3/430: creat d0/d7/d1f/d4d/f82 x:0 0 0 2024-04-26T04:53:08.812 INFO:tasks.workunit.client.0.smithi077.stdout:1/474: creat d9/d53/d7d/f83 x:0 0 0 2024-04-26T04:53:08.814 INFO:tasks.workunit.client.1.smithi078.stdout:5/926: dwrite d7/d18/d32/f60 [4194304,4194304] 0 2024-04-26T04:53:08.816 INFO:tasks.workunit.client.0.smithi077.stdout:3/431: symlink d0/d7/l83 0 2024-04-26T04:53:08.820 INFO:tasks.workunit.client.1.smithi078.stdout:8/936: dwrite d0/d19/d52/f24 [0,4194304] 0 2024-04-26T04:53:08.822 INFO:tasks.workunit.client.0.smithi077.stdout:5/519: write d1/d8/f49 [215448,88644] 0 2024-04-26T04:53:08.823 INFO:tasks.workunit.client.0.smithi077.stdout:3/432: creat d0/d9/f84 x:0 0 0 2024-04-26T04:53:08.826 INFO:tasks.workunit.client.0.smithi077.stdout:4/497: dwrite d4/de/d15/d1f/d76/f82 [0,4194304] 0 2024-04-26T04:53:08.827 INFO:tasks.workunit.client.1.smithi078.stdout:2/973: dwrite d8/d11/d18/dbb/f101 [0,4194304] 0 2024-04-26T04:53:08.829 INFO:tasks.workunit.client.0.smithi077.stdout:5/520: mknod d1/d12/d87/cbc 0 2024-04-26T04:53:08.829 INFO:tasks.workunit.client.1.smithi078.stdout:2/974: dread - d8/d11/d18/f11f zero size 2024-04-26T04:53:08.829 INFO:tasks.workunit.client.1.smithi078.stdout:7/911: sync 2024-04-26T04:53:08.830 INFO:tasks.workunit.client.0.smithi077.stdout:1/475: read d9/db/dd/d34/d4a/f59 [192593,37473] 0 2024-04-26T04:53:08.833 INFO:tasks.workunit.client.1.smithi078.stdout:0/949: write d2/d9/d18/db7/d92/f141 [1350299,83654] 0 2024-04-26T04:53:08.833 INFO:tasks.workunit.client.0.smithi077.stdout:3/433: mkdir d0/d9/d32/d2a/d85 0 2024-04-26T04:53:08.833 INFO:tasks.workunit.client.1.smithi078.stdout:3/960: dwrite d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/f52 [0,4194304] 0 2024-04-26T04:53:08.839 INFO:tasks.workunit.client.1.smithi078.stdout:3/961: write d16/d26/d2e/d5f/d49/f4e [4957082,27365] 0 2024-04-26T04:53:08.840 INFO:tasks.workunit.client.1.smithi078.stdout:8/937: truncate d0/d19/d52/d35/d44/d6b/d104/d58/d5b/d61/fd9 1665725 0 2024-04-26T04:53:08.840 INFO:tasks.workunit.client.0.smithi077.stdout:5/521: symlink d1/d42/da4/d25/lbd 0 2024-04-26T04:53:08.842 INFO:tasks.workunit.client.1.smithi078.stdout:8/938: readlink d0/d19/d52/d35/d44/d6b/d104/lbc 0 2024-04-26T04:53:08.842 INFO:tasks.workunit.client.1.smithi078.stdout:4/960: dwrite d0/de/fad [0,4194304] 0 2024-04-26T04:53:08.843 INFO:tasks.workunit.client.1.smithi078.stdout:3/962: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d13a/f143 [10652,111031] 0 2024-04-26T04:53:08.844 INFO:tasks.workunit.client.0.smithi077.stdout:3/434: dwrite d0/d7/d15/f3f [0,4194304] 0 2024-04-26T04:53:08.846 INFO:tasks.workunit.client.0.smithi077.stdout:3/435: readlink d0/d9/d32/l1e 0 2024-04-26T04:53:08.847 INFO:tasks.workunit.client.1.smithi078.stdout:7/912: mkdir d9/da/d33/d4b/d4c/d5b/dc3/d128 0 2024-04-26T04:53:08.848 INFO:tasks.workunit.client.0.smithi077.stdout:4/498: write d4/de/d15/d1f/f34 [462999,103951] 0 2024-04-26T04:53:08.852 INFO:tasks.workunit.client.1.smithi078.stdout:5/927: dwrite d7/d18/d32/f4a [4194304,4194304] 0 2024-04-26T04:53:08.854 INFO:tasks.workunit.client.1.smithi078.stdout:0/950: unlink d2/d9/lba 0 2024-04-26T04:53:08.857 INFO:tasks.workunit.client.0.smithi077.stdout:5/522: symlink d1/d8/lbe 0 2024-04-26T04:53:08.857 INFO:tasks.workunit.client.0.smithi077.stdout:1/476: link d9/f73 d9/d53/d6e/d7c/f84 0 2024-04-26T04:53:08.858 INFO:tasks.workunit.client.1.smithi078.stdout:7/913: dwrite d9/da/d11/fe7 [0,4194304] 0 2024-04-26T04:53:08.862 INFO:tasks.workunit.client.1.smithi078.stdout:4/961: mkdir d0/d1d/de1/d20/d31/d47/db2/d4c/d151 0 2024-04-26T04:53:08.863 INFO:tasks.workunit.client.1.smithi078.stdout:3/963: symlink d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/l14c 0 2024-04-26T04:53:08.869 INFO:tasks.workunit.client.1.smithi078.stdout:7/914: dwrite d9/da/d20/f106 [0,4194304] 0 2024-04-26T04:53:08.870 INFO:tasks.workunit.client.0.smithi077.stdout:3/436: symlink d0/d7/d1f/l86 0 2024-04-26T04:53:08.870 INFO:tasks.workunit.client.1.smithi078.stdout:7/915: chown d9/c7f 54 1 2024-04-26T04:53:08.870 INFO:tasks.workunit.client.1.smithi078.stdout:2/975: write d8/d53/f71 [1338702,96995] 0 2024-04-26T04:53:08.872 INFO:tasks.workunit.client.1.smithi078.stdout:5/928: rename d7/d48/d57/d65/d46/de7/d85 to d7/d48/d57/d65/d46/de7/d13b 0 2024-04-26T04:53:08.873 INFO:tasks.workunit.client.1.smithi078.stdout:5/929: chown d7/d21/d2f/d90/c9b 76 1 2024-04-26T04:53:08.874 INFO:tasks.workunit.client.0.smithi077.stdout:4/499: getdents d4/d17/d18/d43/d53/d56 0 2024-04-26T04:53:08.875 INFO:tasks.workunit.client.1.smithi078.stdout:9/878: sync 2024-04-26T04:53:08.876 INFO:tasks.workunit.client.0.smithi077.stdout:1/477: creat d9/db/dd/d68/d81/f85 x:0 0 0 2024-04-26T04:53:08.880 INFO:tasks.workunit.client.0.smithi077.stdout:3/437: dwrite d0/d7/d1f/f72 [0,4194304] 0 2024-04-26T04:53:08.884 INFO:tasks.workunit.client.1.smithi078.stdout:7/916: mknod d9/da/d11/d13/d1a/d52/d5a/d28/dab/c129 0 2024-04-26T04:53:08.884 INFO:tasks.workunit.client.0.smithi077.stdout:4/500: symlink d4/laa 0 2024-04-26T04:53:08.885 INFO:tasks.workunit.client.1.smithi078.stdout:7/917: fdatasync d9/da/d11/d13/d50/f11d 0 2024-04-26T04:53:08.886 INFO:tasks.workunit.client.1.smithi078.stdout:2/976: mknod d8/d11/d18/d5d/d127/c14e 0 2024-04-26T04:53:08.888 INFO:tasks.workunit.client.1.smithi078.stdout:7/918: write d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/f119 [496895,51011] 0 2024-04-26T04:53:08.889 INFO:tasks.workunit.client.1.smithi078.stdout:5/930: mkdir d7/d48/da9/d10f/d13c 0 2024-04-26T04:53:08.889 INFO:tasks.workunit.client.0.smithi077.stdout:3/438: dwrite d0/f3d [0,4194304] 0 2024-04-26T04:53:08.891 INFO:tasks.workunit.client.0.smithi077.stdout:4/501: read f2 [4370005,52916] 0 2024-04-26T04:53:08.892 INFO:tasks.workunit.client.1.smithi078.stdout:9/879: rename dc/df/d27/d81/lc2 to dc/df/d27/d81/l10a 0 2024-04-26T04:53:08.893 INFO:tasks.workunit.client.1.smithi078.stdout:9/880: fsync dc/df/d4b/d22/d2a/ffa 0 2024-04-26T04:53:08.894 INFO:tasks.workunit.client.1.smithi078.stdout:0/951: rename d2/d39/c44 to d2/c142 0 2024-04-26T04:53:08.895 INFO:tasks.workunit.client.0.smithi077.stdout:1/478: link d9/db/dd/f23 d9/d53/f86 0 2024-04-26T04:53:08.896 INFO:tasks.workunit.client.1.smithi078.stdout:0/952: read - d2/d9/d31/df2/f11e zero size 2024-04-26T04:53:08.899 INFO:tasks.workunit.client.1.smithi078.stdout:7/919: creat d9/da/d33/d4b/d4c/d6c/f12a x:0 0 0 2024-04-26T04:53:08.900 INFO:tasks.workunit.client.1.smithi078.stdout:4/962: dread d0/d1d/f144 [0,4194304] 0 2024-04-26T04:53:08.900 INFO:tasks.workunit.client.0.smithi077.stdout:4/502: truncate d4/f27 743183 0 2024-04-26T04:53:08.901 INFO:tasks.workunit.client.1.smithi078.stdout:7/920: write d9/da/d33/d4b/d4c/fe5 [3607836,75677] 0 2024-04-26T04:53:08.902 INFO:tasks.workunit.client.1.smithi078.stdout:0/953: write d2/d9/d18/d7d/da9/dab/dc6/fb2 [8175386,93798] 0 2024-04-26T04:53:08.902 INFO:tasks.workunit.client.0.smithi077.stdout:3/439: creat d0/f87 x:0 0 0 2024-04-26T04:53:08.903 INFO:tasks.workunit.client.1.smithi078.stdout:9/881: truncate dc/df/d4b/d22/d2a/d57/d69/f96 3784063 0 2024-04-26T04:53:08.904 INFO:tasks.workunit.client.0.smithi077.stdout:1/479: link d9/f73 d9/d53/d6e/f87 0 2024-04-26T04:53:08.906 INFO:tasks.workunit.client.1.smithi078.stdout:5/931: truncate d7/da/d1e/d91/dd6/fe4 782771 0 2024-04-26T04:53:08.906 INFO:tasks.workunit.client.0.smithi077.stdout:4/503: fdatasync d4/d17/f3d 0 2024-04-26T04:53:08.908 INFO:tasks.workunit.client.0.smithi077.stdout:3/440: chown d0/d7/f70 14505 1 2024-04-26T04:53:08.909 INFO:tasks.workunit.client.0.smithi077.stdout:1/480: fdatasync d9/db/dd/d34/f51 0 2024-04-26T04:53:08.911 INFO:tasks.workunit.client.1.smithi078.stdout:4/963: symlink d0/de/d22/d27/db5/l152 0 2024-04-26T04:53:08.913 INFO:tasks.workunit.client.1.smithi078.stdout:0/954: creat d2/d9/d31/d73/f143 x:0 0 0 2024-04-26T04:53:08.914 INFO:tasks.workunit.client.0.smithi077.stdout:4/504: truncate d4/de/d15/d1f/f32 7917057 0 2024-04-26T04:53:08.915 INFO:tasks.workunit.client.1.smithi078.stdout:7/921: truncate d9/da/d33/d4b/d4c/d6c/fb6 781431 0 2024-04-26T04:53:08.916 INFO:tasks.workunit.client.0.smithi077.stdout:3/441: mknod d0/d7/d15/c88 0 2024-04-26T04:53:08.918 INFO:tasks.workunit.client.0.smithi077.stdout:3/442: truncate d0/d7/d15/f22 4350321 0 2024-04-26T04:53:08.918 INFO:tasks.workunit.client.1.smithi078.stdout:9/882: truncate dc/df/d4b/d3e/d80/f95 4493911 0 2024-04-26T04:53:08.918 INFO:tasks.workunit.client.1.smithi078.stdout:9/883: write dc/f54 [1424736,22673] 0 2024-04-26T04:53:08.919 INFO:tasks.workunit.client.1.smithi078.stdout:9/884: write dc/df/d27/f9b [1540808,58948] 0 2024-04-26T04:53:08.921 INFO:tasks.workunit.client.1.smithi078.stdout:5/932: mknod d7/d48/d57/d65/d46/c13d 0 2024-04-26T04:53:08.925 INFO:tasks.workunit.client.1.smithi078.stdout:4/964: mknod d0/d1d/de1/d20/d31/d47/db2/d4c/c153 0 2024-04-26T04:53:08.926 INFO:tasks.workunit.client.0.smithi077.stdout:4/505: symlink d4/d4e/lab 0 2024-04-26T04:53:08.927 INFO:tasks.workunit.client.0.smithi077.stdout:4/506: write d4/de/f2b [701960,116914] 0 2024-04-26T04:53:08.928 INFO:tasks.workunit.client.0.smithi077.stdout:4/507: chown d4/de/fa6 191 1 2024-04-26T04:53:08.929 INFO:tasks.workunit.client.1.smithi078.stdout:8/939: write d0/d19/d52/d35/d44/d6b/d104/f37 [124766,3530] 0 2024-04-26T04:53:08.930 INFO:tasks.workunit.client.1.smithi078.stdout:7/922: symlink d9/da/d33/l12b 0 2024-04-26T04:53:08.932 INFO:tasks.workunit.client.1.smithi078.stdout:7/923: chown d9/da/d11/d13/d1a/d52/d5a/l39 0 1 2024-04-26T04:53:08.936 INFO:tasks.workunit.client.0.smithi077.stdout:3/443: creat d0/d7/d1f/d81/f89 x:0 0 0 2024-04-26T04:53:08.937 INFO:tasks.workunit.client.0.smithi077.stdout:5/523: dwrite d1/d12/f19 [0,4194304] 0 2024-04-26T04:53:08.940 INFO:tasks.workunit.client.1.smithi078.stdout:5/933: mknod d7/d48/d57/d65/d46/c13e 0 2024-04-26T04:53:08.940 INFO:tasks.workunit.client.1.smithi078.stdout:3/964: write d16/d26/d2e/d74/f83 [106922,114083] 0 2024-04-26T04:53:08.941 INFO:tasks.workunit.client.1.smithi078.stdout:5/934: stat d7/d11/dee/d111/f127 0 2024-04-26T04:53:08.942 INFO:tasks.workunit.client.1.smithi078.stdout:8/940: dread d0/d19/d25/d32/d8a/d92/fcf [0,4194304] 0 2024-04-26T04:53:08.942 INFO:tasks.workunit.client.1.smithi078.stdout:4/965: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fe6 zero size 2024-04-26T04:53:08.944 INFO:tasks.workunit.client.1.smithi078.stdout:8/941: write d0/d19/d52/d35/d44/d6b/d104/d58/d5b/dee/f10a [625292,127850] 0 2024-04-26T04:53:08.947 INFO:tasks.workunit.client.1.smithi078.stdout:7/924: creat d9/d57/f12c x:0 0 0 2024-04-26T04:53:08.971 INFO:tasks.workunit.client.0.smithi077.stdout:9/495: sync 2024-04-26T04:53:08.972 INFO:tasks.workunit.client.0.smithi077.stdout:7/577: sync 2024-04-26T04:53:08.972 INFO:tasks.workunit.client.0.smithi077.stdout:7/578: read - da/d15/f94 zero size 2024-04-26T04:53:08.974 INFO:tasks.workunit.client.0.smithi077.stdout:9/496: dread d9/dd/d1b/d2b/f72 [0,4194304] 0 2024-04-26T04:53:08.975 INFO:tasks.workunit.client.0.smithi077.stdout:9/497: truncate d9/dd/d1b/d2b/f38 4401672 0 2024-04-26T04:53:08.977 INFO:tasks.workunit.client.1.smithi078.stdout:5/935: unlink d7/cf5 0 2024-04-26T04:53:08.980 INFO:tasks.workunit.client.1.smithi078.stdout:2/977: dwrite d8/d11/d18/d5d/d61/ff7 [0,4194304] 0 2024-04-26T04:53:08.982 INFO:tasks.workunit.client.0.smithi077.stdout:5/524: symlink d1/d42/da4/d25/d31/lbf 0 2024-04-26T04:53:08.982 INFO:tasks.workunit.client.1.smithi078.stdout:3/965: symlink d16/d26/d2e/d5f/d49/dad/df0/l14d 0 2024-04-26T04:53:08.982 INFO:tasks.workunit.client.1.smithi078.stdout:2/978: dread - d8/d11/d18/d137/d34/dd6/f146 zero size 2024-04-26T04:53:08.985 INFO:tasks.workunit.client.1.smithi078.stdout:4/966: read d0/d1d/de1/d20/d31/d47/db2/f5b [88929,18197] 0 2024-04-26T04:53:08.987 INFO:tasks.workunit.client.0.smithi077.stdout:3/444: creat d0/d9/d32/f8a x:0 0 0 2024-04-26T04:53:08.991 INFO:tasks.workunit.client.0.smithi077.stdout:4/508: getdents d4/d17/d18/d43/d6e 0 2024-04-26T04:53:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:08 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2620106838' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:53:08.993 INFO:tasks.workunit.client.0.smithi077.stdout:9/498: fdatasync d9/f1c 0 2024-04-26T04:53:09.001 INFO:tasks.workunit.client.1.smithi078.stdout:3/966: rename d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d13a to d16/d26/d2e/d5f/d49/dcc/d14e 0 2024-04-26T04:53:09.003 INFO:tasks.workunit.client.1.smithi078.stdout:4/967: unlink d0/d1d/de1/d20/d31/d47/db2/d4c/d62/dd7/d118/fa4 0 2024-04-26T04:53:09.004 INFO:tasks.workunit.client.1.smithi078.stdout:0/955: write d2/f70 [2744492,59841] 0 2024-04-26T04:53:09.009 INFO:tasks.workunit.client.0.smithi077.stdout:1/481: write d9/db/dd/d34/f38 [209315,103613] 0 2024-04-26T04:53:09.014 INFO:tasks.workunit.client.1.smithi078.stdout:9/885: truncate dc/df/d4b/d22/d2d/f108 7997450 0 2024-04-26T04:53:09.018 INFO:tasks.workunit.client.0.smithi077.stdout:5/525: symlink d1/d12/d87/lc0 0 2024-04-26T04:53:09.026 INFO:tasks.workunit.client.0.smithi077.stdout:4/509: fdatasync d4/d17/f3d 0 2024-04-26T04:53:09.031 INFO:tasks.workunit.client.1.smithi078.stdout:7/925: creat d9/da/d11/d13/d1a/d52/d5a/d41/f12d x:0 0 0 2024-04-26T04:53:09.033 INFO:tasks.workunit.client.0.smithi077.stdout:9/499: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae 0 2024-04-26T04:53:09.033 INFO:tasks.workunit.client.0.smithi077.stdout:9/500: dread - d9/d3a/d79/f68 zero size 2024-04-26T04:53:09.034 INFO:tasks.workunit.client.0.smithi077.stdout:9/501: stat d9/dd/d1b/c5f 0 2024-04-26T04:53:09.034 INFO:tasks.workunit.client.0.smithi077.stdout:9/502: chown d9/dd/d1b/d2b/f83 41927 1 2024-04-26T04:53:09.036 INFO:tasks.workunit.client.1.smithi078.stdout:8/942: truncate d0/d19/d52/d35/d44/d6b/d104/f7f 306735 0 2024-04-26T04:53:09.039 INFO:tasks.workunit.client.1.smithi078.stdout:5/936: truncate d7/d18/d1c/d8e/f103 244378 0 2024-04-26T04:53:09.039 INFO:tasks.workunit.client.0.smithi077.stdout:7/579: link da/d13/c9d da/d15/d1a/d39/cc5 0 2024-04-26T04:53:09.043 INFO:tasks.workunit.client.1.smithi078.stdout:7/926: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/d12e 0 2024-04-26T04:53:09.045 INFO:tasks.workunit.client.1.smithi078.stdout:2/979: truncate d8/d11/d18/d137/fe3 336899 0 2024-04-26T04:53:09.045 INFO:tasks.workunit.client.1.smithi078.stdout:2/980: stat d8/d11/d18/d137/d34/dd3/de4/d8e/d91 0 2024-04-26T04:53:09.046 INFO:tasks.workunit.client.0.smithi077.stdout:4/510: mknod d4/d17/d7f/cac 0 2024-04-26T04:53:09.049 INFO:tasks.workunit.client.1.smithi078.stdout:2/981: dread d8/d11/d18/d137/d34/dd3/de4/d8e/f112 [0,4194304] 0 2024-04-26T04:53:09.049 INFO:tasks.workunit.client.1.smithi078.stdout:8/943: symlink d0/d7a/d83/l130 0 2024-04-26T04:53:09.049 INFO:tasks.workunit.client.1.smithi078.stdout:4/968: write d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/ff2 [576249,70154] 0 2024-04-26T04:53:09.050 INFO:tasks.workunit.client.0.smithi077.stdout:9/503: creat d9/dd/d3d/faf x:0 0 0 2024-04-26T04:53:09.051 INFO:tasks.workunit.client.1.smithi078.stdout:4/969: truncate d0/d1d/f13d 565651 0 2024-04-26T04:53:09.051 INFO:tasks.workunit.client.0.smithi077.stdout:9/504: read d9/f22 [8895159,5065] 0 2024-04-26T04:53:09.053 INFO:tasks.workunit.client.1.smithi078.stdout:0/956: rmdir d2/d9/d18/d7d/da9/dab/dfd/d110 0 2024-04-26T04:53:09.054 INFO:tasks.workunit.client.0.smithi077.stdout:3/445: creat d0/d9/f8b x:0 0 0 2024-04-26T04:53:09.055 INFO:tasks.workunit.client.1.smithi078.stdout:5/937: dread d7/d18/d32/d37/f51 [0,4194304] 0 2024-04-26T04:53:09.056 INFO:tasks.workunit.client.1.smithi078.stdout:7/927: mkdir d9/da/d33/d4b/d4c/d6c/d12f 0 2024-04-26T04:53:09.060 INFO:tasks.workunit.client.1.smithi078.stdout:0/957: dwrite d2/d9/d18/db7/d92/fad [0,4194304] 0 2024-04-26T04:53:09.061 INFO:tasks.workunit.client.0.smithi077.stdout:3/446: dwrite d0/d7/d15/f3f [0,4194304] 0 2024-04-26T04:53:09.062 INFO:tasks.workunit.client.1.smithi078.stdout:0/958: write d2/d9/d60/dd5/f108 [3482370,119546] 0 2024-04-26T04:53:09.062 INFO:tasks.workunit.client.0.smithi077.stdout:3/447: chown d0/d9/d32 572 1 2024-04-26T04:53:09.163 INFO:tasks.workunit.client.0.smithi077.stdout:2/518: sync 2024-04-26T04:53:09.191 INFO:tasks.workunit.client.1.smithi078.stdout:3/967: getdents d16/d26/d2e/d5f/d49/dad/df0 0 2024-04-26T04:53:09.192 INFO:tasks.workunit.client.0.smithi077.stdout:0/480: sync 2024-04-26T04:53:09.192 INFO:tasks.workunit.client.0.smithi077.stdout:6/586: sync 2024-04-26T04:53:09.192 INFO:tasks.workunit.client.0.smithi077.stdout:8/614: sync 2024-04-26T04:53:09.193 INFO:tasks.workunit.client.0.smithi077.stdout:8/615: chown d4/d17/f6a 62 1 2024-04-26T04:53:09.202 INFO:tasks.workunit.client.0.smithi077.stdout:3/448: creat d0/d7/d1f/d76/f8c x:0 0 0 2024-04-26T04:53:09.207 INFO:tasks.workunit.client.0.smithi077.stdout:4/511: unlink d4/d17/d18/l72 0 2024-04-26T04:53:09.209 INFO:tasks.workunit.client.1.smithi078.stdout:9/886: getdents dc/df/d4b/d22/d2d/d98 0 2024-04-26T04:53:09.214 INFO:tasks.workunit.client.0.smithi077.stdout:8/616: symlink d4/df/d22/d80/lcc 0 2024-04-26T04:53:09.215 INFO:tasks.workunit.client.1.smithi078.stdout:0/959: truncate d2/f12 2111717 0 2024-04-26T04:53:09.220 INFO:tasks.workunit.client.0.smithi077.stdout:3/449: creat d0/d7/d1f/d76/f8d x:0 0 0 2024-04-26T04:53:09.223 INFO:tasks.workunit.client.1.smithi078.stdout:3/968: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130/d123/d14f 0 2024-04-26T04:53:09.224 INFO:tasks.workunit.client.1.smithi078.stdout:3/969: readlink d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/l47 0 2024-04-26T04:53:09.225 INFO:tasks.workunit.client.0.smithi077.stdout:8/617: write d4/df/d22/d2d/d2b/f5d [2397463,92932] 0 2024-04-26T04:53:09.228 INFO:tasks.workunit.client.0.smithi077.stdout:7/580: getdents da/d15/d42 0 2024-04-26T04:53:09.228 INFO:tasks.workunit.client.0.smithi077.stdout:7/581: chown da/d13/c38 562938254 1 2024-04-26T04:53:09.231 INFO:tasks.workunit.client.0.smithi077.stdout:3/450: chown d0/c58 65996 1 2024-04-26T04:53:09.236 INFO:tasks.workunit.client.1.smithi078.stdout:0/960: truncate d2/d9/d60/f68 1181524 0 2024-04-26T04:53:09.237 INFO:tasks.workunit.client.0.smithi077.stdout:7/582: mkdir da/d15/dc6 0 2024-04-26T04:53:09.238 INFO:tasks.workunit.client.0.smithi077.stdout:7/583: write da/d15/d1a/fb5 [476999,110407] 0 2024-04-26T04:53:09.238 INFO:tasks.workunit.client.1.smithi078.stdout:3/970: mkdir d16/d26/d2e/d74/d150 0 2024-04-26T04:53:09.243 INFO:tasks.workunit.client.1.smithi078.stdout:3/971: dwrite d16/d3c/f11b [0,4194304] 0 2024-04-26T04:53:09.244 INFO:tasks.workunit.client.0.smithi077.stdout:8/618: mknod d4/df/d6f/da3/dc7/ccd 0 2024-04-26T04:53:09.247 INFO:tasks.workunit.client.0.smithi077.stdout:7/584: symlink da/d13/d22/d5d/d8d/lc7 0 2024-04-26T04:53:09.249 INFO:tasks.workunit.client.1.smithi078.stdout:9/887: symlink dc/df/d27/d81/da8/l10b 0 2024-04-26T04:53:09.250 INFO:tasks.workunit.client.1.smithi078.stdout:9/888: write dc/df/d27/fb1 [830661,43986] 0 2024-04-26T04:53:09.251 INFO:tasks.workunit.client.1.smithi078.stdout:0/961: dread d2/d9/d18/f4e [0,4194304] 0 2024-04-26T04:53:09.252 INFO:tasks.workunit.client.1.smithi078.stdout:0/962: write d2/d9/d18/db7/d92/fd3 [803661,93098] 0 2024-04-26T04:53:09.253 INFO:tasks.workunit.client.1.smithi078.stdout:0/963: chown d2/d9/d31/f4f 13193657 1 2024-04-26T04:53:09.254 INFO:tasks.workunit.client.1.smithi078.stdout:0/964: chown d2/d9/d31/l63 421 1 2024-04-26T04:53:09.255 INFO:tasks.workunit.client.0.smithi077.stdout:7/585: unlink da/d13/d22/f58 0 2024-04-26T04:53:09.261 INFO:tasks.workunit.client.1.smithi078.stdout:9/889: read - dc/df/d4b/d3e/fd3 zero size 2024-04-26T04:53:09.261 INFO:tasks.workunit.client.1.smithi078.stdout:3/972: dwrite d16/f69 [0,4194304] 0 2024-04-26T04:53:09.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.261+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa5c1047a0 con 0x7faa5c0fcf60 2024-04-26T04:53:09.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.261+0000 7faa51ffb700 1 -- 172.21.15.77:0/92809792 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7faa4c0582a0 con 0x7faa5c0fcf60 2024-04-26T04:53:09.263 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:53:09.263 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:53:09.263 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 msgr2=0x7faa4806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 0x7faa4806dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa58007fd0 tx=0x7faa58009040 comp rx=0 tx=0).stop 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 msgr2=0x7faa5c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009150 tx=0x7faa4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 shutdown_connections 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7faa4806b860 0x7faa4806dd10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0fbcb0 0x7faa5c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 --2- 172.21.15.77:0/92809792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa5c0fcf60 0x7faa5c10dfc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:09.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 >> 172.21.15.77:0/92809792 conn(0x7faa5c0f7430 msgr2=0x7faa5c065c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:09.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 shutdown_connections 2024-04-26T04:53:09.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:09.264+0000 7faa6387c700 1 -- 172.21.15.77:0/92809792 wait complete. 2024-04-26T04:53:09.267 INFO:tasks.workunit.client.1.smithi078.stdout:9/890: fdatasync dc/df/d8a/fb9 0 2024-04-26T04:53:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:08 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2620106838' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:53:09.273 INFO:tasks.workunit.client.1.smithi078.stdout:3/973: unlink d16/f8d 0 2024-04-26T04:53:09.274 INFO:tasks.workunit.client.1.smithi078.stdout:0/965: creat d2/d9/d60/d6a/f144 x:0 0 0 2024-04-26T04:53:09.276 INFO:tasks.workunit.client.1.smithi078.stdout:3/974: truncate d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/f107 1579877 0 2024-04-26T04:53:09.276 INFO:tasks.workunit.client.1.smithi078.stdout:9/891: symlink dc/dd9/de1/l10c 0 2024-04-26T04:53:09.279 INFO:tasks.workunit.client.1.smithi078.stdout:0/966: dwrite d2/d9/d31/f4f [0,4194304] 0 2024-04-26T04:53:09.282 INFO:tasks.workunit.client.1.smithi078.stdout:0/967: creat d2/d9/d18/db7/d92/f145 x:0 0 0 2024-04-26T04:53:09.283 INFO:tasks.workunit.client.1.smithi078.stdout:2/982: rmdir d8/d11/d18/d137/d34/d70/def 39 2024-04-26T04:53:09.284 INFO:tasks.workunit.client.1.smithi078.stdout:2/983: write d8/d11/f48 [516911,44776] 0 2024-04-26T04:53:09.291 INFO:tasks.workunit.client.1.smithi078.stdout:2/984: link d8/d11/d18/d137/d34/la7 d8/d11/d18/d137/d34/d70/def/l14f 0 2024-04-26T04:53:09.294 INFO:tasks.workunit.client.1.smithi078.stdout:2/985: creat d8/d11/d18/dfe/d14a/f150 x:0 0 0 2024-04-26T04:53:09.300 INFO:tasks.workunit.client.0.smithi077.stdout:6/587: symlink d8/d2a/lc9 0 2024-04-26T04:53:09.310 INFO:tasks.workunit.client.1.smithi078.stdout:9/892: rmdir dc/df/d4b/d22/d2a/d57/d69/db5 39 2024-04-26T04:53:09.313 INFO:tasks.workunit.client.1.smithi078.stdout:9/893: mknod dc/df/d8a/c10d 0 2024-04-26T04:53:09.314 INFO:tasks.workunit.client.0.smithi077.stdout:6/588: dread d8/d52/d53/f80 [0,4194304] 0 2024-04-26T04:53:09.318 INFO:tasks.workunit.client.0.smithi077.stdout:6/589: creat d8/d52/d53/d5e/fca x:0 0 0 2024-04-26T04:53:09.319 INFO:tasks.workunit.client.1.smithi078.stdout:9/894: dwrite dc/fd7 [0,4194304] 0 2024-04-26T04:53:09.328 INFO:tasks.workunit.client.1.smithi078.stdout:8/944: write d0/d19/d52/d35/d44/f93 [4997540,9667] 0 2024-04-26T04:53:09.328 INFO:tasks.workunit.client.0.smithi077.stdout:9/505: write d9/f11 [4272036,114232] 0 2024-04-26T04:53:09.329 INFO:tasks.workunit.client.0.smithi077.stdout:9/506: fsync d9/f2d 0 2024-04-26T04:53:09.330 INFO:tasks.workunit.client.1.smithi078.stdout:8/945: dread - d0/d19/d25/d32/d8a/d92/da4/da7/fda zero size 2024-04-26T04:53:09.333 INFO:tasks.workunit.client.0.smithi077.stdout:9/507: symlink d9/d35/d99/lb0 0 2024-04-26T04:53:09.334 INFO:tasks.workunit.client.1.smithi078.stdout:8/946: truncate d0/d19/d52/d35/d73/fc2 2213845 0 2024-04-26T04:53:09.335 INFO:tasks.workunit.client.1.smithi078.stdout:8/947: chown d0/d19/d52/d35/d44/d6b/c82 1842861470 1 2024-04-26T04:53:09.336 INFO:tasks.workunit.client.1.smithi078.stdout:4/970: write d0/d1d/de1/d20/d31/fb9 [28643,112042] 0 2024-04-26T04:53:09.337 INFO:tasks.workunit.client.0.smithi077.stdout:9/508: mkdir d9/d45/db1 0 2024-04-26T04:53:09.337 INFO:tasks.workunit.client.1.smithi078.stdout:4/971: dread - d0/de/d22/deb/d104/d105/f123 zero size 2024-04-26T04:53:09.338 INFO:tasks.workunit.client.1.smithi078.stdout:4/972: stat d0/de/d22/d27/d56/d5a/f7d 0 2024-04-26T04:53:09.339 INFO:tasks.workunit.client.1.smithi078.stdout:8/948: chown d0/d19/d52/c96 75 1 2024-04-26T04:53:09.340 INFO:tasks.workunit.client.1.smithi078.stdout:4/973: read d0/de/d22/d27/f130 [113714,118687] 0 2024-04-26T04:53:09.341 INFO:tasks.workunit.client.1.smithi078.stdout:8/949: write d0/f116 [2850326,74028] 0 2024-04-26T04:53:09.342 INFO:tasks.workunit.client.1.smithi078.stdout:4/974: fdatasync d0/d1d/de1/d20/d31/d47/db2/f125 0 2024-04-26T04:53:09.345 INFO:tasks.workunit.client.0.smithi077.stdout:9/509: mkdir d9/d3a/d79/d95/dad/db2 0 2024-04-26T04:53:09.347 INFO:tasks.workunit.client.1.smithi078.stdout:8/950: readlink d0/d19/d52/d95/d9d/lf2 0 2024-04-26T04:53:09.351 INFO:tasks.workunit.client.1.smithi078.stdout:4/975: truncate d0/f37 1159316 0 2024-04-26T04:53:09.352 INFO:tasks.workunit.client.1.smithi078.stdout:5/938: write d7/f87 [333083,58706] 0 2024-04-26T04:53:09.378 INFO:tasks.workunit.client.0.smithi077.stdout:3/451: write d0/d9/d32/f35 [1220917,53532] 0 2024-04-26T04:53:09.379 INFO:tasks.workunit.client.0.smithi077.stdout:2/519: rename d9/dc/d1d/d34/d39/d63/d79/d2b/f53 to d9/dc/d1d/fac 0 2024-04-26T04:53:09.380 INFO:tasks.workunit.client.0.smithi077.stdout:2/520: readlink d9/d15/l86 0 2024-04-26T04:53:09.384 INFO:tasks.workunit.client.0.smithi077.stdout:4/512: rename d4/de/d15/d1f/d35/d5c to d4/d4e/dad 0 2024-04-26T04:53:09.385 INFO:tasks.workunit.client.0.smithi077.stdout:4/513: dread - d4/d4e/f8d zero size 2024-04-26T04:53:09.390 INFO:tasks.workunit.client.0.smithi077.stdout:4/514: mkdir d4/d17/d18/d43/d6e/d6c/dae 0 2024-04-26T04:53:09.392 INFO:tasks.workunit.client.0.smithi077.stdout:8/619: rename d4/d17/d35 to d4/df/dce 0 2024-04-26T04:53:09.394 INFO:tasks.workunit.client.1.smithi078.stdout:8/951: mknod d0/d19/d52/d35/d44/c131 0 2024-04-26T04:53:09.400 INFO:tasks.workunit.client.0.smithi077.stdout:8/620: truncate d4/df/d18/f4c 741891 0 2024-04-26T04:53:09.403 INFO:tasks.workunit.client.0.smithi077.stdout:2/521: dread d9/f10 [0,4194304] 0 2024-04-26T04:53:09.404 INFO:tasks.workunit.client.0.smithi077.stdout:8/621: creat d4/df/d22/d75/fcf x:0 0 0 2024-04-26T04:53:09.405 INFO:tasks.workunit.client.0.smithi077.stdout:2/522: chown d9/dc/d5f/d78/l7f 536379 1 2024-04-26T04:53:09.406 INFO:tasks.workunit.client.0.smithi077.stdout:7/586: dwrite da/d15/d1a/d39/f7a [0,4194304] 0 2024-04-26T04:53:09.408 INFO:tasks.workunit.client.0.smithi077.stdout:2/523: dread - d9/dc/d1d/d34/d39/d63/d79/f94 zero size 2024-04-26T04:53:09.409 INFO:tasks.workunit.client.0.smithi077.stdout:8/622: mkdir d4/df/d22/d80/d86/dd0 0 2024-04-26T04:53:09.414 INFO:tasks.workunit.client.1.smithi078.stdout:0/968: write d2/d9/d31/d4a/fbe [542203,55583] 0 2024-04-26T04:53:09.416 INFO:tasks.workunit.client.1.smithi078.stdout:3/975: dwrite d16/d3c/d81/fde [0,4194304] 0 2024-04-26T04:53:09.417 INFO:tasks.workunit.client.1.smithi078.stdout:2/986: rmdir d8/d11 39 2024-04-26T04:53:09.421 INFO:tasks.workunit.client.1.smithi078.stdout:4/976: unlink d0/l11 0 2024-04-26T04:53:09.424 INFO:tasks.workunit.client.1.smithi078.stdout:4/977: symlink d0/de/d22/d27/l154 0 2024-04-26T04:53:09.425 INFO:tasks.workunit.client.1.smithi078.stdout:4/978: dread - d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fe6 zero size 2024-04-26T04:53:09.427 INFO:tasks.workunit.client.1.smithi078.stdout:4/979: creat d0/de/d22/deb/d104/d105/f155 x:0 0 0 2024-04-26T04:53:09.430 INFO:tasks.workunit.client.1.smithi078.stdout:4/980: dread - d0/de/d22/d27/d7b/fdb zero size 2024-04-26T04:53:09.430 INFO:tasks.workunit.client.1.smithi078.stdout:9/895: dwrite dc/df/d4b/d22/f97 [0,4194304] 0 2024-04-26T04:53:09.432 INFO:tasks.workunit.client.1.smithi078.stdout:9/896: dread - dc/df/d4b/dae/fdc zero size 2024-04-26T04:53:09.434 INFO:tasks.workunit.client.1.smithi078.stdout:9/897: symlink dc/df/d17/l10e 0 2024-04-26T04:53:09.435 INFO:tasks.workunit.client.1.smithi078.stdout:4/981: mkdir d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121/d131/d156 0 2024-04-26T04:53:09.435 INFO:tasks.workunit.client.1.smithi078.stdout:9/898: symlink dc/df/d27/l10f 0 2024-04-26T04:53:09.438 INFO:tasks.workunit.client.1.smithi078.stdout:9/899: creat dc/df/d4b/d22/d2a/d57/d93/f110 x:0 0 0 2024-04-26T04:53:09.439 INFO:tasks.workunit.client.1.smithi078.stdout:9/900: write dc/df/d17/f88 [2564135,63195] 0 2024-04-26T04:53:09.441 INFO:tasks.workunit.client.1.smithi078.stdout:9/901: chown dc/df/d4b/d22/d2a/d57/d93/ldf 201127 1 2024-04-26T04:53:09.441 INFO:tasks.workunit.client.1.smithi078.stdout:4/982: dwrite d0/d1d/de1/d20/d31/f73 [4194304,4194304] 0 2024-04-26T04:53:09.444 INFO:tasks.workunit.client.1.smithi078.stdout:4/983: fdatasync d0/d1d/d46/f6e 0 2024-04-26T04:53:09.445 INFO:tasks.workunit.client.1.smithi078.stdout:9/902: unlink dc/df/d4b/d22/d2a/d57/ld0 0 2024-04-26T04:53:09.447 INFO:tasks.workunit.client.1.smithi078.stdout:9/903: creat dc/dd9/de1/f111 x:0 0 0 2024-04-26T04:53:09.448 INFO:tasks.workunit.client.1.smithi078.stdout:9/904: mknod dc/df/d4b/d58/c112 0 2024-04-26T04:53:09.450 INFO:tasks.workunit.client.1.smithi078.stdout:9/905: mkdir dc/df/d27/d81/d113 0 2024-04-26T04:53:09.451 INFO:tasks.workunit.client.1.smithi078.stdout:9/906: dread - dc/df/d4b/d22/d2d/d98/fd4 zero size 2024-04-26T04:53:09.453 INFO:tasks.workunit.client.1.smithi078.stdout:9/907: mknod dc/df/d4b/d22/d2a/d57/d69/d106/c114 0 2024-04-26T04:53:09.469 INFO:tasks.workunit.client.0.smithi077.stdout:1/482: sync 2024-04-26T04:53:09.471 INFO:tasks.workunit.client.0.smithi077.stdout:2/524: unlink d9/dc/d5f/l65 0 2024-04-26T04:53:09.473 INFO:tasks.workunit.client.0.smithi077.stdout:8/623: creat d4/d17/d90/fd1 x:0 0 0 2024-04-26T04:53:09.473 INFO:tasks.workunit.client.0.smithi077.stdout:1/483: write d9/d53/f6f [4058111,19419] 0 2024-04-26T04:53:09.474 INFO:tasks.workunit.client.0.smithi077.stdout:5/526: creat d1/fc1 x:0 0 0 2024-04-26T04:53:09.478 INFO:tasks.workunit.client.0.smithi077.stdout:2/525: mknod d9/dc/d5f/d8e/cad 0 2024-04-26T04:53:09.481 INFO:tasks.workunit.client.0.smithi077.stdout:4/515: unlink d4/de/d15/f5b 0 2024-04-26T04:53:09.484 INFO:tasks.workunit.client.0.smithi077.stdout:8/624: dwrite d4/df/d22/d2d/fc1 [0,4194304] 0 2024-04-26T04:53:09.484 INFO:tasks.workunit.client.0.smithi077.stdout:5/527: dread d1/d8/f10 [4194304,4194304] 0 2024-04-26T04:53:09.489 INFO:tasks.workunit.client.0.smithi077.stdout:4/516: creat d4/de/faf x:0 0 0 2024-04-26T04:53:09.491 INFO:tasks.workunit.client.0.smithi077.stdout:5/528: creat d1/d42/da4/d84/fc2 x:0 0 0 2024-04-26T04:53:09.494 INFO:tasks.workunit.client.0.smithi077.stdout:8/625: rename d4/d69/d73/c78 to d4/df/d22/d80/d86/cd2 0 2024-04-26T04:53:09.495 INFO:tasks.workunit.client.0.smithi077.stdout:1/484: getdents d9/db/dd/d34 0 2024-04-26T04:53:09.498 INFO:tasks.workunit.client.0.smithi077.stdout:5/529: unlink d1/d42/dac/cb8 0 2024-04-26T04:53:09.500 INFO:tasks.workunit.client.0.smithi077.stdout:8/626: creat d4/df/dc6/fd3 x:0 0 0 2024-04-26T04:53:09.501 INFO:tasks.workunit.client.0.smithi077.stdout:8/627: dread - d4/df/d18/f74 zero size 2024-04-26T04:53:09.502 INFO:tasks.workunit.client.0.smithi077.stdout:1/485: creat d9/db/dd/d68/d81/f88 x:0 0 0 2024-04-26T04:53:09.502 INFO:tasks.workunit.client.0.smithi077.stdout:8/628: dread - d4/d17/fc5 zero size 2024-04-26T04:53:09.502 INFO:tasks.workunit.client.1.smithi078.stdout:4/984: dread d0/d1d/d46/d100/f132 [0,4194304] 0 2024-04-26T04:53:09.503 INFO:tasks.workunit.client.0.smithi077.stdout:5/530: creat d1/d12/d87/fc3 x:0 0 0 2024-04-26T04:53:09.504 INFO:tasks.workunit.client.0.smithi077.stdout:5/531: chown d1/d30/d33/l88 0 1 2024-04-26T04:53:09.505 INFO:tasks.workunit.client.1.smithi078.stdout:5/939: sync 2024-04-26T04:53:09.505 INFO:tasks.workunit.client.0.smithi077.stdout:5/532: chown d1/d8/fc 259 1 2024-04-26T04:53:09.505 INFO:tasks.workunit.client.0.smithi077.stdout:8/629: readlink d4/df/d22/d2d/d2b/l6d 0 2024-04-26T04:53:09.507 INFO:tasks.workunit.client.0.smithi077.stdout:1/486: creat d9/db/dd/d68/f89 x:0 0 0 2024-04-26T04:53:09.507 INFO:tasks.workunit.client.0.smithi077.stdout:8/630: write d4/df/d22/d75/fcf [690043,54962] 0 2024-04-26T04:53:09.510 INFO:tasks.workunit.client.1.smithi078.stdout:5/940: mkdir d7/d48/d57/d65/d46/de7/d13b/d13f 0 2024-04-26T04:53:09.510 INFO:tasks.workunit.client.1.smithi078.stdout:4/985: dread d0/d1d/de1/d20/d31/d47/db2/f5b [0,4194304] 0 2024-04-26T04:53:09.511 INFO:tasks.workunit.client.1.smithi078.stdout:5/941: dread - d7/d18/d32/fbe zero size 2024-04-26T04:53:09.513 INFO:tasks.workunit.client.0.smithi077.stdout:1/487: creat d9/db/dd/d34/d63/f8a x:0 0 0 2024-04-26T04:53:09.515 INFO:tasks.workunit.client.1.smithi078.stdout:5/942: read d7/d18/d1c/d8e/fcd [1006090,16622] 0 2024-04-26T04:53:09.517 INFO:tasks.workunit.client.1.smithi078.stdout:4/986: dwrite d0/d1d/de1/d20/d31/f73 [4194304,4194304] 0 2024-04-26T04:53:09.517 INFO:tasks.workunit.client.1.smithi078.stdout:5/943: write d7/d11/f47 [3405296,38568] 0 2024-04-26T04:53:09.517 INFO:tasks.workunit.client.0.smithi077.stdout:9/510: link d9/d35/d99/fa3 d9/dd/fb3 0 2024-04-26T04:53:09.520 INFO:tasks.workunit.client.0.smithi077.stdout:1/488: truncate d9/db/f13 4346323 0 2024-04-26T04:53:09.522 INFO:tasks.workunit.client.1.smithi078.stdout:5/944: rmdir d7/d48/d57/d65/d46/dce 39 2024-04-26T04:53:09.524 INFO:tasks.workunit.client.0.smithi077.stdout:9/511: symlink d9/d35/lb4 0 2024-04-26T04:53:09.527 INFO:tasks.workunit.client.0.smithi077.stdout:5/533: dread d1/d12/d28/d38/f8b [0,4194304] 0 2024-04-26T04:53:09.527 INFO:tasks.workunit.client.0.smithi077.stdout:1/489: fsync d9/d53/d6e/f87 0 2024-04-26T04:53:09.532 INFO:tasks.workunit.client.0.smithi077.stdout:1/490: mkdir d9/d53/d8b 0 2024-04-26T04:53:09.533 INFO:tasks.workunit.client.1.smithi078.stdout:7/928: rename d9/da/d11/d13/d1a/d52/d5a/d41/d68/d12e to d9/da/d130 0 2024-04-26T04:53:09.535 INFO:tasks.workunit.client.0.smithi077.stdout:9/512: symlink d9/dd/d1b/d2b/d32/lb5 0 2024-04-26T04:53:09.537 INFO:tasks.workunit.client.1.smithi078.stdout:0/969: rename d2/d1e/f2a to d2/d9/d78/deb/f146 0 2024-04-26T04:53:09.539 INFO:tasks.workunit.client.1.smithi078.stdout:7/929: creat d9/da/d33/d4b/d4c/d5b/d83/de6/f131 x:0 0 0 2024-04-26T04:53:09.540 INFO:tasks.workunit.client.1.smithi078.stdout:4/987: dread d0/d1d/de1/d20/d31/d47/db2/d44/fb1 [0,4194304] 0 2024-04-26T04:53:09.540 INFO:tasks.workunit.client.0.smithi077.stdout:5/534: link d1/f5 d1/d42/d62/fc4 0 2024-04-26T04:53:09.542 INFO:tasks.workunit.client.0.smithi077.stdout:9/513: write d9/f82 [1021504,55409] 0 2024-04-26T04:53:09.547 INFO:tasks.workunit.client.1.smithi078.stdout:3/976: rename d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f114 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/dd6/f151 0 2024-04-26T04:53:09.550 INFO:tasks.workunit.client.0.smithi077.stdout:1/491: dread d9/db/dd/d34/d63/f77 [0,4194304] 0 2024-04-26T04:53:09.551 INFO:tasks.workunit.client.0.smithi077.stdout:9/514: creat d9/dd/d1b/d2b/d32/d36/fb6 x:0 0 0 2024-04-26T04:53:09.553 INFO:tasks.workunit.client.1.smithi078.stdout:8/952: truncate d0/d19/d52/d35/d73/fe8 492166 0 2024-04-26T04:53:09.559 INFO:tasks.workunit.client.0.smithi077.stdout:9/515: mknod d9/d3a/d79/d50/cb7 0 2024-04-26T04:53:09.559 INFO:tasks.workunit.client.0.smithi077.stdout:7/587: truncate da/d6b/f93 3158097 0 2024-04-26T04:53:09.563 INFO:tasks.workunit.client.1.smithi078.stdout:8/953: rmdir d0/d19/d52/d35/d44/d6b/d6c/db4 39 2024-04-26T04:53:09.564 INFO:tasks.workunit.client.0.smithi077.stdout:9/516: rmdir d9/dd/d3d 39 2024-04-26T04:53:09.564 INFO:tasks.workunit.client.1.smithi078.stdout:8/954: write d0/d7a/dbe/fd8 [3137659,10854] 0 2024-04-26T04:53:09.565 INFO:tasks.workunit.client.0.smithi077.stdout:9/517: write d9/dd/d1b/d2b/f83 [450530,67326] 0 2024-04-26T04:53:09.569 INFO:tasks.workunit.client.1.smithi078.stdout:0/970: creat d2/f147 x:0 0 0 2024-04-26T04:53:09.571 INFO:tasks.workunit.client.0.smithi077.stdout:7/588: dread da/d15/d1a/f20 [0,4194304] 0 2024-04-26T04:53:09.572 INFO:tasks.workunit.client.0.smithi077.stdout:7/589: dread - da/d6b/fa4 zero size 2024-04-26T04:53:09.573 INFO:tasks.workunit.client.1.smithi078.stdout:4/988: link d0/de/d22/d27/d56/ca0 d0/d1d/de1/d20/dd5/c157 0 2024-04-26T04:53:09.574 INFO:tasks.workunit.client.1.smithi078.stdout:0/971: creat d2/d9/d60/d6a/daa/f148 x:0 0 0 2024-04-26T04:53:09.575 INFO:tasks.workunit.client.0.smithi077.stdout:7/590: write da/d13/d22/d5d/f60 [290509,82028] 0 2024-04-26T04:53:09.577 INFO:tasks.workunit.client.1.smithi078.stdout:2/987: rename d8/d11/l65 to d8/d11/d18/d5d/d8b/l151 0 2024-04-26T04:53:09.577 INFO:tasks.workunit.client.0.smithi077.stdout:9/518: dwrite d9/dd/d3d/f3e [0,4194304] 0 2024-04-26T04:53:09.578 INFO:tasks.workunit.client.1.smithi078.stdout:2/988: chown d8/d11/d14/d5b/f118 16104 1 2024-04-26T04:53:09.579 INFO:tasks.workunit.client.1.smithi078.stdout:4/989: symlink d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/d2e/d85/d121/l158 0 2024-04-26T04:53:09.582 INFO:tasks.workunit.client.1.smithi078.stdout:0/972: symlink d2/d9/d18/db7/d92/l149 0 2024-04-26T04:53:09.586 INFO:tasks.workunit.client.1.smithi078.stdout:2/989: mkdir d8/d11/d18/d5d/d127/d130/d152 0 2024-04-26T04:53:09.587 INFO:tasks.workunit.client.1.smithi078.stdout:4/990: fsync d0/de/f91 0 2024-04-26T04:53:09.587 INFO:tasks.workunit.client.1.smithi078.stdout:2/990: dread - d8/db7/f11c zero size 2024-04-26T04:53:09.588 INFO:tasks.workunit.client.1.smithi078.stdout:2/991: chown d8/f84 1483922636 1 2024-04-26T04:53:09.590 INFO:tasks.workunit.client.1.smithi078.stdout:9/908: dwrite dc/df/f1e [0,4194304] 0 2024-04-26T04:53:09.592 INFO:tasks.workunit.client.1.smithi078.stdout:2/992: symlink d8/d11/d18/dfe/d14a/l153 0 2024-04-26T04:53:09.593 INFO:tasks.workunit.client.0.smithi077.stdout:4/517: getdents d4/de 0 2024-04-26T04:53:09.594 INFO:tasks.workunit.client.0.smithi077.stdout:2/526: dwrite d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:53:09.594 INFO:tasks.workunit.client.1.smithi078.stdout:0/973: getdents d2/d9/d18 0 2024-04-26T04:53:09.595 INFO:tasks.workunit.client.0.smithi077.stdout:2/527: read - d9/dc/d1d/d34/d39/fa3 zero size 2024-04-26T04:53:09.598 INFO:tasks.workunit.client.1.smithi078.stdout:2/993: creat d8/db7/f154 x:0 0 0 2024-04-26T04:53:09.614 INFO:tasks.workunit.client.1.smithi078.stdout:4/991: dread d0/d1d/de1/d20/d31/d47/db2/d4c/d62/daa/f14f [0,4194304] 0 2024-04-26T04:53:09.630 INFO:tasks.workunit.client.1.smithi078.stdout:8/955: sync 2024-04-26T04:53:09.634 INFO:tasks.workunit.client.1.smithi078.stdout:8/956: write d0/f23 [903128,9640] 0 2024-04-26T04:53:09.725 INFO:tasks.workunit.client.0.smithi077.stdout:2/528: creat d9/dc/d6e/fae x:0 0 0 2024-04-26T04:53:09.728 INFO:tasks.workunit.client.1.smithi078.stdout:9/909: truncate dc/df/d4b/d3e/f5d 986980 0 2024-04-26T04:53:09.729 INFO:tasks.workunit.client.1.smithi078.stdout:2/994: mknod d8/d11/d18/d5d/d8b/db0/c155 0 2024-04-26T04:53:09.732 INFO:tasks.workunit.client.0.smithi077.stdout:8/631: dwrite d4/df/f16 [0,4194304] 0 2024-04-26T04:53:09.733 INFO:tasks.workunit.client.0.smithi077.stdout:2/529: symlink d9/d15/laf 0 2024-04-26T04:53:09.736 INFO:tasks.workunit.client.0.smithi077.stdout:4/518: getdents d4/de/d15/d1f 0 2024-04-26T04:53:09.737 INFO:tasks.workunit.client.0.smithi077.stdout:4/519: dread - d4/de/fa6 zero size 2024-04-26T04:53:09.737 INFO:tasks.workunit.client.0.smithi077.stdout:1/492: read d9/db/f13 [1249313,119906] 0 2024-04-26T04:53:09.737 INFO:tasks.workunit.client.1.smithi078.stdout:8/957: chown d0/d19/d52/d35/d44/d6b/d104/d58/d112/l126 65286131 1 2024-04-26T04:53:09.738 INFO:tasks.workunit.client.0.smithi077.stdout:4/520: write d4/d17/f96 [255455,89048] 0 2024-04-26T04:53:09.739 INFO:tasks.workunit.client.0.smithi077.stdout:2/530: mknod d9/dc/d1d/d1e/cb0 0 2024-04-26T04:53:09.739 INFO:tasks.workunit.client.0.smithi077.stdout:4/521: chown d4/d17/d18/l73 128617 1 2024-04-26T04:53:09.739 INFO:tasks.workunit.client.0.smithi077.stdout:4/522: readlink d4/d4e/l97 0 2024-04-26T04:53:09.740 INFO:tasks.workunit.client.0.smithi077.stdout:2/531: write d9/dc/f7a [933402,71483] 0 2024-04-26T04:53:09.740 INFO:tasks.workunit.client.0.smithi077.stdout:4/523: stat d4/d4e/dad/d8b/fa5 0 2024-04-26T04:53:09.741 INFO:tasks.workunit.client.0.smithi077.stdout:4/524: fdatasync d4/d17/f96 0 2024-04-26T04:53:09.742 INFO:tasks.workunit.client.0.smithi077.stdout:4/525: write d4/de/faf [945391,73683] 0 2024-04-26T04:53:09.744 INFO:tasks.workunit.client.0.smithi077.stdout:1/493: symlink d9/d53/d6e/d7c/l8c 0 2024-04-26T04:53:09.746 INFO:tasks.workunit.client.1.smithi078.stdout:5/945: dwrite d7/d48/d57/d65/d46/dc4/fd5 [0,4194304] 0 2024-04-26T04:53:09.753 INFO:tasks.workunit.client.0.smithi077.stdout:4/526: truncate d4/d17/d18/d43/d6e/f84 637931 0 2024-04-26T04:53:09.755 INFO:tasks.workunit.client.1.smithi078.stdout:3/977: rmdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a 39 2024-04-26T04:53:09.757 INFO:tasks.workunit.client.0.smithi077.stdout:1/494: mknod d9/db/d2b/d5c/c8d 0 2024-04-26T04:53:09.758 INFO:tasks.workunit.client.1.smithi078.stdout:7/930: write d9/fd5 [12159688,68093] 0 2024-04-26T04:53:09.758 INFO:tasks.workunit.client.0.smithi077.stdout:1/495: chown d9/db/f1c 1165086797 1 2024-04-26T04:53:09.760 INFO:tasks.workunit.client.0.smithi077.stdout:1/496: chown d9/db/d2b/d3f/f52 12880 1 2024-04-26T04:53:09.760 INFO:tasks.workunit.client.0.smithi077.stdout:8/632: link d4/df/d18/d56/f72 d4/fd4 0 2024-04-26T04:53:09.762 INFO:tasks.workunit.client.0.smithi077.stdout:5/535: dwrite d1/d30/d35/f57 [0,4194304] 0 2024-04-26T04:53:09.762 INFO:tasks.workunit.client.0.smithi077.stdout:2/532: symlink d9/dc/d1d/d34/lb1 0 2024-04-26T04:53:09.772 INFO:tasks.workunit.client.0.smithi077.stdout:8/633: mknod d4/df/dc6/cd5 0 2024-04-26T04:53:09.775 INFO:tasks.workunit.client.0.smithi077.stdout:5/536: mknod d1/d42/cc5 0 2024-04-26T04:53:09.779 INFO:tasks.workunit.client.0.smithi077.stdout:1/497: fsync d9/db/dd/f23 0 2024-04-26T04:53:09.785 INFO:tasks.workunit.client.0.smithi077.stdout:7/591: dwrite da/f68 [0,4194304] 0 2024-04-26T04:53:09.788 INFO:tasks.workunit.client.0.smithi077.stdout:5/537: dread d1/d42/da4/d25/f4b [0,4194304] 0 2024-04-26T04:53:09.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/527: dread d4/de/d15/d1f/f32 [0,4194304] 0 2024-04-26T04:53:09.790 INFO:tasks.workunit.client.0.smithi077.stdout:4/528: truncate d4/de/fa6 1015536 0 2024-04-26T04:53:09.794 INFO:tasks.workunit.client.1.smithi078.stdout:4/992: write d0/de/d22/d27/d56/f6a [880152,48176] 0 2024-04-26T04:53:09.796 INFO:tasks.workunit.client.1.smithi078.stdout:0/974: dwrite d2/d9/d60/dae/f109 [0,4194304] 0 2024-04-26T04:53:09.797 INFO:tasks.workunit.client.1.smithi078.stdout:0/975: chown d2/d9/l13f 138075023 1 2024-04-26T04:53:09.798 INFO:tasks.workunit.client.1.smithi078.stdout:0/976: dread - d2/d9/d31/f12e zero size 2024-04-26T04:53:09.801 INFO:tasks.workunit.client.0.smithi077.stdout:1/498: creat d9/d53/d8b/f8e x:0 0 0 2024-04-26T04:53:09.803 INFO:tasks.workunit.client.0.smithi077.stdout:4/529: rename d4/d17/d18/d43/d6e/d6c/c6f to d4/d17/d18/d43/d51/cb0 0 2024-04-26T04:53:09.807 INFO:tasks.workunit.client.0.smithi077.stdout:8/634: creat d4/df/fd6 x:0 0 0 2024-04-26T04:53:09.807 INFO:tasks.workunit.client.0.smithi077.stdout:8/635: dread - d4/df/fd6 zero size 2024-04-26T04:53:09.816 INFO:tasks.workunit.client.0.smithi077.stdout:4/530: rename d4/de/d15/d1f/d25/l48 to d4/d17/d18/d43/d6e/d6c/lb1 0 2024-04-26T04:53:09.818 INFO:tasks.workunit.client.1.smithi078.stdout:4/993: dread d0/d1d/de1/d20/d31/d47/db2/d44/f68 [0,4194304] 0 2024-04-26T04:53:09.818 INFO:tasks.workunit.client.0.smithi077.stdout:8/636: creat d4/df/dce/d5b/d8d/fd7 x:0 0 0 2024-04-26T04:53:09.819 INFO:tasks.workunit.client.1.smithi078.stdout:4/994: chown d0/de/d22/d27/d7b/f82 9 1 2024-04-26T04:53:09.819 INFO:tasks.workunit.client.0.smithi077.stdout:5/538: dread d1/d12/f60 [0,4194304] 0 2024-04-26T04:53:09.820 INFO:tasks.workunit.client.0.smithi077.stdout:1/499: mknod d9/c8f 0 2024-04-26T04:53:09.823 INFO:tasks.workunit.client.0.smithi077.stdout:8/637: mkdir d4/df/d22/d2d/d2b/d33/dd8 0 2024-04-26T04:53:09.825 INFO:tasks.workunit.client.0.smithi077.stdout:5/539: fsync d1/d42/da4/f34 0 2024-04-26T04:53:09.827 INFO:tasks.workunit.client.0.smithi077.stdout:1/500: rmdir d9/d53/d6e 39 2024-04-26T04:53:09.828 INFO:tasks.workunit.client.0.smithi077.stdout:1/501: chown d9/db/d2b/d3f/f41 2866 1 2024-04-26T04:53:09.829 INFO:tasks.workunit.client.0.smithi077.stdout:4/531: truncate d4/d17/d18/d43/d53/d75/f8e 168221 0 2024-04-26T04:53:09.830 INFO:tasks.workunit.client.0.smithi077.stdout:8/638: fdatasync d4/df/d22/d80/fae 0 2024-04-26T04:53:09.834 INFO:tasks.workunit.client.0.smithi077.stdout:2/533: dwrite d9/dc/d1d/fac [0,4194304] 0 2024-04-26T04:53:09.837 INFO:tasks.workunit.client.0.smithi077.stdout:4/532: mknod d4/d4e/dad/cb2 0 2024-04-26T04:53:09.841 INFO:tasks.workunit.client.0.smithi077.stdout:5/540: creat d1/d8/d7e/fc6 x:0 0 0 2024-04-26T04:53:09.842 INFO:tasks.workunit.client.0.smithi077.stdout:5/541: fsync d1/d8d/da7/fb4 0 2024-04-26T04:53:09.845 INFO:tasks.workunit.client.0.smithi077.stdout:2/534: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/cb2 0 2024-04-26T04:53:09.846 INFO:tasks.workunit.client.0.smithi077.stdout:7/592: truncate da/d15/faa 3151605 0 2024-04-26T04:53:09.850 INFO:tasks.workunit.client.0.smithi077.stdout:2/535: write d9/d15/d44/fa8 [901780,35513] 0 2024-04-26T04:53:09.851 INFO:tasks.workunit.client.0.smithi077.stdout:0/481: sync 2024-04-26T04:53:09.854 INFO:tasks.workunit.client.1.smithi078.stdout:8/958: mkdir d0/d19/d25/d32/d132 0 2024-04-26T04:53:09.855 INFO:tasks.workunit.client.0.smithi077.stdout:1/502: getdents d9/db/dd/d34/d63 0 2024-04-26T04:53:09.856 INFO:tasks.workunit.client.0.smithi077.stdout:0/482: dread d0/d19/f32 [0,4194304] 0 2024-04-26T04:53:09.859 INFO:tasks.workunit.client.0.smithi077.stdout:4/533: rename d4/de/d15/d1f/d35/f52 to d4/d17/d18/d43/d63/fb3 0 2024-04-26T04:53:09.861 INFO:tasks.workunit.client.0.smithi077.stdout:5/542: dread d1/d42/d62/d6b/f85 [0,4194304] 0 2024-04-26T04:53:09.863 INFO:tasks.workunit.client.1.smithi078.stdout:7/931: fdatasync d9/da/d11/d13/d1a/f10a 0 2024-04-26T04:53:09.865 INFO:tasks.workunit.client.1.smithi078.stdout:7/932: write d9/da/d33/d4b/d4c/d5b/dc3/fdc [1725584,75821] 0 2024-04-26T04:53:09.871 INFO:tasks.workunit.client.1.smithi078.stdout:0/977: rmdir d2/d9/d78/d84/d103 39 2024-04-26T04:53:09.872 INFO:tasks.workunit.client.0.smithi077.stdout:4/534: stat d4/d17/d18/d43/d53/d75/la8 0 2024-04-26T04:53:09.874 INFO:tasks.workunit.client.0.smithi077.stdout:2/536: creat d9/dc/d5f/d8e/da6/fb3 x:0 0 0 2024-04-26T04:53:09.875 INFO:tasks.workunit.client.0.smithi077.stdout:2/537: chown d9/dc/d1d/d34/d39/c76 1 1 2024-04-26T04:53:09.877 INFO:tasks.workunit.client.0.smithi077.stdout:5/543: dread d1/d8/f49 [0,4194304] 0 2024-04-26T04:53:09.877 INFO:tasks.workunit.client.0.smithi077.stdout:2/538: readlink d9/dc/d1d/d34/d39/d63/l48 0 2024-04-26T04:53:09.881 INFO:tasks.workunit.client.1.smithi078.stdout:8/959: rename d0/d19/d52/d35/d44/d6b/d104 to d0/d19/d25/d32/d8a/d92/da4/d133 0 2024-04-26T04:53:09.886 INFO:tasks.workunit.client.0.smithi077.stdout:4/535: rename d4/de to d4/d17/d18/d43/d51/db4 0 2024-04-26T04:53:09.888 INFO:tasks.workunit.client.0.smithi077.stdout:7/593: write da/d13/f30 [1560184,84879] 0 2024-04-26T04:53:09.889 INFO:tasks.workunit.client.0.smithi077.stdout:7/594: chown da/d13/d22/d5d/la5 5270 1 2024-04-26T04:53:09.889 INFO:tasks.workunit.client.0.smithi077.stdout:5/544: unlink d1/lba 0 2024-04-26T04:53:09.890 INFO:tasks.workunit.client.0.smithi077.stdout:0/483: write d0/d19/d1e/d2f/f44 [905145,80731] 0 2024-04-26T04:53:09.890 INFO:tasks.workunit.client.1.smithi078.stdout:5/946: dwrite d7/d18/d32/d37/d79/f115 [0,4194304] 0 2024-04-26T04:53:09.894 INFO:tasks.workunit.client.0.smithi077.stdout:4/536: creat d4/d17/d18/d43/fb5 x:0 0 0 2024-04-26T04:53:09.895 INFO:tasks.workunit.client.1.smithi078.stdout:9/910: creat dc/df/d4b/d22/d2a/d57/d69/f115 x:0 0 0 2024-04-26T04:53:09.898 INFO:tasks.workunit.client.0.smithi077.stdout:7/595: unlink da/d13/d22/d34/f92 0 2024-04-26T04:53:09.902 INFO:tasks.workunit.client.1.smithi078.stdout:3/978: truncate d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/f68 2532565 0 2024-04-26T04:53:09.903 INFO:tasks.workunit.client.1.smithi078.stdout:2/995: dwrite d8/d11/d18/d5d/d61/f76 [0,4194304] 0 2024-04-26T04:53:09.904 INFO:tasks.workunit.client.1.smithi078.stdout:3/979: read - d16/d26/d2e/d5f/d49/f141 zero size 2024-04-26T04:53:09.904 INFO:tasks.workunit.client.0.smithi077.stdout:5/545: rename d1/d8d/cb0 to d1/d42/d62/d6b/cc7 0 2024-04-26T04:53:09.906 INFO:tasks.workunit.client.1.smithi078.stdout:5/947: chown d7/d18/d1c/c56 5 1 2024-04-26T04:53:09.907 INFO:tasks.workunit.client.1.smithi078.stdout:4/995: sync 2024-04-26T04:53:09.907 INFO:tasks.workunit.client.0.smithi077.stdout:4/537: chown d4/d17/d18/d43/d6e/f84 38 1 2024-04-26T04:53:09.908 INFO:tasks.workunit.client.0.smithi077.stdout:4/538: fdatasync d4/d17/f98 0 2024-04-26T04:53:09.912 INFO:tasks.workunit.client.0.smithi077.stdout:7/596: fsync da/ff 0 2024-04-26T04:53:09.915 INFO:tasks.workunit.client.0.smithi077.stdout:0/484: symlink d0/d19/d8d/lac 0 2024-04-26T04:53:09.916 INFO:tasks.workunit.client.0.smithi077.stdout:0/485: read d0/d2c/d4e/f14 [938247,75234] 0 2024-04-26T04:53:09.917 INFO:tasks.workunit.client.0.smithi077.stdout:4/539: dread d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:09.917 INFO:tasks.workunit.client.1.smithi078.stdout:7/933: link f6 d9/da/d33/d4b/d4c/d5b/dc3/d128/f132 0 2024-04-26T04:53:09.918 INFO:tasks.workunit.client.0.smithi077.stdout:2/539: creat d9/d15/fb4 x:0 0 0 2024-04-26T04:53:09.920 INFO:tasks.workunit.client.1.smithi078.stdout:8/960: dwrite d0/d19/d52/d35/d44/d6b/f9b [0,4194304] 0 2024-04-26T04:53:09.922 INFO:tasks.workunit.client.0.smithi077.stdout:5/546: truncate d1/d42/d62/f89 561296 0 2024-04-26T04:53:09.926 INFO:tasks.workunit.client.0.smithi077.stdout:5/547: stat d1/d30/d35/f6c 0 2024-04-26T04:53:09.926 INFO:tasks.workunit.client.0.smithi077.stdout:4/540: dwrite d4/d17/f98 [0,4194304] 0 2024-04-26T04:53:09.927 INFO:tasks.workunit.client.1.smithi078.stdout:0/978: truncate d2/d9/d18/db7/fff 264169 0 2024-04-26T04:53:09.927 INFO:tasks.workunit.client.1.smithi078.stdout:4/996: rmdir d0/d1d/de1/d20/d31/d47/db2/d4c/dba 39 2024-04-26T04:53:09.927 INFO:tasks.workunit.client.1.smithi078.stdout:8/961: dwrite d0/d19/d52/d35/d44/f93 [4194304,4194304] 0 2024-04-26T04:53:09.930 INFO:tasks.workunit.client.0.smithi077.stdout:0/486: readlink d0/l1d 0 2024-04-26T04:53:09.931 INFO:tasks.workunit.client.0.smithi077.stdout:5/548: dwrite d1/d12/f50 [0,4194304] 0 2024-04-26T04:53:09.932 INFO:tasks.workunit.client.0.smithi077.stdout:0/487: write d0/d19/f49 [4970385,6613] 0 2024-04-26T04:53:09.933 INFO:tasks.workunit.client.1.smithi078.stdout:9/911: dwrite dc/f4e [0,4194304] 0 2024-04-26T04:53:09.934 INFO:tasks.workunit.client.0.smithi077.stdout:2/540: creat d9/dc/d1d/d1e/d77/fb5 x:0 0 0 2024-04-26T04:53:09.943 INFO:tasks.workunit.client.1.smithi078.stdout:5/948: mknod d7/d11/c140 0 2024-04-26T04:53:09.945 INFO:tasks.workunit.client.1.smithi078.stdout:8/962: stat d0/d19/d52/f69 0 2024-04-26T04:53:09.947 INFO:tasks.workunit.client.0.smithi077.stdout:5/549: unlink d1/d8/f22 0 2024-04-26T04:53:09.948 INFO:tasks.workunit.client.0.smithi077.stdout:5/550: write d1/d42/da4/d84/fc2 [168168,90516] 0 2024-04-26T04:53:09.949 INFO:tasks.workunit.client.0.smithi077.stdout:5/551: fsync d1/d12/d87/fc3 0 2024-04-26T04:53:09.950 INFO:tasks.workunit.client.0.smithi077.stdout:5/552: chown d1/d30/d35 7 1 2024-04-26T04:53:09.950 INFO:tasks.workunit.client.0.smithi077.stdout:4/541: mkdir d4/d17/d18/d43/d63/d80/db6 0 2024-04-26T04:53:09.951 INFO:tasks.workunit.client.1.smithi078.stdout:7/934: mkdir d9/da/d33/d4b/d4c/d6c/d111/d133 0 2024-04-26T04:53:09.952 INFO:tasks.workunit.client.1.smithi078.stdout:7/935: read - d9/d57/f12c zero size 2024-04-26T04:53:09.956 INFO:tasks.workunit.client.1.smithi078.stdout:2/996: rename d8/d11/d18/d137/d34/dd3/de4/fea to d8/d11/d18/d5d/d61/f156 0 2024-04-26T04:53:09.957 INFO:tasks.workunit.client.0.smithi077.stdout:5/553: unlink d1/d12/d45/f5e 0 2024-04-26T04:53:09.958 INFO:tasks.workunit.client.1.smithi078.stdout:7/936: dwrite d9/da/d11/d13/d1a/f10a [0,4194304] 0 2024-04-26T04:53:09.959 INFO:tasks.workunit.client.0.smithi077.stdout:4/542: dwrite d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:09.959 INFO:tasks.workunit.client.0.smithi077.stdout:7/597: getdents da/d15 0 2024-04-26T04:53:09.960 INFO:tasks.workunit.client.1.smithi078.stdout:5/949: mknod d7/d48/da9/c141 0 2024-04-26T04:53:09.963 INFO:tasks.workunit.client.1.smithi078.stdout:8/963: mknod d0/d19/d52/d5c/da2/c134 0 2024-04-26T04:53:09.966 INFO:tasks.workunit.client.0.smithi077.stdout:5/554: rename d1/d8/d72/da6 to d1/d42/d62/dc8 0 2024-04-26T04:53:09.967 INFO:tasks.workunit.client.0.smithi077.stdout:7/598: mknod da/d13/d22/d5d/dbd/cc8 0 2024-04-26T04:53:09.969 INFO:tasks.workunit.client.0.smithi077.stdout:4/543: unlink d4/d17/d18/d43/d51/db4/d15/d1f/d76/l78 0 2024-04-26T04:53:09.972 INFO:tasks.workunit.client.0.smithi077.stdout:7/599: mknod da/d13/d22/cc9 0 2024-04-26T04:53:09.975 INFO:tasks.workunit.client.0.smithi077.stdout:5/555: truncate d1/d12/f1f 3872655 0 2024-04-26T04:53:09.977 INFO:tasks.workunit.client.0.smithi077.stdout:4/544: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7 0 2024-04-26T04:53:09.977 INFO:tasks.workunit.client.1.smithi078.stdout:0/979: link d2/d39/l99 d2/d39/d96/l14a 0 2024-04-26T04:53:09.978 INFO:tasks.workunit.client.0.smithi077.stdout:4/545: chown d4/d17/d18/d43/d51/db4/d15/d1f/d76/f87 10065 1 2024-04-26T04:53:09.979 INFO:tasks.workunit.client.0.smithi077.stdout:7/600: symlink da/d13/d22/d5d/d9f/lca 0 2024-04-26T04:53:09.981 INFO:tasks.workunit.client.1.smithi078.stdout:3/980: rename d16/d26/d2e/d5f/ff8 to d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130/d123/d14f/f152 0 2024-04-26T04:53:09.983 INFO:tasks.workunit.client.1.smithi078.stdout:5/950: mknod d7/d48/d57/d65/d46/de7/d13b/d13f/c142 0 2024-04-26T04:53:09.985 INFO:tasks.workunit.client.0.smithi077.stdout:5/556: mknod d1/d30/d33/cc9 0 2024-04-26T04:53:09.986 INFO:tasks.workunit.client.1.smithi078.stdout:8/964: unlink d0/d19/d25/d32/d8a/d92/da4/d133/d58/d5b/d61/fd9 0 2024-04-26T04:53:09.991 INFO:tasks.workunit.client.1.smithi078.stdout:3/981: unlink l8 0 2024-04-26T04:53:09.992 INFO:tasks.workunit.client.0.smithi077.stdout:4/546: dwrite d4/d4e/f8d [0,4194304] 0 2024-04-26T04:53:09.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:09 smithi077 ceph-mon[26311]: pgmap v24: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 297 MiB/s wr, 605 op/s 2024-04-26T04:53:09.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:09 smithi077 ceph-mon[26311]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:09.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:09 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/92809792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:09.994 INFO:tasks.workunit.client.1.smithi078.stdout:5/951: fsync d7/d48/d57/d65/d46/f68 0 2024-04-26T04:53:09.994 INFO:tasks.workunit.client.1.smithi078.stdout:4/997: dwrite d0/d1d/de1/d20/d31/d47/db2/d4c/d8e/fe6 [0,4194304] 0 2024-04-26T04:53:09.998 INFO:tasks.workunit.client.1.smithi078.stdout:9/912: write dc/df/f2f [1172443,73192] 0 2024-04-26T04:53:09.999 INFO:tasks.workunit.client.1.smithi078.stdout:9/913: fdatasync dc/df/d4b/dae/ff8 0 2024-04-26T04:53:10.000 INFO:tasks.workunit.client.0.smithi077.stdout:4/547: dwrite d4/d4e/f91 [0,4194304] 0 2024-04-26T04:53:10.007 INFO:tasks.workunit.client.0.smithi077.stdout:2/541: dwrite d9/dc/d1d/d34/f69 [0,4194304] 0 2024-04-26T04:53:10.015 INFO:tasks.workunit.client.0.smithi077.stdout:2/542: dwrite d9/d15/f37 [0,4194304] 0 2024-04-26T04:53:10.016 INFO:tasks.workunit.client.0.smithi077.stdout:2/543: chown d9/dc/d1d/d34/d39/d63/c99 12 1 2024-04-26T04:53:10.022 INFO:tasks.workunit.client.1.smithi078.stdout:4/998: unlink d0/de/d22/d27/f12d 0 2024-04-26T04:53:10.023 INFO:tasks.workunit.client.1.smithi078.stdout:7/937: write d9/da/d11/d13/d1a/d52/d5a/fa1 [352499,37466] 0 2024-04-26T04:53:10.028 INFO:tasks.workunit.client.1.smithi078.stdout:0/980: rename d2/d9/d60/dd5 to d2/d9/d18/d7d/d14b 0 2024-04-26T04:53:10.032 INFO:tasks.workunit.client.1.smithi078.stdout:5/952: symlink d7/d21/d138/d104/d96/df0/l143 0 2024-04-26T04:53:10.032 INFO:tasks.workunit.client.1.smithi078.stdout:5/953: read - d7/d18/d32/fbe zero size 2024-04-26T04:53:10.043 INFO:tasks.workunit.client.1.smithi078.stdout:0/981: unlink d2/d9/d18/db7/cc0 0 2024-04-26T04:53:10.044 INFO:tasks.workunit.client.1.smithi078.stdout:8/965: dwrite d0/d19/d25/f72 [0,4194304] 0 2024-04-26T04:53:10.045 INFO:tasks.workunit.client.1.smithi078.stdout:0/982: write d2/d9/d31/d73/f97 [1722019,88303] 0 2024-04-26T04:53:10.046 INFO:tasks.workunit.client.1.smithi078.stdout:5/954: mknod d7/d48/d57/d65/d46/dc4/c144 0 2024-04-26T04:53:10.047 INFO:tasks.workunit.client.1.smithi078.stdout:7/938: unlink d9/da/lc7 0 2024-04-26T04:53:10.048 INFO:tasks.workunit.client.1.smithi078.stdout:7/939: fdatasync d9/da/d33/d4b/d4c/d6c/f12a 0 2024-04-26T04:53:10.049 INFO:tasks.workunit.client.1.smithi078.stdout:7/940: chown d9/da/d33/d4b/d4c/fe5 510459 1 2024-04-26T04:53:10.051 INFO:tasks.workunit.client.1.smithi078.stdout:9/914: mknod dc/df/d27/d81/da8/c116 0 2024-04-26T04:53:10.055 INFO:tasks.workunit.client.1.smithi078.stdout:0/983: creat d2/d1e/f14c x:0 0 0 2024-04-26T04:53:10.056 INFO:tasks.workunit.client.1.smithi078.stdout:5/955: mkdir d7/d18/d32/d37/d145 0 2024-04-26T04:53:10.058 INFO:tasks.workunit.client.1.smithi078.stdout:5/956: chown d7/d21/d2f/l80 12586417 1 2024-04-26T04:53:10.058 INFO:tasks.workunit.client.1.smithi078.stdout:3/982: dwrite d16/d7e/f110 [0,4194304] 0 2024-04-26T04:53:10.060 INFO:tasks.workunit.client.1.smithi078.stdout:7/941: mknod d9/da/d33/d4b/d4c/d6c/d111/c134 0 2024-04-26T04:53:10.065 INFO:tasks.workunit.client.1.smithi078.stdout:4/999: getdents d0/de/d22/d27/d56 0 2024-04-26T04:53:10.066 INFO:tasks.workunit.client.0.smithi077.stdout:7/601: creat da/d13/fcb x:0 0 0 2024-04-26T04:53:10.066 INFO:tasks.workunit.client.1.smithi078.stdout:3/983: dwrite d16/d26/d2e/d5f/d49/f4e [4194304,4194304] 0 2024-04-26T04:53:10.069 INFO:tasks.workunit.client.1.smithi078.stdout:0/984: rename d2/d9/d18/fd to d2/d1e/d5a/da3/f14d 0 2024-04-26T04:53:10.070 INFO:tasks.workunit.client.0.smithi077.stdout:4/548: mknod d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/cb8 0 2024-04-26T04:53:10.070 INFO:tasks.workunit.client.0.smithi077.stdout:4/549: chown d4/d17/f96 4 1 2024-04-26T04:53:10.075 INFO:tasks.workunit.client.1.smithi078.stdout:0/985: dread d2/d9/d31/d4a/fbe [0,4194304] 0 2024-04-26T04:53:10.075 INFO:tasks.workunit.client.0.smithi077.stdout:4/550: dread - d4/d17/d18/d43/d53/f79 zero size 2024-04-26T04:53:10.077 INFO:tasks.workunit.client.1.smithi078.stdout:7/942: creat d9/da/d33/d4b/d4c/d5b/d83/de6/f135 x:0 0 0 2024-04-26T04:53:10.078 INFO:tasks.workunit.client.0.smithi077.stdout:7/602: mkdir da/d15/d1a/dcc 0 2024-04-26T04:53:10.079 INFO:tasks.workunit.client.1.smithi078.stdout:3/984: fsync d16/d26/f2a 0 2024-04-26T04:53:10.081 INFO:tasks.workunit.client.1.smithi078.stdout:3/985: truncate d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/f120 289008 0 2024-04-26T04:53:10.082 INFO:tasks.workunit.client.0.smithi077.stdout:7/603: creat da/d13/d22/d7d/fcd x:0 0 0 2024-04-26T04:53:10.084 INFO:tasks.workunit.client.0.smithi077.stdout:7/604: rmdir da/d6b/d63/d64 39 2024-04-26T04:53:10.085 INFO:tasks.workunit.client.1.smithi078.stdout:0/986: symlink d2/d9/d78/d84/dd7/l14e 0 2024-04-26T04:53:10.086 INFO:tasks.workunit.client.1.smithi078.stdout:3/986: dwrite d16/d3c/d81/f91 [4194304,4194304] 0 2024-04-26T04:53:10.089 INFO:tasks.workunit.client.0.smithi077.stdout:2/544: write d9/dc/d1d/f47 [4177831,3130] 0 2024-04-26T04:53:10.089 INFO:tasks.workunit.client.0.smithi077.stdout:4/551: link d4/l8f d4/d17/d18/d43/d63/d80/db6/lb9 0 2024-04-26T04:53:10.091 INFO:tasks.workunit.client.1.smithi078.stdout:5/957: creat d7/d48/d57/d65/d46/dce/d100/f146 x:0 0 0 2024-04-26T04:53:10.093 INFO:tasks.workunit.client.1.smithi078.stdout:9/915: getdents dc/df/d4b/d58 0 2024-04-26T04:53:10.094 INFO:tasks.workunit.client.1.smithi078.stdout:9/916: chown dc/df/d4b/d22/d2d/d98/fd4 1012871615 1 2024-04-26T04:53:10.096 INFO:tasks.workunit.client.0.smithi077.stdout:3/452: sync 2024-04-26T04:53:10.096 INFO:tasks.workunit.client.0.smithi077.stdout:9/519: sync 2024-04-26T04:53:10.096 INFO:tasks.workunit.client.0.smithi077.stdout:6/590: sync 2024-04-26T04:53:10.099 INFO:tasks.workunit.client.1.smithi078.stdout:5/958: symlink d7/d18/d32/d37/d79/da7/l147 0 2024-04-26T04:53:10.102 INFO:tasks.workunit.client.1.smithi078.stdout:2/997: sync 2024-04-26T04:53:10.103 INFO:tasks.workunit.client.1.smithi078.stdout:3/987: link d16/f22 d16/d3c/f153 0 2024-04-26T04:53:10.105 INFO:tasks.workunit.client.1.smithi078.stdout:2/998: dread d8/d11/d18/d5d/d61/df5/d131/f141 [0,4194304] 0 2024-04-26T04:53:10.108 INFO:tasks.workunit.client.1.smithi078.stdout:5/959: getdents d7/d18/d32/d84 0 2024-04-26T04:53:10.111 INFO:tasks.workunit.client.1.smithi078.stdout:3/988: symlink d16/l154 0 2024-04-26T04:53:10.112 INFO:tasks.workunit.client.1.smithi078.stdout:2/999: dread d8/db7/fe2 [0,4194304] 0 2024-04-26T04:53:10.116 INFO:tasks.workunit.client.1.smithi078.stdout:3/989: fdatasync d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d5a/f112 0 2024-04-26T04:53:10.118 INFO:tasks.workunit.client.1.smithi078.stdout:5/960: write d7/d11/f6a [947274,22367] 0 2024-04-26T04:53:10.118 INFO:tasks.workunit.client.1.smithi078.stdout:5/961: chown d7/da/f62 8756 1 2024-04-26T04:53:10.121 INFO:tasks.workunit.client.1.smithi078.stdout:3/990: mkdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/d98/d155 0 2024-04-26T04:53:10.123 INFO:tasks.workunit.client.1.smithi078.stdout:5/962: creat d7/da/d1e/d91/dd6/f148 x:0 0 0 2024-04-26T04:53:10.123 INFO:tasks.workunit.client.1.smithi078.stdout:3/991: stat d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d130/l12a 0 2024-04-26T04:53:10.125 INFO:tasks.workunit.client.1.smithi078.stdout:3/992: creat d16/d26/d2e/d5f/d49/d138/f156 x:0 0 0 2024-04-26T04:53:10.130 INFO:tasks.workunit.client.1.smithi078.stdout:5/963: getdents d7/d18/d32/d37/d8f/dc5 0 2024-04-26T04:53:10.130 INFO:tasks.workunit.client.1.smithi078.stdout:3/993: dread d16/d3c/d81/fde [0,4194304] 0 2024-04-26T04:53:10.132 INFO:tasks.workunit.client.1.smithi078.stdout:3/994: fdatasync d16/d26/d2e/d5f/d49/dad/db0/f117 0 2024-04-26T04:53:10.138 INFO:tasks.workunit.client.1.smithi078.stdout:5/964: write d7/d48/d57/d65/fb0 [3418514,125456] 0 2024-04-26T04:53:10.138 INFO:tasks.workunit.client.1.smithi078.stdout:3/995: link d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/cab d16/d26/d2e/d5f/d49/d138/c157 0 2024-04-26T04:53:10.140 INFO:tasks.workunit.client.1.smithi078.stdout:5/965: creat d7/d21/f149 x:0 0 0 2024-04-26T04:53:10.145 INFO:tasks.workunit.client.1.smithi078.stdout:5/966: truncate f5 586110 0 2024-04-26T04:53:10.160 INFO:tasks.workunit.client.1.smithi078.stdout:5/967: dread d7/da/d1e/d91/f102 [0,4194304] 0 2024-04-26T04:53:10.162 INFO:tasks.workunit.client.1.smithi078.stdout:5/968: creat d7/d11/dee/d12a/f14a x:0 0 0 2024-04-26T04:53:10.174 INFO:tasks.workunit.client.1.smithi078.stdout:5/969: dread d7/d48/d57/d65/fb0 [0,4194304] 0 2024-04-26T04:53:10.176 INFO:tasks.workunit.client.1.smithi078.stdout:5/970: symlink d7/d18/d32/d37/d79/l14b 0 2024-04-26T04:53:10.178 INFO:tasks.workunit.client.1.smithi078.stdout:5/971: creat d7/d18/d32/d37/d8f/f14c x:0 0 0 2024-04-26T04:53:10.180 INFO:tasks.workunit.client.1.smithi078.stdout:5/972: truncate d7/d18/d32/d37/d79/fad 9429403 0 2024-04-26T04:53:10.180 INFO:tasks.workunit.client.1.smithi078.stdout:5/973: fdatasync d7/d11/f6a 0 2024-04-26T04:53:10.181 INFO:tasks.workunit.client.1.smithi078.stdout:5/974: chown d7/d21 317537 1 2024-04-26T04:53:10.183 INFO:tasks.workunit.client.1.smithi078.stdout:5/975: getdents d7/d21/d2f/db6/d125 0 2024-04-26T04:53:10.185 INFO:tasks.workunit.client.1.smithi078.stdout:5/976: rmdir d7/d11/dee 39 2024-04-26T04:53:10.186 INFO:tasks.workunit.client.1.smithi078.stdout:5/977: symlink d7/da/d1e/l14d 0 2024-04-26T04:53:10.188 INFO:tasks.workunit.client.1.smithi078.stdout:5/978: creat d7/d11/f14e x:0 0 0 2024-04-26T04:53:10.189 INFO:tasks.workunit.client.1.smithi078.stdout:8/966: sync 2024-04-26T04:53:10.189 INFO:tasks.workunit.client.1.smithi078.stdout:5/979: fsync d7/d18/f27 0 2024-04-26T04:53:10.193 INFO:tasks.workunit.client.1.smithi078.stdout:8/967: getdents d0/d19/d52/d35/d44/df7 0 2024-04-26T04:53:10.195 INFO:tasks.workunit.client.1.smithi078.stdout:8/968: rmdir d0/d19/d52/d35/d73 39 2024-04-26T04:53:10.198 INFO:tasks.workunit.client.1.smithi078.stdout:5/980: dwrite d7/d21/d138/fde [0,4194304] 0 2024-04-26T04:53:10.199 INFO:tasks.workunit.client.1.smithi078.stdout:8/969: creat d0/d19/d52/f135 x:0 0 0 2024-04-26T04:53:10.199 INFO:tasks.workunit.client.1.smithi078.stdout:5/981: read - d7/da/d1e/ff8 zero size 2024-04-26T04:53:10.200 INFO:tasks.workunit.client.1.smithi078.stdout:8/970: write d0/d19/d52/d35/d44/d6b/f9b [577206,20477] 0 2024-04-26T04:53:10.204 INFO:tasks.workunit.client.1.smithi078.stdout:8/971: dread d0/d19/d25/d32/d8a/d92/da4/d133/d58/d5b/dee/f10a [0,4194304] 0 2024-04-26T04:53:10.206 INFO:tasks.workunit.client.0.smithi077.stdout:7/605: symlink da/lce 0 2024-04-26T04:53:10.206 INFO:tasks.workunit.client.0.smithi077.stdout:4/552: dread - d4/d17/d18/d43/d51/db4/d15/d1f/d35/f90 zero size 2024-04-26T04:53:10.210 INFO:tasks.workunit.client.0.smithi077.stdout:3/453: dread - d0/d9/d32/d2a/f4e zero size 2024-04-26T04:53:10.210 INFO:tasks.workunit.client.0.smithi077.stdout:7/606: read da/d15/d1a/d39/f7a [4063903,33622] 0 2024-04-26T04:53:10.213 INFO:tasks.workunit.client.0.smithi077.stdout:2/545: dwrite d9/d15/f93 [0,4194304] 0 2024-04-26T04:53:10.217 INFO:tasks.workunit.client.0.smithi077.stdout:7/607: fsync da/d13/d22/d5d/f9a 0 2024-04-26T04:53:10.218 INFO:tasks.workunit.client.0.smithi077.stdout:7/608: write da/f3c [6426617,69611] 0 2024-04-26T04:53:10.221 INFO:tasks.workunit.client.0.smithi077.stdout:3/454: truncate d0/d7/d1f/d4d/f5f 1665020 0 2024-04-26T04:53:10.222 INFO:tasks.workunit.client.0.smithi077.stdout:6/591: link d8/l9c d8/d52/d53/d5e/da6/lcb 0 2024-04-26T04:53:10.223 INFO:tasks.workunit.client.0.smithi077.stdout:6/592: chown d8/c1f 441546 1 2024-04-26T04:53:10.224 INFO:tasks.workunit.client.0.smithi077.stdout:7/609: creat da/d13/d22/d7d/d80/fcf x:0 0 0 2024-04-26T04:53:10.225 INFO:tasks.workunit.client.0.smithi077.stdout:4/553: write d4/f27 [1223283,107569] 0 2024-04-26T04:53:10.227 INFO:tasks.workunit.client.0.smithi077.stdout:7/610: write da/d15/d1a/fb5 [1249209,52793] 0 2024-04-26T04:53:10.228 INFO:tasks.workunit.client.0.smithi077.stdout:3/455: mkdir d0/d9/d8e 0 2024-04-26T04:53:10.230 INFO:tasks.workunit.client.0.smithi077.stdout:6/593: read d8/d55/f90 [2492285,102801] 0 2024-04-26T04:53:10.231 INFO:tasks.workunit.client.1.smithi078.stdout:5/982: dread d7/da/d1e/d91/ff6 [0,4194304] 0 2024-04-26T04:53:10.231 INFO:tasks.workunit.client.0.smithi077.stdout:9/520: dwrite d9/d3a/d79/f33 [8388608,4194304] 0 2024-04-26T04:53:10.234 INFO:tasks.workunit.client.0.smithi077.stdout:3/456: symlink d0/d9/l8f 0 2024-04-26T04:53:10.235 INFO:tasks.workunit.client.1.smithi078.stdout:5/983: creat d7/d48/d57/d65/d70/dca/f14f x:0 0 0 2024-04-26T04:53:10.235 INFO:tasks.workunit.client.0.smithi077.stdout:3/457: fsync d0/f3d 0 2024-04-26T04:53:10.237 INFO:tasks.workunit.client.0.smithi077.stdout:4/554: truncate d4/d17/d18/d43/d51/db4/d15/f50 161427 0 2024-04-26T04:53:10.238 INFO:tasks.workunit.client.0.smithi077.stdout:2/546: dwrite d9/dc/d1d/d34/d39/d63/d79/f22 [0,4194304] 0 2024-04-26T04:53:10.239 INFO:tasks.workunit.client.1.smithi078.stdout:5/984: mkdir d7/d21/d2f/d90/d150 0 2024-04-26T04:53:10.242 INFO:tasks.workunit.client.0.smithi077.stdout:9/521: rename d9/dd/f5d to d9/d3a/d79/d95/fb8 0 2024-04-26T04:53:10.243 INFO:tasks.workunit.client.0.smithi077.stdout:4/555: mkdir d4/d4e/dad/d8b/dba 0 2024-04-26T04:53:10.244 INFO:tasks.workunit.client.0.smithi077.stdout:2/547: creat d9/dc/d1d/d34/d39/d63/d79/d5b/fb6 x:0 0 0 2024-04-26T04:53:10.246 INFO:tasks.workunit.client.0.smithi077.stdout:7/611: rmdir da/d6b/d63/d64/dc4 0 2024-04-26T04:53:10.247 INFO:tasks.workunit.client.0.smithi077.stdout:2/548: write d9/dc/d1d/d34/d39/d63/d79/d5b/fb6 [580020,108862] 0 2024-04-26T04:53:10.247 INFO:tasks.workunit.client.1.smithi078.stdout:5/985: dwrite d7/da/d1e/d91/dd6/f11a [0,4194304] 0 2024-04-26T04:53:10.247 INFO:tasks.workunit.client.0.smithi077.stdout:7/612: fdatasync da/d6b/d63/d64/fbc 0 2024-04-26T04:53:10.249 INFO:tasks.workunit.client.0.smithi077.stdout:4/556: fdatasync d4/d17/d18/d43/d51/db4/d15/d1f/f32 0 2024-04-26T04:53:10.250 INFO:tasks.workunit.client.0.smithi077.stdout:3/458: creat d0/d7/f90 x:0 0 0 2024-04-26T04:53:10.252 INFO:tasks.workunit.client.0.smithi077.stdout:9/522: mknod d9/cb9 0 2024-04-26T04:53:10.252 INFO:tasks.workunit.client.0.smithi077.stdout:4/557: read d4/d17/d18/d43/d51/db4/faf [632760,62660] 0 2024-04-26T04:53:10.252 INFO:tasks.workunit.client.0.smithi077.stdout:9/523: stat d9/d43 0 2024-04-26T04:53:10.256 INFO:tasks.workunit.client.0.smithi077.stdout:7/613: unlink da/d13/d22/d7d/fcd 0 2024-04-26T04:53:10.258 INFO:tasks.workunit.client.0.smithi077.stdout:9/524: fsync d9/d35/f69 0 2024-04-26T04:53:10.261 INFO:tasks.workunit.client.0.smithi077.stdout:4/558: creat d4/d4e/dad/d8b/dba/fbb x:0 0 0 2024-04-26T04:53:10.262 INFO:tasks.workunit.client.0.smithi077.stdout:4/559: read d4/f27 [707963,19860] 0 2024-04-26T04:53:10.264 INFO:tasks.workunit.client.0.smithi077.stdout:4/560: write d4/d17/d18/d43/d51/db4/faf [530016,18160] 0 2024-04-26T04:53:10.266 INFO:tasks.workunit.client.0.smithi077.stdout:4/561: stat d4/d17/d18/d43/d51/db4/d15/d1f/c71 0 2024-04-26T04:53:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:09 smithi078 ceph-mon[33746]: pgmap v24: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 297 MiB/s wr, 605 op/s 2024-04-26T04:53:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:09 smithi078 ceph-mon[33746]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:10.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:09 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/92809792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:10.272 INFO:tasks.workunit.client.1.smithi078.stdout:5/986: dread d7/d48/d57/d65/f3a [0,4194304] 0 2024-04-26T04:53:10.277 INFO:tasks.workunit.client.1.smithi078.stdout:5/987: dread d7/d18/d1c/d8e/fcd [0,4194304] 0 2024-04-26T04:53:10.278 INFO:tasks.workunit.client.1.smithi078.stdout:5/988: rmdir d7/d21/d2f/db6/d125 39 2024-04-26T04:53:10.278 INFO:tasks.workunit.client.1.smithi078.stdout:5/989: chown d7/d48/d57/d65/c97 131 1 2024-04-26T04:53:10.280 INFO:tasks.workunit.client.1.smithi078.stdout:5/990: mkdir d7/da/d1e/d91/dd6/d151 0 2024-04-26T04:53:10.282 INFO:tasks.workunit.client.1.smithi078.stdout:5/991: creat d7/d48/da9/f152 x:0 0 0 2024-04-26T04:53:10.282 INFO:tasks.workunit.client.1.smithi078.stdout:5/992: write d7/da/f93 [3976150,33126] 0 2024-04-26T04:53:10.284 INFO:tasks.workunit.client.1.smithi078.stdout:5/993: mknod d7/d48/d57/d65/d70/c153 0 2024-04-26T04:53:10.286 INFO:tasks.workunit.client.1.smithi078.stdout:5/994: mknod d7/d21/d2f/d90/d130/c154 0 2024-04-26T04:53:10.288 INFO:tasks.workunit.client.1.smithi078.stdout:5/995: mknod d7/d18/d32/d37/d8f/dc5/c155 0 2024-04-26T04:53:10.289 INFO:tasks.workunit.client.1.smithi078.stdout:5/996: mkdir d7/d18/d32/d84/d156 0 2024-04-26T04:53:10.292 INFO:tasks.workunit.client.1.smithi078.stdout:5/997: rename d7/d21/c58 to d7/d48/da9/deb/dfe/c157 0 2024-04-26T04:53:10.293 INFO:tasks.workunit.client.1.smithi078.stdout:5/998: mknod d7/c158 0 2024-04-26T04:53:10.298 INFO:tasks.workunit.client.1.smithi078.stdout:5/999: dwrite d7/d48/da9/d10f/f128 [0,4194304] 0 2024-04-26T04:53:10.310 INFO:tasks.workunit.client.0.smithi077.stdout:4/562: link d4/d17/d18/d43/d51/db4/d15/d1f/c71 d4/d4e/dad/cbc 0 2024-04-26T04:53:10.313 INFO:tasks.workunit.client.1.smithi078.stdout:7/943: sync 2024-04-26T04:53:10.315 INFO:tasks.workunit.client.0.smithi077.stdout:2/549: dwrite d9/dc/d1d/d1e/f21 [4194304,4194304] 0 2024-04-26T04:53:10.316 INFO:tasks.workunit.client.0.smithi077.stdout:9/525: dwrite d9/dd/d1b/d2b/d32/d36/d7f/f85 [0,4194304] 0 2024-04-26T04:53:10.317 INFO:tasks.workunit.client.0.smithi077.stdout:2/550: chown d9/f3d 650 1 2024-04-26T04:53:10.319 INFO:tasks.workunit.client.1.smithi078.stdout:7/944: dwrite d9/da/d33/d4b/d4c/d5b/ff0 [0,4194304] 0 2024-04-26T04:53:10.320 INFO:tasks.workunit.client.0.smithi077.stdout:3/459: dread d0/d9/d32/d2a/f4c [0,4194304] 0 2024-04-26T04:53:10.320 INFO:tasks.workunit.client.0.smithi077.stdout:4/563: creat d4/d17/d18/d43/d63/d80/fbd x:0 0 0 2024-04-26T04:53:10.322 INFO:tasks.workunit.client.0.smithi077.stdout:2/551: creat d9/d15/d44/d4e/fb7 x:0 0 0 2024-04-26T04:53:10.322 INFO:tasks.workunit.client.0.smithi077.stdout:3/460: read - d0/d7/d1f/d76/f8d zero size 2024-04-26T04:53:10.326 INFO:tasks.workunit.client.0.smithi077.stdout:7/614: dread da/d6b/d63/f81 [0,4194304] 0 2024-04-26T04:53:10.326 INFO:tasks.workunit.client.0.smithi077.stdout:9/526: rename d9/dd/d1b/d2b/d32/laa to d9/dd/d1b/d4e/lba 0 2024-04-26T04:53:10.328 INFO:tasks.workunit.client.0.smithi077.stdout:7/615: write da/d15/d42/f77 [4877189,2676] 0 2024-04-26T04:53:10.334 INFO:tasks.workunit.client.0.smithi077.stdout:4/564: unlink d4/d17/d18/d43/d53/d56/l37 0 2024-04-26T04:53:10.336 INFO:tasks.workunit.client.0.smithi077.stdout:9/527: dwrite d9/dd/d3d/f9e [0,4194304] 0 2024-04-26T04:53:10.337 INFO:tasks.workunit.client.1.smithi078.stdout:7/945: dwrite d9/da/d33/fd2 [0,4194304] 0 2024-04-26T04:53:10.338 INFO:tasks.workunit.client.0.smithi077.stdout:9/528: read d9/dd/d1b/fab [42643,107039] 0 2024-04-26T04:53:10.339 INFO:tasks.workunit.client.0.smithi077.stdout:4/565: rename d4/f14 to d4/d17/d18/d43/d53/fbe 0 2024-04-26T04:53:10.339 INFO:tasks.workunit.client.0.smithi077.stdout:7/616: dwrite da/d15/d1a/f79 [0,4194304] 0 2024-04-26T04:53:10.342 INFO:tasks.workunit.client.1.smithi078.stdout:7/946: write d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc2 [1784108,119827] 0 2024-04-26T04:53:10.346 INFO:tasks.workunit.client.0.smithi077.stdout:4/566: symlink d4/lbf 0 2024-04-26T04:53:10.351 INFO:tasks.workunit.client.0.smithi077.stdout:9/529: link d9/c81 d9/d3a/d79/d95/dad/db2/cbb 0 2024-04-26T04:53:10.353 INFO:tasks.workunit.client.0.smithi077.stdout:7/617: link da/d6b/fa4 da/d13/fd0 0 2024-04-26T04:53:10.353 INFO:tasks.workunit.client.1.smithi078.stdout:7/947: mkdir d9/da/d11/d13/d1a/d52/d5a/d136 0 2024-04-26T04:53:10.357 INFO:tasks.workunit.client.0.smithi077.stdout:7/618: mknod da/d6b/d63/d64/cd1 0 2024-04-26T04:53:10.358 INFO:tasks.workunit.client.0.smithi077.stdout:3/461: write d0/f1b [1149457,64589] 0 2024-04-26T04:53:10.359 INFO:tasks.workunit.client.0.smithi077.stdout:4/567: dwrite d4/d17/d18/d43/d63/fb3 [0,4194304] 0 2024-04-26T04:53:10.360 INFO:tasks.workunit.client.0.smithi077.stdout:2/552: dread d9/dc/d1d/d34/f84 [0,4194304] 0 2024-04-26T04:53:10.363 INFO:tasks.workunit.client.0.smithi077.stdout:9/530: dread d9/d3a/d79/f6b [0,4194304] 0 2024-04-26T04:53:10.363 INFO:tasks.workunit.client.0.smithi077.stdout:7/619: stat da/d15/d1a/d39/cc5 0 2024-04-26T04:53:10.364 INFO:tasks.workunit.client.0.smithi077.stdout:4/568: chown d4/d4e/dad/cbc 931905 1 2024-04-26T04:53:10.365 INFO:tasks.workunit.client.0.smithi077.stdout:3/462: symlink d0/d7/l91 0 2024-04-26T04:53:10.366 INFO:tasks.workunit.client.0.smithi077.stdout:2/553: mkdir d9/dc/d1d/d34/d39/d63/d79/d5b/db8 0 2024-04-26T04:53:10.368 INFO:tasks.workunit.client.0.smithi077.stdout:9/531: symlink d9/d35/d99/lbc 0 2024-04-26T04:53:10.369 INFO:tasks.workunit.client.0.smithi077.stdout:7/620: creat da/d13/d22/fd2 x:0 0 0 2024-04-26T04:53:10.371 INFO:tasks.workunit.client.0.smithi077.stdout:3/463: creat d0/d7/d1f/d4d/f92 x:0 0 0 2024-04-26T04:53:10.373 INFO:tasks.workunit.client.0.smithi077.stdout:9/532: creat d9/d3a/d79/d50/fbd x:0 0 0 2024-04-26T04:53:10.377 INFO:tasks.workunit.client.1.smithi078.stdout:9/917: sync 2024-04-26T04:53:10.377 INFO:tasks.workunit.client.1.smithi078.stdout:0/987: sync 2024-04-26T04:53:10.377 INFO:tasks.workunit.client.1.smithi078.stdout:3/996: sync 2024-04-26T04:53:10.378 INFO:tasks.workunit.client.0.smithi077.stdout:3/464: truncate d0/d7/d1f/f60 645522 0 2024-04-26T04:53:10.380 INFO:tasks.workunit.client.0.smithi077.stdout:3/465: write d0/f3d [541603,30171] 0 2024-04-26T04:53:10.382 INFO:tasks.workunit.client.1.smithi078.stdout:0/988: creat d2/d9/d60/dae/de4/f14f x:0 0 0 2024-04-26T04:53:10.385 INFO:tasks.workunit.client.1.smithi078.stdout:9/918: fdatasync dc/df/d27/d81/f87 0 2024-04-26T04:53:10.387 INFO:tasks.workunit.client.1.smithi078.stdout:9/919: chown dc/d55/de2/fef 264269916 1 2024-04-26T04:53:10.389 INFO:tasks.workunit.client.0.smithi077.stdout:3/466: write d0/d9/f30 [8635829,59843] 0 2024-04-26T04:53:10.389 INFO:tasks.workunit.client.1.smithi078.stdout:0/989: dwrite d2/d9/d31/df2/f11e [0,4194304] 0 2024-04-26T04:53:10.392 INFO:tasks.workunit.client.0.smithi077.stdout:7/621: rename da/f36 to da/d13/fd3 0 2024-04-26T04:53:10.400 INFO:tasks.workunit.client.1.smithi078.stdout:7/948: truncate d9/da/d33/d4b/d4c/d6c/fcf 647093 0 2024-04-26T04:53:10.400 INFO:tasks.workunit.client.0.smithi077.stdout:7/622: symlink da/d6b/d63/d64/db0/ld4 0 2024-04-26T04:53:10.401 INFO:tasks.workunit.client.0.smithi077.stdout:7/623: chown da/d15/d1a/d39/faf 50 1 2024-04-26T04:53:10.403 INFO:tasks.workunit.client.1.smithi078.stdout:3/997: rmdir d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/d12e/d31/d71/d8a/d98/d14b 0 2024-04-26T04:53:10.406 INFO:tasks.workunit.client.0.smithi077.stdout:7/624: getdents da/d13/d22/d5d/d9f/da7 0 2024-04-26T04:53:10.407 INFO:tasks.workunit.client.0.smithi077.stdout:7/625: fsync da/d13/fcb 0 2024-04-26T04:53:10.408 INFO:tasks.workunit.client.1.smithi078.stdout:0/990: symlink d2/d9/d18/db7/d92/l150 0 2024-04-26T04:53:10.409 INFO:tasks.workunit.client.0.smithi077.stdout:7/626: creat da/d13/d22/d5d/d9f/fd5 x:0 0 0 2024-04-26T04:53:10.411 INFO:tasks.workunit.client.1.smithi078.stdout:0/991: creat d2/d9/d31/d4a/d11c/f151 x:0 0 0 2024-04-26T04:53:10.411 INFO:tasks.workunit.client.1.smithi078.stdout:7/949: getdents d9/da/d11/d13/d1a/d52/d5a/d41/d68/d121 0 2024-04-26T04:53:10.412 INFO:tasks.workunit.client.1.smithi078.stdout:7/950: stat d9/da/d20/l64 0 2024-04-26T04:53:10.412 INFO:tasks.workunit.client.1.smithi078.stdout:0/992: write d2/d9/d18/db7/d92/f141 [2083943,82688] 0 2024-04-26T04:53:10.413 INFO:tasks.workunit.client.0.smithi077.stdout:2/554: write d9/dc/d1d/d1e/f5e [593828,41141] 0 2024-04-26T04:53:10.415 INFO:tasks.workunit.client.1.smithi078.stdout:7/951: fsync d9/da/d11/d13/d1a/dd4/fdf 0 2024-04-26T04:53:10.416 INFO:tasks.workunit.client.1.smithi078.stdout:0/993: creat d2/d9/d60/dae/de4/f152 x:0 0 0 2024-04-26T04:53:10.419 INFO:tasks.workunit.client.0.smithi077.stdout:4/569: dwrite d4/d17/f57 [0,4194304] 0 2024-04-26T04:53:10.423 INFO:tasks.workunit.client.0.smithi077.stdout:9/533: dwrite d9/d45/f7e [0,4194304] 0 2024-04-26T04:53:10.423 INFO:tasks.workunit.client.1.smithi078.stdout:7/952: rename d9/da/d11/d13/d1a/d52/d5a/d41/d68/ldd to d9/da/d11/d13/d50/d8a/l137 0 2024-04-26T04:53:10.425 INFO:tasks.workunit.client.1.smithi078.stdout:0/994: mknod d2/d9/d31/d73/c153 0 2024-04-26T04:53:10.426 INFO:tasks.workunit.client.0.smithi077.stdout:3/467: dread d0/d9/d32/d29/f31 [4194304,4194304] 0 2024-04-26T04:53:10.428 INFO:tasks.workunit.client.0.smithi077.stdout:2/555: unlink d9/dc/d1d/d34/d39/d63/d6a/c70 0 2024-04-26T04:53:10.428 INFO:tasks.workunit.client.1.smithi078.stdout:9/920: write dc/f60 [1443155,127969] 0 2024-04-26T04:53:10.430 INFO:tasks.workunit.client.0.smithi077.stdout:4/570: chown d4/d17/d18/d43/d51/cb0 391870576 1 2024-04-26T04:53:10.431 INFO:tasks.workunit.client.1.smithi078.stdout:8/972: sync 2024-04-26T04:53:10.432 INFO:tasks.workunit.client.0.smithi077.stdout:7/627: dread da/d15/f44 [0,4194304] 0 2024-04-26T04:53:10.435 INFO:tasks.workunit.client.0.smithi077.stdout:7/628: read da/d13/f19 [973813,100596] 0 2024-04-26T04:53:10.438 INFO:tasks.workunit.client.1.smithi078.stdout:7/953: dread d9/da/d33/d4b/d4c/d5b/dc3/d128/f132 [0,4194304] 0 2024-04-26T04:53:10.439 INFO:tasks.workunit.client.1.smithi078.stdout:8/973: chown d0/d19/d25/c75 203035556 1 2024-04-26T04:53:10.440 INFO:tasks.workunit.client.0.smithi077.stdout:2/556: mkdir d9/dc/d5f/d8e/db9 0 2024-04-26T04:53:10.441 INFO:tasks.workunit.client.1.smithi078.stdout:3/998: dwrite d16/d3c/d81/fd7 [0,4194304] 0 2024-04-26T04:53:10.441 INFO:tasks.workunit.client.0.smithi077.stdout:2/557: chown d9/fe 440177 1 2024-04-26T04:53:10.442 INFO:tasks.workunit.client.1.smithi078.stdout:8/974: chown d0/d19/d25/d32/d8a/d92/da4/d133/d58/l121 59033 1 2024-04-26T04:53:10.444 INFO:tasks.workunit.client.1.smithi078.stdout:9/921: fdatasync dc/df/d4b/d22/d2a/d57/f90 0 2024-04-26T04:53:10.444 INFO:tasks.workunit.client.1.smithi078.stdout:3/999: write d16/d26/d2e/d5f/d49/dad/db0/db7/dc1/dfb/d56/f7d [1600314,4130] 0 2024-04-26T04:53:10.445 INFO:tasks.workunit.client.1.smithi078.stdout:9/922: write dc/f60 [2212479,8903] 0 2024-04-26T04:53:10.448 INFO:tasks.workunit.client.0.smithi077.stdout:3/468: mkdir d0/d9/d32/d93 0 2024-04-26T04:53:10.448 INFO:tasks.workunit.client.0.smithi077.stdout:3/469: readlink d0/d7/l24 0 2024-04-26T04:53:10.449 INFO:tasks.workunit.client.1.smithi078.stdout:8/975: stat d0/d19/d52/d95/cbd 0 2024-04-26T04:53:10.450 INFO:tasks.workunit.client.1.smithi078.stdout:8/976: fsync d0/d19/d25/d32/d8a/d92/da4/da7/fda 0 2024-04-26T04:53:10.450 INFO:tasks.workunit.client.0.smithi077.stdout:2/558: symlink d9/dc/d5f/d8e/lba 0 2024-04-26T04:53:10.456 INFO:tasks.workunit.client.1.smithi078.stdout:0/995: dwrite d2/d9/d31/d73/ff3 [0,4194304] 0 2024-04-26T04:53:10.456 INFO:tasks.workunit.client.1.smithi078.stdout:9/923: unlink dc/df/d4b/d22/d2a/d57/d93/f110 0 2024-04-26T04:53:10.480 INFO:tasks.workunit.client.0.smithi077.stdout:8/639: sync 2024-04-26T04:53:10.480 INFO:tasks.workunit.client.0.smithi077.stdout:7/629: mknod da/d13/d22/d5d/d9f/da7/cd6 0 2024-04-26T04:53:10.481 INFO:tasks.workunit.client.0.smithi077.stdout:4/571: write d4/d17/d18/d43/d51/db4/d15/f6d [833275,64715] 0 2024-04-26T04:53:10.483 INFO:tasks.workunit.client.1.smithi078.stdout:7/954: rename d9/da/d33/d4b/d4c/d5b/d83/da5/d113 to d9/da/d11/d13/d138 0 2024-04-26T04:53:10.484 INFO:tasks.workunit.client.0.smithi077.stdout:3/470: dwrite d0/d9/f8b [0,4194304] 0 2024-04-26T04:53:10.485 INFO:tasks.workunit.client.0.smithi077.stdout:9/534: dread d9/d3a/d79/d95/fb8 [0,4194304] 0 2024-04-26T04:53:10.485 INFO:tasks.workunit.client.1.smithi078.stdout:0/996: stat d2/d9/d78/deb/ced 0 2024-04-26T04:53:10.486 INFO:tasks.workunit.client.1.smithi078.stdout:9/924: truncate dc/df/d4b/f1c 583416 0 2024-04-26T04:53:10.488 INFO:tasks.workunit.client.0.smithi077.stdout:2/559: rename d9/f72 to d9/fbb 0 2024-04-26T04:53:10.490 INFO:tasks.workunit.client.0.smithi077.stdout:7/630: read da/d13/d22/d34/f5a [4290184,119194] 0 2024-04-26T04:53:10.490 INFO:tasks.workunit.client.0.smithi077.stdout:2/560: truncate d9/dc/d5f/f62 525924 0 2024-04-26T04:53:10.491 INFO:tasks.workunit.client.1.smithi078.stdout:8/977: rename d0/d19/d52/d35/d44/f5d to d0/d19/d52/d5c/da2/d103/f136 0 2024-04-26T04:53:10.492 INFO:tasks.workunit.client.0.smithi077.stdout:4/572: dwrite d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:10.493 INFO:tasks.workunit.client.1.smithi078.stdout:9/925: dwrite dc/df/d8a/fea [0,4194304] 0 2024-04-26T04:53:10.493 INFO:tasks.workunit.client.0.smithi077.stdout:7/631: write da/d13/d22/fd2 [957361,22312] 0 2024-04-26T04:53:10.494 INFO:tasks.workunit.client.1.smithi078.stdout:7/955: creat d9/da/d33/f139 x:0 0 0 2024-04-26T04:53:10.498 INFO:tasks.workunit.client.1.smithi078.stdout:0/997: rmdir d2/d9/d60/d6a/da5 39 2024-04-26T04:53:10.498 INFO:tasks.workunit.client.1.smithi078.stdout:9/926: chown dc/df/d4b/d22/d2a/d103/ff0 1233373 1 2024-04-26T04:53:10.498 INFO:tasks.workunit.client.1.smithi078.stdout:7/956: truncate d9/da/d33/fb0 5545 0 2024-04-26T04:53:10.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/471: read d0/d7/d15/f44 [636697,87070] 0 2024-04-26T04:53:10.500 INFO:tasks.workunit.client.0.smithi077.stdout:8/640: symlink d4/df/d22/ld9 0 2024-04-26T04:53:10.501 INFO:tasks.workunit.client.1.smithi078.stdout:9/927: write dc/f39 [1823280,54684] 0 2024-04-26T04:53:10.501 INFO:tasks.workunit.client.1.smithi078.stdout:8/978: dwrite d0/d19/d52/d35/d44/f26 [0,4194304] 0 2024-04-26T04:53:10.510 INFO:tasks.workunit.client.0.smithi077.stdout:4/573: rename d4/d17/d18/d43/d51/db4/d15/d1f/d35/c54 to d4/d17/d18/d43/d51/db4/d15/d1f/cc0 0 2024-04-26T04:53:10.511 INFO:tasks.workunit.client.0.smithi077.stdout:4/574: stat d4/f8 0 2024-04-26T04:53:10.511 INFO:tasks.workunit.client.1.smithi078.stdout:0/998: dwrite d2/d9/d31/d4a/d11c/f151 [0,4194304] 0 2024-04-26T04:53:10.514 INFO:tasks.workunit.client.0.smithi077.stdout:9/535: symlink d9/d3a/d79/d95/dad/db2/lbe 0 2024-04-26T04:53:10.515 INFO:tasks.workunit.client.0.smithi077.stdout:2/561: creat d9/dc/d5f/d8e/db9/fbc x:0 0 0 2024-04-26T04:53:10.517 INFO:tasks.workunit.client.0.smithi077.stdout:7/632: mkdir da/d15/d1a/dd7 0 2024-04-26T04:53:10.518 INFO:tasks.workunit.client.0.smithi077.stdout:3/472: getdents d0/d9/d32/d93 0 2024-04-26T04:53:10.519 INFO:tasks.workunit.client.1.smithi078.stdout:9/928: mknod dc/df/d4b/d22/d2a/c117 0 2024-04-26T04:53:10.520 INFO:tasks.workunit.client.0.smithi077.stdout:4/575: fsync d4/d17/d18/d43/d6e/f84 0 2024-04-26T04:53:10.523 INFO:tasks.workunit.client.1.smithi078.stdout:7/957: getdents d9/da/d33/d4b/dc0 0 2024-04-26T04:53:10.523 INFO:tasks.workunit.client.0.smithi077.stdout:9/536: creat d9/d7d/fbf x:0 0 0 2024-04-26T04:53:10.524 INFO:tasks.workunit.client.1.smithi078.stdout:7/958: dread - d9/da/d11/d13/d1a/d65/f114 zero size 2024-04-26T04:53:10.527 INFO:tasks.workunit.client.1.smithi078.stdout:8/979: link d0/d19/d52/d5c/fb3 d0/d7a/d83/df3/f137 0 2024-04-26T04:53:10.529 INFO:tasks.workunit.client.1.smithi078.stdout:7/959: rmdir d9/da/d33/d4b/d4c/d5b/dc3 39 2024-04-26T04:53:10.530 INFO:tasks.workunit.client.1.smithi078.stdout:7/960: chown d9/da/d33/d4b/d4c/d6c 15115411 1 2024-04-26T04:53:10.535 INFO:tasks.workunit.client.1.smithi078.stdout:8/980: mkdir d0/d19/d52/d5c/da2/d138 0 2024-04-26T04:53:10.536 INFO:tasks.workunit.client.1.smithi078.stdout:7/961: mkdir d9/da/d33/d4b/d4c/d6c/d111/d13a 0 2024-04-26T04:53:10.538 INFO:tasks.workunit.client.1.smithi078.stdout:8/981: creat d0/d19/d52/d5c/da2/dd3/f139 x:0 0 0 2024-04-26T04:53:10.543 INFO:tasks.workunit.client.1.smithi078.stdout:7/962: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc2 [0,4194304] 0 2024-04-26T04:53:10.543 INFO:tasks.workunit.client.0.smithi077.stdout:2/562: getdents d9/dc/d6e 0 2024-04-26T04:53:10.544 INFO:tasks.workunit.client.1.smithi078.stdout:8/982: creat d0/d19/d52/d5c/da2/d138/f13a x:0 0 0 2024-04-26T04:53:10.544 INFO:tasks.workunit.client.0.smithi077.stdout:2/563: write d9/dc/d5f/d78/f98 [330565,32705] 0 2024-04-26T04:53:10.547 INFO:tasks.workunit.client.1.smithi078.stdout:7/963: unlink d9/da/d11/d13/d1a/d52/cae 0 2024-04-26T04:53:10.549 INFO:tasks.workunit.client.0.smithi077.stdout:4/576: creat d4/d17/d18/d43/d51/db4/d15/d1f/d35/fc1 x:0 0 0 2024-04-26T04:53:10.551 INFO:tasks.workunit.client.1.smithi078.stdout:8/983: creat d0/d19/d52/d5c/da2/f13b x:0 0 0 2024-04-26T04:53:10.551 INFO:tasks.workunit.client.0.smithi077.stdout:8/641: write d4/df/d22/d2d/f1b [585266,124428] 0 2024-04-26T04:53:10.551 INFO:tasks.workunit.client.1.smithi078.stdout:8/984: dread - d0/d19/d52/d5c/da2/f13b zero size 2024-04-26T04:53:10.552 INFO:tasks.workunit.client.0.smithi077.stdout:2/564: mkdir d9/dc/d5f/d8e/da6/dbd 0 2024-04-26T04:53:10.552 INFO:tasks.workunit.client.1.smithi078.stdout:8/985: chown d0/d19/d25/d32/d8a/f123 217986 1 2024-04-26T04:53:10.555 INFO:tasks.workunit.client.1.smithi078.stdout:7/964: mknod d9/da/d33/d4b/d4c/d5b/d83/dee/c13b 0 2024-04-26T04:53:10.555 INFO:tasks.workunit.client.0.smithi077.stdout:7/633: write da/d15/d42/f50 [175169,104175] 0 2024-04-26T04:53:10.556 INFO:tasks.workunit.client.0.smithi077.stdout:3/473: write d0/d9/d32/d2a/f40 [1850371,116075] 0 2024-04-26T04:53:10.558 INFO:tasks.workunit.client.1.smithi078.stdout:0/999: dwrite d2/d9/d18/f49 [4194304,4194304] 0 2024-04-26T04:53:10.558 INFO:tasks.workunit.client.1.smithi078.stdout:8/986: fsync d0/d19/d52/d35/f4e 0 2024-04-26T04:53:10.559 INFO:tasks.workunit.client.0.smithi077.stdout:8/642: dwrite d4/df/dce/d5b/fa0 [0,4194304] 0 2024-04-26T04:53:10.560 INFO:tasks.workunit.client.1.smithi078.stdout:9/929: dwrite dc/df/d27/f76 [0,4194304] 0 2024-04-26T04:53:10.560 INFO:tasks.workunit.client.1.smithi078.stdout:8/987: readlink d0/d19/d25/d32/d8a/d92/da4/da7/lfd 0 2024-04-26T04:53:10.561 INFO:tasks.workunit.client.0.smithi077.stdout:4/577: mknod d4/d17/d18/d43/d63/d80/db6/cc2 0 2024-04-26T04:53:10.562 INFO:tasks.workunit.client.1.smithi078.stdout:7/965: fdatasync d9/da/d33/f70 0 2024-04-26T04:53:10.568 INFO:tasks.workunit.client.0.smithi077.stdout:9/537: truncate d9/dd/d1b/f73 753701 0 2024-04-26T04:53:10.569 INFO:tasks.workunit.client.0.smithi077.stdout:9/538: chown d9/d3a/d79/d50 768090634 1 2024-04-26T04:53:10.570 INFO:tasks.workunit.client.0.smithi077.stdout:3/474: fdatasync d0/d7/f70 0 2024-04-26T04:53:10.571 INFO:tasks.workunit.client.0.smithi077.stdout:3/475: chown d0/d7/d15/f65 162036 1 2024-04-26T04:53:10.575 INFO:tasks.workunit.client.0.smithi077.stdout:8/643: creat d4/df/d22/d2d/d2b/d33/fda x:0 0 0 2024-04-26T04:53:10.579 INFO:tasks.workunit.client.0.smithi077.stdout:9/539: dwrite d9/d45/f71 [0,4194304] 0 2024-04-26T04:53:10.583 INFO:tasks.workunit.client.0.smithi077.stdout:4/578: unlink d4/d4e/dad/cbc 0 2024-04-26T04:53:10.585 INFO:tasks.workunit.client.0.smithi077.stdout:8/644: dwrite d4/df/d18/fb0 [0,4194304] 0 2024-04-26T04:53:10.586 INFO:tasks.workunit.client.1.smithi078.stdout:7/966: mknod d9/da/d11/d13/d1a/d52/d5a/d41/d68/d121/c13c 0 2024-04-26T04:53:10.591 INFO:tasks.workunit.client.1.smithi078.stdout:7/967: chown d9/da/d33/d4b/d4c/d6c/d109/l10d 7819640 1 2024-04-26T04:53:10.592 INFO:tasks.workunit.client.1.smithi078.stdout:7/968: write d9/da/d11/d13/d1a/d52/d5a/f26 [1360505,71721] 0 2024-04-26T04:53:10.595 INFO:tasks.workunit.client.0.smithi077.stdout:7/634: dwrite da/f3c [0,4194304] 0 2024-04-26T04:53:10.596 INFO:tasks.workunit.client.0.smithi077.stdout:2/565: rename d9/dc/d5f/d78/f98 to d9/dc/d1d/d34/d39/fbe 0 2024-04-26T04:53:10.596 INFO:tasks.workunit.client.0.smithi077.stdout:7/635: fsync da/d13/d22/fd2 0 2024-04-26T04:53:10.599 INFO:tasks.workunit.client.0.smithi077.stdout:4/579: mknod d4/d4e/dad/d8b/dba/cc3 0 2024-04-26T04:53:10.603 INFO:tasks.workunit.client.0.smithi077.stdout:8/645: rename d4/f13 to d4/d69/d73/fdb 0 2024-04-26T04:53:10.604 INFO:tasks.workunit.client.0.smithi077.stdout:9/540: creat d9/d45/db1/fc0 x:0 0 0 2024-04-26T04:53:10.605 INFO:tasks.workunit.client.0.smithi077.stdout:9/541: chown d9/d3a/d79/f6b 220929 1 2024-04-26T04:53:10.606 INFO:tasks.workunit.client.0.smithi077.stdout:4/580: mknod d4/d17/d18/d43/d6e/cc4 0 2024-04-26T04:53:10.607 INFO:tasks.workunit.client.0.smithi077.stdout:3/476: creat d0/f94 x:0 0 0 2024-04-26T04:53:10.608 INFO:tasks.workunit.client.0.smithi077.stdout:8/646: mkdir d4/d17/ddc 0 2024-04-26T04:53:10.609 INFO:tasks.workunit.client.0.smithi077.stdout:2/566: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/db8/cbf 0 2024-04-26T04:53:10.611 INFO:tasks.workunit.client.0.smithi077.stdout:4/581: dread d4/d17/d18/d43/d63/fb3 [0,4194304] 0 2024-04-26T04:53:10.611 INFO:tasks.workunit.client.0.smithi077.stdout:3/477: mknod d0/d7/d15/c95 0 2024-04-26T04:53:10.615 INFO:tasks.workunit.client.0.smithi077.stdout:2/567: rename d9/dc/d1d/d34/d39/d63/f6c to d9/dc/d1d/d1e/fc0 0 2024-04-26T04:53:10.617 INFO:tasks.workunit.client.0.smithi077.stdout:3/478: chown d0/l39 0 1 2024-04-26T04:53:10.619 INFO:tasks.workunit.client.0.smithi077.stdout:2/568: rename d9/dc to d9/dc/d5f/d8e/da6/dc1 22 2024-04-26T04:53:10.619 INFO:tasks.workunit.client.1.smithi078.stdout:7/969: dread d9/da/d11/d13/d1a/d52/d5a/d28/d6d/d9e/fa7 [0,4194304] 0 2024-04-26T04:53:10.620 INFO:tasks.workunit.client.0.smithi077.stdout:8/647: dwrite d4/f9 [0,4194304] 0 2024-04-26T04:53:10.621 INFO:tasks.workunit.client.0.smithi077.stdout:4/582: dwrite d4/d17/f57 [0,4194304] 0 2024-04-26T04:53:10.622 INFO:tasks.workunit.client.1.smithi078.stdout:7/970: rename d9/da/d11/d13/d50/d8a to d9/da/d11/d13/d138/d13d 0 2024-04-26T04:53:10.624 INFO:tasks.workunit.client.1.smithi078.stdout:9/930: dwrite dc/df/d4b/d22/d2a/d57/fa4 [0,4194304] 0 2024-04-26T04:53:10.624 INFO:tasks.workunit.client.0.smithi077.stdout:3/479: symlink d0/d9/l96 0 2024-04-26T04:53:10.625 INFO:tasks.workunit.client.1.smithi078.stdout:7/971: write d9/da/d11/d13/d1a/d52/d5a/f44 [12483482,96151] 0 2024-04-26T04:53:10.629 INFO:tasks.workunit.client.0.smithi077.stdout:2/569: mkdir d9/dc/d1d/d34/d39/d63/d79/dc2 0 2024-04-26T04:53:10.633 INFO:tasks.workunit.client.0.smithi077.stdout:8/648: creat d4/d17/d90/fdd x:0 0 0 2024-04-26T04:53:10.635 INFO:tasks.workunit.client.0.smithi077.stdout:3/480: dwrite d0/f87 [0,4194304] 0 2024-04-26T04:53:10.636 INFO:tasks.workunit.client.1.smithi078.stdout:9/931: unlink dc/dd9/de1/l10c 0 2024-04-26T04:53:10.636 INFO:tasks.workunit.client.1.smithi078.stdout:7/972: dwrite d9/da/d33/d4b/d4c/f55 [0,4194304] 0 2024-04-26T04:53:10.637 INFO:tasks.workunit.client.1.smithi078.stdout:8/988: dwrite d0/d19/d52/d35/d44/d6b/f120 [0,4194304] 0 2024-04-26T04:53:10.638 INFO:tasks.workunit.client.0.smithi077.stdout:3/481: fdatasync d0/d7/d15/f3f 0 2024-04-26T04:53:10.638 INFO:tasks.workunit.client.0.smithi077.stdout:9/542: dread d9/dd/d1b/d2b/d32/d36/f6d [0,4194304] 0 2024-04-26T04:53:10.639 INFO:tasks.workunit.client.1.smithi078.stdout:8/989: chown d0/d19/d25/c75 2914 1 2024-04-26T04:53:10.640 INFO:tasks.workunit.client.0.smithi077.stdout:3/482: chown d0/d7/d1f/d4d 76 1 2024-04-26T04:53:10.641 INFO:tasks.workunit.client.0.smithi077.stdout:7/636: dread da/d6b/d63/d64/f83 [0,4194304] 0 2024-04-26T04:53:10.642 INFO:tasks.workunit.client.0.smithi077.stdout:8/649: mkdir d4/df/d6f/da3/dc7/dde 0 2024-04-26T04:53:10.646 INFO:tasks.workunit.client.0.smithi077.stdout:9/543: dwrite d9/d45/f7e [0,4194304] 0 2024-04-26T04:53:10.647 INFO:tasks.workunit.client.1.smithi078.stdout:9/932: mkdir dc/df/d4b/d22/d2a/d103/ddb/d118 0 2024-04-26T04:53:10.649 INFO:tasks.workunit.client.1.smithi078.stdout:7/973: mkdir d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/d13e 0 2024-04-26T04:53:10.652 INFO:tasks.workunit.client.1.smithi078.stdout:8/990: mknod d0/d19/d25/d32/d8a/d92/da4/d133/d58/d5b/d61/c13c 0 2024-04-26T04:53:10.652 INFO:tasks.workunit.client.0.smithi077.stdout:9/544: dread d9/d35/d99/fa3 [0,4194304] 0 2024-04-26T04:53:10.652 INFO:tasks.workunit.client.0.smithi077.stdout:9/545: stat d9/d45/f7e 0 2024-04-26T04:53:10.653 INFO:tasks.workunit.client.1.smithi078.stdout:8/991: chown d0/d19/d25/d32/d8a/d92/da4/d133/d58/d5b/dee 202 1 2024-04-26T04:53:10.667 INFO:tasks.workunit.client.0.smithi077.stdout:3/483: creat d0/d9/d32/d93/f97 x:0 0 0 2024-04-26T04:53:10.669 INFO:tasks.workunit.client.0.smithi077.stdout:8/650: mknod d4/cdf 0 2024-04-26T04:53:10.673 INFO:tasks.workunit.client.0.smithi077.stdout:2/570: rename d9/dc/d1d/d34/d39/d6b to d9/dc/d1d/d34/d39/d63/d6a/dc3 0 2024-04-26T04:53:10.673 INFO:tasks.workunit.client.0.smithi077.stdout:3/484: dwrite d0/d9/f79 [4194304,4194304] 0 2024-04-26T04:53:10.676 INFO:tasks.workunit.client.0.smithi077.stdout:9/546: mknod d9/d3a/d79/cc1 0 2024-04-26T04:53:10.677 INFO:tasks.workunit.client.0.smithi077.stdout:9/547: readlink d9/d3a/d79/d50/l5c 0 2024-04-26T04:53:10.680 INFO:tasks.workunit.client.0.smithi077.stdout:8/651: rename d4/d17/l28 to d4/df/d18/le0 0 2024-04-26T04:53:10.681 INFO:tasks.workunit.client.0.smithi077.stdout:8/652: write d4/df/dce/fcb [255732,118948] 0 2024-04-26T04:53:10.681 INFO:tasks.workunit.client.0.smithi077.stdout:3/485: mknod d0/d7/d1f/d4d/c98 0 2024-04-26T04:53:10.683 INFO:tasks.workunit.client.0.smithi077.stdout:1/503: sync 2024-04-26T04:53:10.683 INFO:tasks.workunit.client.0.smithi077.stdout:0/488: sync 2024-04-26T04:53:10.683 INFO:tasks.workunit.client.0.smithi077.stdout:2/571: mkdir d9/dc/d1d/d34/d39/d63/d6a/dc3/dc4 0 2024-04-26T04:53:10.684 INFO:tasks.workunit.client.0.smithi077.stdout:0/489: dread - d0/d47/d68/fa5 zero size 2024-04-26T04:53:10.685 INFO:tasks.workunit.client.0.smithi077.stdout:9/548: truncate d9/dd/d1b/fab 201390 0 2024-04-26T04:53:10.687 INFO:tasks.workunit.client.0.smithi077.stdout:8/653: creat d4/df/d22/d2d/d2b/d33/fe1 x:0 0 0 2024-04-26T04:53:10.689 INFO:tasks.workunit.client.0.smithi077.stdout:3/486: creat d0/d7/d1f/d4d/f99 x:0 0 0 2024-04-26T04:53:10.693 INFO:tasks.workunit.client.0.smithi077.stdout:2/572: dwrite d9/dc/d1d/d34/d39/d63/d79/f30 [0,4194304] 0 2024-04-26T04:53:10.695 INFO:tasks.workunit.client.0.smithi077.stdout:0/490: unlink d0/d47/d64/c4a 0 2024-04-26T04:53:10.699 INFO:tasks.workunit.client.0.smithi077.stdout:9/549: write d9/dd/d1b/d2b/d32/d36/f6d [5621840,32994] 0 2024-04-26T04:53:10.701 INFO:tasks.workunit.client.0.smithi077.stdout:9/550: truncate d9/d45/db1/fc0 876761 0 2024-04-26T04:53:10.702 INFO:tasks.workunit.client.0.smithi077.stdout:9/551: chown d9/dd/d1b/f66 6431 1 2024-04-26T04:53:10.703 INFO:tasks.workunit.client.0.smithi077.stdout:4/583: truncate d4/d17/d18/d43/d51/db4/d15/f6d 791341 0 2024-04-26T04:53:10.706 INFO:tasks.workunit.client.0.smithi077.stdout:3/487: unlink d0/f3d 0 2024-04-26T04:53:10.708 INFO:tasks.workunit.client.0.smithi077.stdout:2/573: mknod d9/dc/d1d/d34/d39/d63/d6a/dc3/cc5 0 2024-04-26T04:53:10.709 INFO:tasks.workunit.client.0.smithi077.stdout:2/574: chown d9/dc/d1d/d34/d39/d63/d79/f30 6267229 1 2024-04-26T04:53:10.710 INFO:tasks.workunit.client.0.smithi077.stdout:2/575: stat d9/dc/d1d/d34/d39/d63/d79/c20 0 2024-04-26T04:53:10.712 INFO:tasks.workunit.client.0.smithi077.stdout:2/576: write d9/dc/d1d/d1e/f9b [430268,4818] 0 2024-04-26T04:53:10.714 INFO:tasks.workunit.client.0.smithi077.stdout:9/552: write d9/dd/fb3 [2986365,113972] 0 2024-04-26T04:53:10.716 INFO:tasks.workunit.client.0.smithi077.stdout:1/504: dread d9/db/d2b/d3f/f52 [0,4194304] 0 2024-04-26T04:53:10.722 INFO:tasks.workunit.client.0.smithi077.stdout:7/637: write da/d6b/f93 [3417711,63718] 0 2024-04-26T04:53:10.723 INFO:tasks.workunit.client.0.smithi077.stdout:2/577: mkdir d9/dc/d1d/d34/d39/d63/d6a/dc3/dc6 0 2024-04-26T04:53:10.724 INFO:tasks.workunit.client.0.smithi077.stdout:7/638: truncate da/d13/fcb 270702 0 2024-04-26T04:53:10.724 INFO:tasks.workunit.client.0.smithi077.stdout:7/639: chown da/d13/c9d 30869 1 2024-04-26T04:53:10.730 INFO:tasks.workunit.client.0.smithi077.stdout:4/584: mkdir d4/d17/d18/d43/d6e/d6c/dae/dc5 0 2024-04-26T04:53:10.732 INFO:tasks.workunit.client.0.smithi077.stdout:5/557: sync 2024-04-26T04:53:10.732 INFO:tasks.workunit.client.0.smithi077.stdout:6/594: sync 2024-04-26T04:53:10.735 INFO:tasks.workunit.client.0.smithi077.stdout:2/578: creat d9/dc/d1d/d1e/d77/fc7 x:0 0 0 2024-04-26T04:53:10.736 INFO:tasks.workunit.client.0.smithi077.stdout:5/558: dread d1/d12/f60 [0,4194304] 0 2024-04-26T04:53:10.742 INFO:tasks.workunit.client.0.smithi077.stdout:4/585: fsync d4/d17/d18/d43/d53/d56/f2d 0 2024-04-26T04:53:10.746 INFO:tasks.workunit.client.0.smithi077.stdout:5/559: rmdir d1/d8c 39 2024-04-26T04:53:10.751 INFO:tasks.workunit.client.0.smithi077.stdout:8/654: write d4/df/d18/f26 [1376918,127974] 0 2024-04-26T04:53:10.752 INFO:tasks.workunit.client.0.smithi077.stdout:5/560: mknod d1/d8d/da7/cca 0 2024-04-26T04:53:10.753 INFO:tasks.workunit.client.0.smithi077.stdout:8/655: write d4/df/d22/d75/fac [1037634,8741] 0 2024-04-26T04:53:10.757 INFO:tasks.workunit.client.1.smithi078.stdout:8/992: creat d0/d7a/f13d x:0 0 0 2024-04-26T04:53:10.758 INFO:tasks.workunit.client.1.smithi078.stdout:8/993: fdatasync d0/d19/d25/d32/d8a/f57 0 2024-04-26T04:53:10.760 INFO:tasks.workunit.client.0.smithi077.stdout:8/656: dread d4/df/dce/d5b/f8b [0,4194304] 0 2024-04-26T04:53:10.761 INFO:tasks.workunit.client.0.smithi077.stdout:8/657: write d4/f9 [4047579,81406] 0 2024-04-26T04:53:10.761 INFO:tasks.workunit.client.0.smithi077.stdout:5/561: fsync d1/d42/d62/fc4 0 2024-04-26T04:53:10.762 INFO:tasks.workunit.client.0.smithi077.stdout:8/658: chown d4/d69/fa2 153 1 2024-04-26T04:53:10.764 INFO:tasks.workunit.client.0.smithi077.stdout:7/640: getdents da/d13/d22/d7d 0 2024-04-26T04:53:10.764 INFO:tasks.workunit.client.0.smithi077.stdout:8/659: stat d4/df/d18/f74 0 2024-04-26T04:53:10.766 INFO:tasks.workunit.client.1.smithi078.stdout:8/994: mknod d0/d19/d52/d5c/da2/d138/c13e 0 2024-04-26T04:53:10.767 INFO:tasks.workunit.client.0.smithi077.stdout:7/641: write da/d6b/d63/d64/fbc [956280,26531] 0 2024-04-26T04:53:10.767 INFO:tasks.workunit.client.0.smithi077.stdout:8/660: write d4/df/dce/f70 [1005492,103771] 0 2024-04-26T04:53:10.768 INFO:tasks.workunit.client.0.smithi077.stdout:0/491: dwrite d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:10.769 INFO:tasks.workunit.client.1.smithi078.stdout:9/933: creat dc/df/f119 x:0 0 0 2024-04-26T04:53:10.770 INFO:tasks.workunit.client.0.smithi077.stdout:8/661: chown d4/df/d22/c3b 21817609 1 2024-04-26T04:53:10.770 INFO:tasks.workunit.client.0.smithi077.stdout:7/642: stat da/d15/d1a/d39/d96/fbf 0 2024-04-26T04:53:10.771 INFO:tasks.workunit.client.0.smithi077.stdout:8/662: write d4/df/dce/d5b/fa5 [730938,46668] 0 2024-04-26T04:53:10.772 INFO:tasks.workunit.client.0.smithi077.stdout:8/663: write d4/df/d22/d2d/d2b/d33/fa7 [22784,23220] 0 2024-04-26T04:53:10.776 INFO:tasks.workunit.client.0.smithi077.stdout:7/643: dread da/d15/d1a/fb5 [0,4194304] 0 2024-04-26T04:53:10.776 INFO:tasks.workunit.client.1.smithi078.stdout:8/995: mknod d0/d19/d52/d95/c13f 0 2024-04-26T04:53:10.777 INFO:tasks.workunit.client.0.smithi077.stdout:7/644: write da/d13/f30 [1831128,59844] 0 2024-04-26T04:53:10.783 INFO:tasks.workunit.client.1.smithi078.stdout:8/996: creat d0/d19/d52/d35/d44/d6b/dc3/f140 x:0 0 0 2024-04-26T04:53:10.783 INFO:tasks.workunit.client.0.smithi077.stdout:9/553: dwrite d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:53:10.786 INFO:tasks.workunit.client.0.smithi077.stdout:3/488: truncate d0/fb 9617356 0 2024-04-26T04:53:10.787 INFO:tasks.workunit.client.1.smithi078.stdout:8/997: read d0/d19/d52/d35/d44/d6b/fd5 [3816449,126025] 0 2024-04-26T04:53:10.790 INFO:tasks.workunit.client.0.smithi077.stdout:9/554: dread d9/f8d [0,4194304] 0 2024-04-26T04:53:10.794 INFO:tasks.workunit.client.0.smithi077.stdout:4/586: getdents d4/d17/d18/d43/d51/db4/d15 0 2024-04-26T04:53:10.796 INFO:tasks.workunit.client.1.smithi078.stdout:8/998: creat d0/d19/d52/d35/d44/d6b/dce/f141 x:0 0 0 2024-04-26T04:53:10.797 INFO:tasks.workunit.client.0.smithi077.stdout:0/492: readlink d0/d28/l72 0 2024-04-26T04:53:10.799 INFO:tasks.workunit.client.0.smithi077.stdout:1/505: dwrite d9/d53/d6e/f87 [4194304,4194304] 0 2024-04-26T04:53:10.799 INFO:tasks.workunit.client.0.smithi077.stdout:6/595: write d8/d52/d53/d5e/fab [617904,122427] 0 2024-04-26T04:53:10.799 INFO:tasks.workunit.client.1.smithi078.stdout:8/999: dread - d0/d19/d52/d35/d44/df7/f10d zero size 2024-04-26T04:53:10.802 INFO:tasks.workunit.client.0.smithi077.stdout:6/596: dread d8/d2a/d86/f94 [0,4194304] 0 2024-04-26T04:53:10.802 INFO:tasks.workunit.client.0.smithi077.stdout:7/645: read - da/d15/d1a/d39/f71 zero size 2024-04-26T04:53:10.807 INFO:tasks.workunit.client.0.smithi077.stdout:4/587: mknod d4/d17/d18/d43/d51/db4/cc6 0 2024-04-26T04:53:10.807 INFO:tasks.workunit.client.0.smithi077.stdout:6/597: dwrite d8/d1c/f79 [0,4194304] 0 2024-04-26T04:53:10.809 INFO:tasks.workunit.client.0.smithi077.stdout:4/588: chown d4/d17/d18/d43/ca4 27978585 1 2024-04-26T04:53:10.812 INFO:tasks.workunit.client.1.smithi078.stdout:7/974: dwrite d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fc1 [4194304,4194304] 0 2024-04-26T04:53:10.813 INFO:tasks.workunit.client.1.smithi078.stdout:7/975: dread d9/da/d33/f56 [0,4194304] 0 2024-04-26T04:53:10.821 INFO:tasks.workunit.client.1.smithi078.stdout:9/934: write dc/df/d27/f5c [469104,93230] 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:7/976: dread d9/da/d33/d4b/d4c/d5b/dc3/d128/f132 [0,4194304] 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:9/935: truncate dc/df/f59 2114574 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:9/936: creat dc/df/d4b/d58/f11a x:0 0 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:7/977: truncate d9/da/d11/d13/d1a/f29 354181 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:9/937: dwrite dc/df/f119 [0,4194304] 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:9/938: stat dc/df/d27/d81/c6c 0 2024-04-26T04:53:10.868 INFO:tasks.workunit.client.1.smithi078.stdout:9/939: rename dc/df/c107 to dc/dd9/c11b 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:6/598: creat d8/d52/d53/fcc x:0 0 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:5/562: write d1/d8/d72/fa2 [50836,85522] 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:4/589: symlink d4/d17/d18/d43/d51/db4/lc7 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:4/590: chown d4/d17/d18/l31 30384 1 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:9/555: creat d9/dd/d1b/d2b/d32/fc2 x:0 0 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:9/556: chown d9/d35/ca6 77187452 1 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:6/599: mknod d8/d52/d53/da7/ccd 0 2024-04-26T04:53:10.869 INFO:tasks.workunit.client.0.smithi077.stdout:0/493: write d0/d28/d40/f85 [252902,18385] 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:0/494: readlink d0/l1d 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:1/506: rename d9/d53/d7d to d9/d90 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:8/664: link d4/df/d22/d80/d86/cd2 d4/df/dc6/ce2 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:6/600: dwrite d8/d8f/f9f [0,4194304] 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:8/665: chown d4/df/d22/d2d/d2b/d33/dbc 1013119411 1 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:3/489: truncate d0/d7/d1f/f4a 1482330 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:0/495: dread d0/d47/f50 [0,4194304] 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:7/646: rename da/d13/d22/f27 to da/d15/d1a/d39/fd8 0 2024-04-26T04:53:10.870 INFO:tasks.workunit.client.0.smithi077.stdout:1/507: creat d9/db/d2b/d75/f91 x:0 0 0 2024-04-26T04:53:10.871 INFO:tasks.workunit.client.0.smithi077.stdout:8/666: mkdir d4/df/d22/d80/d86/de3 0 2024-04-26T04:53:10.871 INFO:tasks.workunit.client.0.smithi077.stdout:4/591: link d4/d17/d18/d43/d51/cb0 d4/d4e/dad/d8b/dba/cc8 0 2024-04-26T04:53:10.871 INFO:tasks.workunit.client.0.smithi077.stdout:3/490: dread d0/d7/f33 [0,4194304] 0 2024-04-26T04:53:10.874 INFO:tasks.workunit.client.0.smithi077.stdout:5/563: rename d1/d8/d72/f99 to d1/d12/d87/fcb 0 2024-04-26T04:53:10.875 INFO:tasks.workunit.client.0.smithi077.stdout:5/564: chown d1/d42/da4/d25/d31/lbf 326 1 2024-04-26T04:53:10.876 INFO:tasks.workunit.client.0.smithi077.stdout:7/647: creat da/d6b/d63/d64/fd9 x:0 0 0 2024-04-26T04:53:10.877 INFO:tasks.workunit.client.0.smithi077.stdout:1/508: creat d9/db/dd/d34/d4a/f92 x:0 0 0 2024-04-26T04:53:10.879 INFO:tasks.workunit.client.0.smithi077.stdout:6/601: truncate d8/f88 369967 0 2024-04-26T04:53:10.881 INFO:tasks.workunit.client.0.smithi077.stdout:3/491: readlink d0/d7/l24 0 2024-04-26T04:53:10.881 INFO:tasks.workunit.client.0.smithi077.stdout:9/557: truncate d9/dd/d3d/f9e 1583261 0 2024-04-26T04:53:10.882 INFO:tasks.workunit.client.1.smithi078.stdout:9/940: dwrite dc/df/d4b/d22/d2a/d57/d69/f96 [0,4194304] 0 2024-04-26T04:53:10.884 INFO:tasks.workunit.client.0.smithi077.stdout:7/648: dread da/d13/f30 [0,4194304] 0 2024-04-26T04:53:10.884 INFO:tasks.workunit.client.0.smithi077.stdout:6/602: symlink d8/d1c/da4/lce 0 2024-04-26T04:53:10.885 INFO:tasks.workunit.client.0.smithi077.stdout:6/603: fsync d8/d52/d53/fcc 0 2024-04-26T04:53:10.886 INFO:tasks.workunit.client.0.smithi077.stdout:7/649: truncate da/d15/f2f 2931423 0 2024-04-26T04:53:10.887 INFO:tasks.workunit.client.0.smithi077.stdout:9/558: chown d9/dd/d1b/d2b/ca0 633 1 2024-04-26T04:53:10.889 INFO:tasks.workunit.client.0.smithi077.stdout:8/667: rmdir d4/df/d18/d56/dc0 0 2024-04-26T04:53:10.890 INFO:tasks.workunit.client.0.smithi077.stdout:3/492: mkdir d0/d7/d9a 0 2024-04-26T04:53:10.891 INFO:tasks.workunit.client.0.smithi077.stdout:6/604: symlink d8/d8f/lcf 0 2024-04-26T04:53:10.893 INFO:tasks.workunit.client.0.smithi077.stdout:9/559: readlink d9/dd/d1b/d2b/d32/d36/l3f 0 2024-04-26T04:53:10.895 INFO:tasks.workunit.client.0.smithi077.stdout:0/496: rename d0/d19/d1e/d59/d5a/c7a to d0/d19/d1e/cad 0 2024-04-26T04:53:10.898 INFO:tasks.workunit.client.0.smithi077.stdout:4/592: link d4/laa d4/d17/d18/d43/d51/db4/d15/d1f/lc9 0 2024-04-26T04:53:10.899 INFO:tasks.workunit.client.0.smithi077.stdout:4/593: write d4/f8 [7508104,65840] 0 2024-04-26T04:53:10.900 INFO:tasks.workunit.client.0.smithi077.stdout:9/560: creat d9/d7d/d94/fc3 x:0 0 0 2024-04-26T04:53:10.901 INFO:tasks.workunit.client.0.smithi077.stdout:5/565: rename d1/d42/cc5 to d1/d42/da4/d84/ccc 0 2024-04-26T04:53:10.903 INFO:tasks.workunit.client.0.smithi077.stdout:0/497: unlink d0/d19/d1e/d82/l91 0 2024-04-26T04:53:10.907 INFO:tasks.workunit.client.0.smithi077.stdout:4/594: symlink d4/d17/lca 0 2024-04-26T04:53:10.908 INFO:tasks.workunit.client.0.smithi077.stdout:9/561: mkdir d9/d45/dc4 0 2024-04-26T04:53:10.911 INFO:tasks.workunit.client.0.smithi077.stdout:1/509: rename d9/db/d2b/d5c to d9/d53/d8b/d93 0 2024-04-26T04:53:10.913 INFO:tasks.workunit.client.0.smithi077.stdout:5/566: symlink d1/d12/d87/lcd 0 2024-04-26T04:53:10.913 INFO:tasks.workunit.client.1.smithi078.stdout:9/941: truncate dc/df/d27/f9b 1819291 0 2024-04-26T04:53:10.917 INFO:tasks.workunit.client.0.smithi077.stdout:7/650: write da/ff [551096,80050] 0 2024-04-26T04:53:10.919 INFO:tasks.workunit.client.0.smithi077.stdout:5/567: dwrite d1/d42/da4/d25/d31/f92 [0,4194304] 0 2024-04-26T04:53:10.919 INFO:tasks.workunit.client.0.smithi077.stdout:7/651: fsync da/d13/d22/f99 0 2024-04-26T04:53:10.920 INFO:tasks.workunit.client.0.smithi077.stdout:3/493: rename d0/d9/d32/f36 to d0/d7/d1f/d4d/f9b 0 2024-04-26T04:53:10.926 INFO:tasks.workunit.client.1.smithi078.stdout:9/942: creat dc/df/d37/f11c x:0 0 0 2024-04-26T04:53:10.928 INFO:tasks.workunit.client.0.smithi077.stdout:8/668: dwrite d4/f5 [0,4194304] 0 2024-04-26T04:53:10.931 INFO:tasks.workunit.client.0.smithi077.stdout:4/595: creat d4/d17/d18/d43/d51/db4/d15/d1f/d25/fcb x:0 0 0 2024-04-26T04:53:10.931 INFO:tasks.workunit.client.0.smithi077.stdout:6/605: write d8/f88 [1022536,21702] 0 2024-04-26T04:53:10.932 INFO:tasks.workunit.client.0.smithi077.stdout:9/562: symlink d9/dd/d1b/d2b/d32/lc5 0 2024-04-26T04:53:10.933 INFO:tasks.workunit.client.0.smithi077.stdout:9/563: fsync d9/f2d 0 2024-04-26T04:53:10.938 INFO:tasks.workunit.client.0.smithi077.stdout:5/568: fdatasync d1/d42/da4/d25/d31/d3c/f8e 0 2024-04-26T04:53:10.939 INFO:tasks.workunit.client.0.smithi077.stdout:9/564: dread d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:53:10.940 INFO:tasks.workunit.client.0.smithi077.stdout:9/565: stat d9/dd/d1b/d2b/d32/c6f 0 2024-04-26T04:53:10.946 INFO:tasks.workunit.client.0.smithi077.stdout:8/669: rename d4/fe to d4/df/d22/d80/fe4 0 2024-04-26T04:53:10.947 INFO:tasks.workunit.client.0.smithi077.stdout:6/606: fdatasync d8/f6d 0 2024-04-26T04:53:10.948 INFO:tasks.workunit.client.0.smithi077.stdout:8/670: chown d4/df/dce/la4 27409 1 2024-04-26T04:53:10.950 INFO:tasks.workunit.client.0.smithi077.stdout:0/498: dwrite d0/d47/ff [0,4194304] 0 2024-04-26T04:53:10.957 INFO:tasks.workunit.client.0.smithi077.stdout:1/510: link d9/db/dd/l17 d9/db/dd/d34/d63/l94 0 2024-04-26T04:53:10.959 INFO:tasks.workunit.client.0.smithi077.stdout:0/499: dwrite d0/d2c/f88 [0,4194304] 0 2024-04-26T04:53:10.959 INFO:tasks.workunit.client.0.smithi077.stdout:1/511: dread - d9/d53/d8b/f8e zero size 2024-04-26T04:53:10.960 INFO:tasks.workunit.client.1.smithi078.stdout:9/943: dwrite dc/df/d4b/d3e/f5d [0,4194304] 0 2024-04-26T04:53:10.961 INFO:tasks.workunit.client.0.smithi077.stdout:0/500: truncate d0/d2c/d4e/f66 20396 0 2024-04-26T04:53:10.968 INFO:tasks.workunit.client.0.smithi077.stdout:6/607: creat d8/d52/d53/d5e/da6/fd0 x:0 0 0 2024-04-26T04:53:10.969 INFO:tasks.workunit.client.0.smithi077.stdout:3/494: write d0/d7/d15/f44 [1482377,92095] 0 2024-04-26T04:53:10.969 INFO:tasks.workunit.client.0.smithi077.stdout:8/671: creat d4/d69/d73/fe5 x:0 0 0 2024-04-26T04:53:10.970 INFO:tasks.workunit.client.0.smithi077.stdout:9/566: write d9/f8c [1003192,36747] 0 2024-04-26T04:53:10.970 INFO:tasks.workunit.client.1.smithi078.stdout:9/944: dread dc/df/d4b/d22/d2a/d57/d69/fa0 [0,4194304] 0 2024-04-26T04:53:10.971 INFO:tasks.workunit.client.0.smithi077.stdout:7/652: truncate da/d6b/d63/d64/fbc 339996 0 2024-04-26T04:53:10.972 INFO:tasks.workunit.client.0.smithi077.stdout:8/672: write d4/df/fd6 [303860,43183] 0 2024-04-26T04:53:10.975 INFO:tasks.workunit.client.0.smithi077.stdout:5/569: dwrite d1/f5 [0,4194304] 0 2024-04-26T04:53:10.976 INFO:tasks.workunit.client.0.smithi077.stdout:1/512: unlink d9/d53/d8b/d93/l67 0 2024-04-26T04:53:10.977 INFO:tasks.workunit.client.0.smithi077.stdout:4/596: truncate d4/d17/d18/d43/f74 986446 0 2024-04-26T04:53:10.993 INFO:tasks.workunit.client.1.smithi078.stdout:9/945: creat dc/df/d4b/d3e/f11d x:0 0 0 2024-04-26T04:53:10.994 INFO:tasks.workunit.client.0.smithi077.stdout:6/608: mkdir d8/d52/d53/d5e/da6/dd1 0 2024-04-26T04:53:10.995 INFO:tasks.workunit.client.0.smithi077.stdout:7/653: rename da/d15/d1a/d39/fd8 to da/d15/dc6/fda 0 2024-04-26T04:53:10.996 INFO:tasks.workunit.client.0.smithi077.stdout:1/513: creat d9/db/dd/d68/d81/f95 x:0 0 0 2024-04-26T04:53:10.997 INFO:tasks.workunit.client.1.smithi078.stdout:9/946: mknod dc/dd9/c11e 0 2024-04-26T04:53:10.998 INFO:tasks.workunit.client.0.smithi077.stdout:7/654: write da/d15/f94 [396299,122963] 0 2024-04-26T04:53:11.000 INFO:tasks.workunit.client.0.smithi077.stdout:8/673: dwrite d4/df/d18/d56/fa6 [0,4194304] 0 2024-04-26T04:53:11.009 INFO:tasks.workunit.client.1.smithi078.stdout:7/978: sync 2024-04-26T04:53:11.020 INFO:tasks.workunit.client.0.smithi077.stdout:8/674: dread d4/d17/f6a [0,4194304] 0 2024-04-26T04:53:11.021 INFO:tasks.workunit.client.0.smithi077.stdout:8/675: dread - d4/df/d22/d2d/d2b/d33/fda zero size 2024-04-26T04:53:11.027 INFO:tasks.workunit.client.0.smithi077.stdout:5/570: chown d1/d30/d35/l54 33628 1 2024-04-26T04:53:11.028 INFO:tasks.workunit.client.1.smithi078.stdout:9/947: write dc/df/d4b/d22/d2d/f108 [8924660,47866] 0 2024-04-26T04:53:11.034 INFO:tasks.workunit.client.1.smithi078.stdout:9/948: rename dc/df/d4b/d22/d2a/d57/d69/db3/fd6 to dc/df/d4b/d22/dbe/f11f 0 2024-04-26T04:53:11.035 INFO:tasks.workunit.client.0.smithi077.stdout:7/655: stat da/d13/d22/c74 0 2024-04-26T04:53:11.039 INFO:tasks.workunit.client.0.smithi077.stdout:1/514: rename d9/d90/f7e to d9/db/dd/f96 0 2024-04-26T04:53:11.039 INFO:tasks.workunit.client.0.smithi077.stdout:1/515: chown d9/db/dd/l6a 4 1 2024-04-26T04:53:11.041 INFO:tasks.workunit.client.1.smithi078.stdout:9/949: creat dc/df/d4b/d22/d2a/d57/f120 x:0 0 0 2024-04-26T04:53:11.041 INFO:tasks.workunit.client.1.smithi078.stdout:9/950: readlink dc/d55/l9c 0 2024-04-26T04:53:11.042 INFO:tasks.workunit.client.1.smithi078.stdout:9/951: stat dc/df/d4b/d22/d2a/d103/lf7 0 2024-04-26T04:53:11.042 INFO:tasks.workunit.client.0.smithi077.stdout:4/597: fsync d4/d17/d18/d43/d51/db4/d15/d1f/d76/f87 0 2024-04-26T04:53:11.043 INFO:tasks.workunit.client.1.smithi078.stdout:9/952: readlink dc/df/d27/dbc/lc3 0 2024-04-26T04:53:11.044 INFO:tasks.workunit.client.0.smithi077.stdout:7/656: symlink da/d15/ldb 0 2024-04-26T04:53:11.046 INFO:tasks.workunit.client.1.smithi078.stdout:7/979: truncate d9/da/d11/d13/d1a/d52/d5a/f44 11960408 0 2024-04-26T04:53:11.047 INFO:tasks.workunit.client.0.smithi077.stdout:1/516: creat d9/db/d2b/d75/f97 x:0 0 0 2024-04-26T04:53:11.048 INFO:tasks.workunit.client.1.smithi078.stdout:9/953: symlink dc/df/d4b/d22/d2a/d103/ddb/d118/l121 0 2024-04-26T04:53:11.048 INFO:tasks.workunit.client.0.smithi077.stdout:1/517: write d9/d90/f83 [556339,39864] 0 2024-04-26T04:53:11.049 INFO:tasks.workunit.client.1.smithi078.stdout:9/954: write dc/df/d27/f76 [3577310,94798] 0 2024-04-26T04:53:11.049 INFO:tasks.workunit.client.0.smithi077.stdout:9/567: getdents d9/d3a 0 2024-04-26T04:53:11.050 INFO:tasks.workunit.client.0.smithi077.stdout:9/568: fsync d9/d35/d99/fa3 0 2024-04-26T04:53:11.051 INFO:tasks.workunit.client.1.smithi078.stdout:7/980: fsync d9/da/d11/d13/d50/fc8 0 2024-04-26T04:53:11.054 INFO:tasks.workunit.client.0.smithi077.stdout:4/598: creat d4/d17/d18/d43/d63/d80/fcc x:0 0 0 2024-04-26T04:53:11.055 INFO:tasks.workunit.client.0.smithi077.stdout:0/501: write d0/d2c/f2e [3234115,112052] 0 2024-04-26T04:53:11.058 INFO:tasks.workunit.client.0.smithi077.stdout:7/657: creat da/d13/d22/d5d/dbd/fdc x:0 0 0 2024-04-26T04:53:11.059 INFO:tasks.workunit.client.1.smithi078.stdout:9/955: dwrite dc/df/d4b/d22/d2a/d57/d69/db5/fb8 [0,4194304] 0 2024-04-26T04:53:11.060 INFO:tasks.workunit.client.1.smithi078.stdout:7/981: symlink d9/dfd/l13f 0 2024-04-26T04:53:11.065 INFO:tasks.workunit.client.0.smithi077.stdout:6/609: dwrite d8/f3f [0,4194304] 0 2024-04-26T04:53:11.065 INFO:tasks.workunit.client.0.smithi077.stdout:0/502: dwrite d0/d2c/d4e/f66 [0,4194304] 0 2024-04-26T04:53:11.065 INFO:tasks.workunit.client.1.smithi078.stdout:9/956: unlink dc/df/d4b/f79 0 2024-04-26T04:53:11.066 INFO:tasks.workunit.client.0.smithi077.stdout:6/610: chown d8/d1c/c44 5 1 2024-04-26T04:53:11.068 INFO:tasks.workunit.client.0.smithi077.stdout:1/518: truncate f6 4185223 0 2024-04-26T04:53:11.073 INFO:tasks.workunit.client.1.smithi078.stdout:9/957: mkdir dc/df/d8a/d122 0 2024-04-26T04:53:11.076 INFO:tasks.workunit.client.1.smithi078.stdout:7/982: dread d9/da/d33/d4b/d4c/d5b/d83/fac [0,4194304] 0 2024-04-26T04:53:11.079 INFO:tasks.workunit.client.0.smithi077.stdout:5/571: rename d1/fa5 to d1/d12/fce 0 2024-04-26T04:53:11.081 INFO:tasks.workunit.client.1.smithi078.stdout:9/958: dwrite dc/df/d4b/d22/d2a/d57/d69/f115 [0,4194304] 0 2024-04-26T04:53:11.083 INFO:tasks.workunit.client.0.smithi077.stdout:0/503: mknod d0/d19/d69/cae 0 2024-04-26T04:53:11.083 INFO:tasks.workunit.client.0.smithi077.stdout:2/579: sync 2024-04-26T04:53:11.084 INFO:tasks.workunit.client.0.smithi077.stdout:6/611: creat d8/d52/d53/da7/fd2 x:0 0 0 2024-04-26T04:53:11.084 INFO:tasks.workunit.client.0.smithi077.stdout:6/612: chown d8/f50 27098511 1 2024-04-26T04:53:11.085 INFO:tasks.workunit.client.0.smithi077.stdout:1/519: symlink d9/db/d2b/d75/l98 0 2024-04-26T04:53:11.086 INFO:tasks.workunit.client.0.smithi077.stdout:6/613: chown d8/d2a/lb4 831 1 2024-04-26T04:53:11.086 INFO:tasks.workunit.client.1.smithi078.stdout:7/983: creat d9/da/d11/d13/f140 x:0 0 0 2024-04-26T04:53:11.087 INFO:tasks.workunit.client.0.smithi077.stdout:6/614: chown d8/d52/d53/fc7 76 1 2024-04-26T04:53:11.091 INFO:tasks.workunit.client.0.smithi077.stdout:6/615: read d8/d52/da0/f77 [3905792,85881] 0 2024-04-26T04:53:11.180 INFO:tasks.workunit.client.0.smithi077.stdout:5/572: creat d1/d42/dac/fcf x:0 0 0 2024-04-26T04:53:11.181 INFO:tasks.workunit.client.0.smithi077.stdout:5/573: chown d1/d42/da4/d25/lbd 437 1 2024-04-26T04:53:11.181 INFO:tasks.workunit.client.1.smithi078.stdout:7/984: creat d9/d57/f141 x:0 0 0 2024-04-26T04:53:11.182 INFO:tasks.workunit.client.0.smithi077.stdout:5/574: write d1/d8f/fbb [244214,52691] 0 2024-04-26T04:53:11.182 INFO:tasks.workunit.client.1.smithi078.stdout:9/959: getdents dc/d55/de2 0 2024-04-26T04:53:11.183 INFO:tasks.workunit.client.0.smithi077.stdout:5/575: stat d1/d42/da4/d25/d31/f92 0 2024-04-26T04:53:11.184 INFO:tasks.workunit.client.0.smithi077.stdout:5/576: fdatasync d1/d8/d7e/fc6 0 2024-04-26T04:53:11.185 INFO:tasks.workunit.client.0.smithi077.stdout:2/580: fdatasync d9/f10 0 2024-04-26T04:53:11.186 INFO:tasks.workunit.client.1.smithi078.stdout:9/960: dread dc/df/d4b/d22/d2a/d57/d69/f96 [0,4194304] 0 2024-04-26T04:53:11.191 INFO:tasks.workunit.client.0.smithi077.stdout:5/577: dwrite d1/d8f/fbb [0,4194304] 0 2024-04-26T04:53:11.195 INFO:tasks.workunit.client.0.smithi077.stdout:9/569: rmdir d9/dd/d1b/d2b/d32/d36/d7f/da5 0 2024-04-26T04:53:11.197 INFO:tasks.workunit.client.0.smithi077.stdout:6/616: symlink d8/d52/d53/da7/ld3 0 2024-04-26T04:53:11.200 INFO:tasks.workunit.client.0.smithi077.stdout:0/504: symlink d0/d2c/laf 0 2024-04-26T04:53:11.202 INFO:tasks.workunit.client.0.smithi077.stdout:7/658: write da/d15/dc6/fda [15033,84194] 0 2024-04-26T04:53:11.203 INFO:tasks.workunit.client.1.smithi078.stdout:9/961: creat dc/df/d4b/d22/d2a/d57/d99/f123 x:0 0 0 2024-04-26T04:53:11.204 INFO:tasks.workunit.client.0.smithi077.stdout:4/599: dwrite d4/d17/d18/d43/d51/db4/d15/f50 [0,4194304] 0 2024-04-26T04:53:11.206 INFO:tasks.workunit.client.0.smithi077.stdout:2/581: symlink d9/dc/d1d/d1e/d77/lc8 0 2024-04-26T04:53:11.207 INFO:tasks.workunit.client.0.smithi077.stdout:2/582: chown d9/dc/d1d/d34/lb1 26051 1 2024-04-26T04:53:11.209 INFO:tasks.workunit.client.0.smithi077.stdout:5/578: mknod d1/d8d/da7/cd0 0 2024-04-26T04:53:11.212 INFO:tasks.workunit.client.0.smithi077.stdout:6/617: creat d8/fd4 x:0 0 0 2024-04-26T04:53:11.215 INFO:tasks.workunit.client.0.smithi077.stdout:7/659: mkdir da/d13/d22/d5d/d9f/ddd 0 2024-04-26T04:53:11.222 INFO:tasks.workunit.client.1.smithi078.stdout:7/985: write d9/da/d11/d13/d1a/f42 [1342788,13628] 0 2024-04-26T04:53:11.224 INFO:tasks.workunit.client.1.smithi078.stdout:7/986: truncate d9/da/d33/d4b/d4c/f55 5231685 0 2024-04-26T04:53:11.224 INFO:tasks.workunit.client.1.smithi078.stdout:7/987: chown d9/da/d11 20308144 1 2024-04-26T04:53:11.224 INFO:tasks.workunit.client.0.smithi077.stdout:1/520: dwrite d9/db/dd/f49 [0,4194304] 0 2024-04-26T04:53:11.225 INFO:tasks.workunit.client.1.smithi078.stdout:7/988: fsync d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/ff1 0 2024-04-26T04:53:11.226 INFO:tasks.workunit.client.0.smithi077.stdout:2/583: mkdir d9/dc/d1d/d34/d39/d63/d6a/dc3/dc6/dc9 0 2024-04-26T04:53:11.230 INFO:tasks.workunit.client.1.smithi078.stdout:7/989: creat d9/da/d11/d13/d1a/d52/d5a/d136/f142 x:0 0 0 2024-04-26T04:53:11.232 INFO:tasks.workunit.client.0.smithi077.stdout:1/521: truncate d9/db/dd/d68/f7b 4161987 0 2024-04-26T04:53:11.233 INFO:tasks.workunit.client.0.smithi077.stdout:2/584: dwrite d9/d15/d44/d4e/fa7 [0,4194304] 0 2024-04-26T04:53:11.234 INFO:tasks.workunit.client.0.smithi077.stdout:1/522: chown d9/db/dd/l71 853 1 2024-04-26T04:53:11.236 INFO:tasks.workunit.client.0.smithi077.stdout:1/523: write d9/db/dd/d34/d63/f8a [131831,94928] 0 2024-04-26T04:53:11.238 INFO:tasks.workunit.client.1.smithi078.stdout:7/990: symlink d9/da/d11/d13/d1a/l143 0 2024-04-26T04:53:11.239 INFO:tasks.workunit.client.1.smithi078.stdout:7/991: write d9/da/d20/f9b [1549056,69737] 0 2024-04-26T04:53:11.239 INFO:tasks.workunit.client.0.smithi077.stdout:4/600: write d4/d17/d18/f5d [4088950,37203] 0 2024-04-26T04:53:11.242 INFO:tasks.workunit.client.0.smithi077.stdout:4/601: chown d4/d17/d18/d43/d51/db4/d15/d1f/d25/c68 12726 1 2024-04-26T04:53:11.246 INFO:tasks.workunit.client.0.smithi077.stdout:2/585: rename d9/dc/d1d/d34/d39/fbe to d9/dc/d1d/d34/d39/d63/fca 0 2024-04-26T04:53:11.248 INFO:tasks.workunit.client.1.smithi078.stdout:7/992: mkdir d9/da/d144 0 2024-04-26T04:53:11.249 INFO:tasks.workunit.client.0.smithi077.stdout:4/602: mknod d4/d17/d7f/ccd 0 2024-04-26T04:53:11.252 INFO:tasks.workunit.client.1.smithi078.stdout:7/993: write d9/da/d11/d13/d1a/f10a [4318472,85947] 0 2024-04-26T04:53:11.253 INFO:tasks.workunit.client.0.smithi077.stdout:5/579: dwrite d1/d30/f36 [0,4194304] 0 2024-04-26T04:53:11.254 INFO:tasks.workunit.client.0.smithi077.stdout:6/618: dwrite d8/d1c/f4f [0,4194304] 0 2024-04-26T04:53:11.259 INFO:tasks.workunit.client.0.smithi077.stdout:0/505: dwrite d0/d19/d1e/d2f/f3b [0,4194304] 0 2024-04-26T04:53:11.259 INFO:tasks.workunit.client.1.smithi078.stdout:7/994: dwrite d9/da/d33/d4b/d4c/f55 [4194304,4194304] 0 2024-04-26T04:53:11.261 INFO:tasks.workunit.client.0.smithi077.stdout:7/660: dwrite da/d15/faa [0,4194304] 0 2024-04-26T04:53:11.264 INFO:tasks.workunit.client.0.smithi077.stdout:0/506: write d0/d19/d1e/d2f/da8/fab [191010,40459] 0 2024-04-26T04:53:11.269 INFO:tasks.workunit.client.0.smithi077.stdout:2/586: rename d9/dc/d6e/l82 to d9/d15/d44/d4e/lcb 0 2024-04-26T04:53:11.271 INFO:tasks.workunit.client.0.smithi077.stdout:5/580: dread d1/d8f/fbb [0,4194304] 0 2024-04-26T04:53:11.272 INFO:tasks.workunit.client.0.smithi077.stdout:1/524: dwrite d9/db/dd/f2f [0,4194304] 0 2024-04-26T04:53:11.273 INFO:tasks.workunit.client.1.smithi078.stdout:7/995: chown d9/da/d11/d13/d1a/d52/d5a/d41/d68/db4/fb8 19 1 2024-04-26T04:53:11.274 INFO:tasks.workunit.client.0.smithi077.stdout:6/619: symlink d8/d52/d53/d5e/ld5 0 2024-04-26T04:53:11.275 INFO:tasks.workunit.client.0.smithi077.stdout:7/661: fdatasync da/d6b/d63/d64/f67 0 2024-04-26T04:53:11.276 INFO:tasks.workunit.client.1.smithi078.stdout:9/962: sync 2024-04-26T04:53:11.278 INFO:tasks.workunit.client.0.smithi077.stdout:7/662: write da/d13/d22/d7d/d80/fcf [466322,65221] 0 2024-04-26T04:53:11.279 INFO:tasks.workunit.client.0.smithi077.stdout:0/507: rmdir d0/d47/d68 39 2024-04-26T04:53:11.280 INFO:tasks.workunit.client.0.smithi077.stdout:4/603: link d4/d17/d18/f5d d4/d4e/fce 0 2024-04-26T04:53:11.281 INFO:tasks.workunit.client.1.smithi078.stdout:7/996: mkdir d9/da/d11/d13/d1a/d145 0 2024-04-26T04:53:11.282 INFO:tasks.workunit.client.0.smithi077.stdout:5/581: rmdir d1/d42/d62 39 2024-04-26T04:53:11.284 INFO:tasks.workunit.client.1.smithi078.stdout:9/963: creat dc/df/d4b/d22/d2d/d98/f124 x:0 0 0 2024-04-26T04:53:11.285 INFO:tasks.workunit.client.0.smithi077.stdout:1/525: mkdir d9/d90/d99 0 2024-04-26T04:53:11.287 INFO:tasks.workunit.client.1.smithi078.stdout:9/964: dread - dc/df/d4b/d22/d2a/d57/f120 zero size 2024-04-26T04:53:11.289 INFO:tasks.workunit.client.0.smithi077.stdout:7/663: symlink da/d13/d22/d7d/d80/lde 0 2024-04-26T04:53:11.290 INFO:tasks.workunit.client.1.smithi078.stdout:7/997: dwrite d9/da/d33/d4b/d4c/d5b/dc3/fdc [0,4194304] 0 2024-04-26T04:53:11.290 INFO:tasks.workunit.client.0.smithi077.stdout:4/604: chown d4/d17/d18/d43/d51/db4/d15/d1f/d35 0 1 2024-04-26T04:53:11.291 INFO:tasks.workunit.client.0.smithi077.stdout:5/582: symlink d1/d8d/ld1 0 2024-04-26T04:53:11.293 INFO:tasks.workunit.client.0.smithi077.stdout:2/587: link d9/dc/d1d/d1e/d77/fa2 d9/d15/d44/fcc 0 2024-04-26T04:53:11.298 INFO:tasks.workunit.client.1.smithi078.stdout:7/998: creat d9/da/d11/d13/d1a/d65/d117/f146 x:0 0 0 2024-04-26T04:53:11.298 INFO:tasks.workunit.client.0.smithi077.stdout:7/664: creat da/d13/d22/d5d/d9f/fdf x:0 0 0 2024-04-26T04:53:11.299 INFO:tasks.workunit.client.1.smithi078.stdout:9/965: link dc/df/d4b/d22/d2a/d57/d93/lc6 dc/df/d4b/d22/d2a/d57/d93/l125 0 2024-04-26T04:53:11.300 INFO:tasks.workunit.client.0.smithi077.stdout:4/605: unlink d4/d17/f98 0 2024-04-26T04:53:11.301 INFO:tasks.workunit.client.0.smithi077.stdout:4/606: dread - d4/d17/d18/d43/d63/d80/fcc zero size 2024-04-26T04:53:11.301 INFO:tasks.workunit.client.0.smithi077.stdout:5/583: fdatasync d1/d42/da4/d25/d31/d3c/f8e 0 2024-04-26T04:53:11.303 INFO:tasks.workunit.client.0.smithi077.stdout:1/526: mkdir d9/d90/d99/d9a 0 2024-04-26T04:53:11.305 INFO:tasks.workunit.client.1.smithi078.stdout:7/999: rename d9/da/d11/d13/d50/l74 to d9/da/d11/d13/d1a/d2d/l147 0 2024-04-26T04:53:11.306 INFO:tasks.workunit.client.1.smithi078.stdout:9/966: dread dc/df/d4b/d22/d2a/d57/d69/f96 [0,4194304] 0 2024-04-26T04:53:11.307 INFO:tasks.workunit.client.1.smithi078.stdout:9/967: write dc/df/d4b/d22/d2a/d57/fff [67972,34192] 0 2024-04-26T04:53:11.308 INFO:tasks.workunit.client.0.smithi077.stdout:5/584: symlink d1/d12/d28/ld2 0 2024-04-26T04:53:11.311 INFO:tasks.workunit.client.0.smithi077.stdout:1/527: rename d9/db/f1c to d9/db/dd/d34/f9b 0 2024-04-26T04:53:11.312 INFO:tasks.workunit.client.0.smithi077.stdout:0/508: link d0/d19/d1e/cad d0/d28/d40/cb0 0 2024-04-26T04:53:11.313 INFO:tasks.workunit.client.1.smithi078.stdout:9/968: dwrite dc/df/f2f [0,4194304] 0 2024-04-26T04:53:11.317 INFO:tasks.workunit.client.0.smithi077.stdout:5/585: mkdir d1/d42/da4/dd3 0 2024-04-26T04:53:11.321 INFO:tasks.workunit.client.0.smithi077.stdout:7/665: getdents da/d13/d22/d7d 0 2024-04-26T04:53:11.323 INFO:tasks.workunit.client.0.smithi077.stdout:4/607: creat d4/d17/d18/d43/d51/db4/d15/d1f/fcf x:0 0 0 2024-04-26T04:53:11.324 INFO:tasks.workunit.client.0.smithi077.stdout:5/586: creat d1/d8/d7e/fd4 x:0 0 0 2024-04-26T04:53:11.324 INFO:tasks.workunit.client.0.smithi077.stdout:4/608: write d4/d4e/dad/d8b/dba/fbb [197829,88924] 0 2024-04-26T04:53:11.325 INFO:tasks.workunit.client.0.smithi077.stdout:5/587: fsync d1/d30/d35/f6c 0 2024-04-26T04:53:11.325 INFO:tasks.workunit.client.0.smithi077.stdout:7/666: mkdir da/d13/d22/d5d/d9f/da7/de0 0 2024-04-26T04:53:11.326 INFO:tasks.workunit.client.0.smithi077.stdout:7/667: chown da/d15/dc6 91174 1 2024-04-26T04:53:11.327 INFO:tasks.workunit.client.0.smithi077.stdout:7/668: readlink da/d13/d22/d34/l3f 0 2024-04-26T04:53:11.330 INFO:tasks.workunit.client.0.smithi077.stdout:4/609: mkdir d4/d17/d18/d43/d6e/d6c/dae/dc5/dd0 0 2024-04-26T04:53:11.335 INFO:tasks.workunit.client.0.smithi077.stdout:6/620: truncate d8/d55/f62 7562682 0 2024-04-26T04:53:11.337 INFO:tasks.workunit.client.0.smithi077.stdout:6/621: truncate d8/d55/f76 1466782 0 2024-04-26T04:53:11.337 INFO:tasks.workunit.client.0.smithi077.stdout:6/622: chown d8/d52/d53/da7 257699670 1 2024-04-26T04:53:11.340 INFO:tasks.workunit.client.0.smithi077.stdout:6/623: link d8/d52/c99 d8/d55/cd6 0 2024-04-26T04:53:11.345 INFO:tasks.workunit.client.1.smithi078.stdout:9/969: dread dc/df/d27/f53 [0,4194304] 0 2024-04-26T04:53:11.345 INFO:tasks.workunit.client.1.smithi078.stdout:9/970: chown dc/df/d4b/d3e/l65 25552 1 2024-04-26T04:53:11.350 INFO:tasks.workunit.client.1.smithi078.stdout:9/971: dwrite dc/f39 [4194304,4194304] 0 2024-04-26T04:53:11.365 INFO:tasks.workunit.client.0.smithi077.stdout:6/624: stat d8/d55/c9d 0 2024-04-26T04:53:11.365 INFO:tasks.workunit.client.0.smithi077.stdout:6/625: chown d8/d52/d53 967996 1 2024-04-26T04:53:11.371 INFO:tasks.workunit.client.1.smithi078.stdout:9/972: link dc/df/d4b/d22/d2d/d98/fd4 dc/df/d4b/d22/dbe/f126 0 2024-04-26T04:53:11.372 INFO:tasks.workunit.client.0.smithi077.stdout:2/588: write d9/dc/d1d/d1e/fc0 [74682,52496] 0 2024-04-26T04:53:11.377 INFO:tasks.workunit.client.1.smithi078.stdout:9/973: symlink dc/df/d4b/d3e/d80/l127 0 2024-04-26T04:53:11.377 INFO:tasks.workunit.client.0.smithi077.stdout:0/509: write d0/d19/d1e/d2f/d52/f62 [4769897,60764] 0 2024-04-26T04:53:11.378 INFO:tasks.workunit.client.0.smithi077.stdout:2/589: mkdir d9/dc/d5f/d8e/dcd 0 2024-04-26T04:53:11.379 INFO:tasks.workunit.client.0.smithi077.stdout:0/510: truncate d0/d2c/d4e/f4c 1044937 0 2024-04-26T04:53:11.381 INFO:tasks.workunit.client.0.smithi077.stdout:0/511: write d0/d47/d93/d94/faa [108538,95772] 0 2024-04-26T04:53:11.381 INFO:tasks.workunit.client.0.smithi077.stdout:1/528: mkdir d9/db/dd/d9c 0 2024-04-26T04:53:11.382 INFO:tasks.workunit.client.0.smithi077.stdout:2/590: symlink d9/dc/d1d/d1e/lce 0 2024-04-26T04:53:11.383 INFO:tasks.workunit.client.0.smithi077.stdout:0/512: readlink d0/d19/d1e/d2f/d52/l58 0 2024-04-26T04:53:11.387 INFO:tasks.workunit.client.0.smithi077.stdout:5/588: dwrite d1/d42/da4/d25/d31/d3c/f68 [0,4194304] 0 2024-04-26T04:53:11.389 INFO:tasks.workunit.client.0.smithi077.stdout:0/513: symlink d0/d19/lb1 0 2024-04-26T04:53:11.391 INFO:tasks.workunit.client.0.smithi077.stdout:4/610: dwrite d4/d17/f7a [0,4194304] 0 2024-04-26T04:53:11.391 INFO:tasks.workunit.client.0.smithi077.stdout:1/529: fsync d9/db/d2b/f3d 0 2024-04-26T04:53:11.398 INFO:tasks.workunit.client.0.smithi077.stdout:5/589: mkdir d1/d42/d62/dc8/dd5 0 2024-04-26T04:53:11.399 INFO:tasks.workunit.client.0.smithi077.stdout:5/590: chown d1/d42/da4/l29 437 1 2024-04-26T04:53:11.402 INFO:tasks.workunit.client.0.smithi077.stdout:0/514: link d0/d47/l3f d0/d19/d1e/d2f/lb2 0 2024-04-26T04:53:11.402 INFO:tasks.workunit.client.0.smithi077.stdout:4/611: link d4/d17/lca d4/d17/d7f/da3/ld1 0 2024-04-26T04:53:11.406 INFO:tasks.workunit.client.0.smithi077.stdout:0/515: mkdir d0/d19/d8d/db3 0 2024-04-26T04:53:11.410 INFO:tasks.workunit.client.0.smithi077.stdout:1/530: dread d9/db/dd/d34/d63/f77 [0,4194304] 0 2024-04-26T04:53:11.412 INFO:tasks.workunit.client.0.smithi077.stdout:4/612: unlink d4/d17/d18/d43/d51/db4/d15/d1f/f5e 0 2024-04-26T04:53:11.412 INFO:tasks.workunit.client.0.smithi077.stdout:6/626: dwrite d8/d52/d53/d5e/fae [0,4194304] 0 2024-04-26T04:53:11.412 INFO:tasks.workunit.client.0.smithi077.stdout:5/591: link d1/d12/d28/c40 d1/d12/d45/cd6 0 2024-04-26T04:53:11.414 INFO:tasks.workunit.client.0.smithi077.stdout:5/592: chown d1/d30/d33/d66/cab 71 1 2024-04-26T04:53:11.418 INFO:tasks.workunit.client.0.smithi077.stdout:6/627: symlink d8/d52/ld7 0 2024-04-26T04:53:11.420 INFO:tasks.workunit.client.0.smithi077.stdout:4/613: mknod d4/d17/d7f/cd2 0 2024-04-26T04:53:11.420 INFO:tasks.workunit.client.1.smithi078.stdout:9/974: dwrite dc/df/d4b/d3e/d80/f95 [4194304,4194304] 0 2024-04-26T04:53:11.424 INFO:tasks.workunit.client.0.smithi077.stdout:2/591: dwrite d9/f2c [4194304,4194304] 0 2024-04-26T04:53:11.426 INFO:tasks.workunit.client.0.smithi077.stdout:2/592: chown d9/dc/d1d/d1e/c73 19767 1 2024-04-26T04:53:11.426 INFO:tasks.workunit.client.0.smithi077.stdout:6/628: symlink d8/d8f/ld8 0 2024-04-26T04:53:11.431 INFO:tasks.workunit.client.1.smithi078.stdout:9/975: link dc/df/d4b/d22/dbb/ld2 dc/d55/de2/l128 0 2024-04-26T04:53:11.432 INFO:tasks.workunit.client.1.smithi078.stdout:9/976: fsync dc/df/d4b/d22/d2a/d57/fa4 0 2024-04-26T04:53:11.432 INFO:tasks.workunit.client.0.smithi077.stdout:0/516: link d0/d19/d8d/lac d0/d19/d1e/d2f/da8/lb4 0 2024-04-26T04:53:11.438 INFO:tasks.workunit.client.0.smithi077.stdout:2/593: creat d9/dc/d1d/d34/d39/d63/d79/d5b/fcf x:0 0 0 2024-04-26T04:53:11.441 INFO:tasks.workunit.client.0.smithi077.stdout:2/594: truncate d9/dc/d1d/d1e/d77/fa2 172140 0 2024-04-26T04:53:11.442 INFO:tasks.workunit.client.0.smithi077.stdout:2/595: fdatasync d9/d15/d91/f96 0 2024-04-26T04:53:11.443 INFO:tasks.workunit.client.0.smithi077.stdout:2/596: read - d9/d15/f9f zero size 2024-04-26T04:53:11.444 INFO:tasks.workunit.client.0.smithi077.stdout:0/517: rmdir d0/d19/d8d/db3 0 2024-04-26T04:53:11.448 INFO:tasks.workunit.client.0.smithi077.stdout:0/518: read d0/d2c/d4e/f4c [322068,97737] 0 2024-04-26T04:53:11.449 INFO:tasks.workunit.client.0.smithi077.stdout:5/593: dwrite d1/d8/f95 [0,4194304] 0 2024-04-26T04:53:11.449 INFO:tasks.workunit.client.1.smithi078.stdout:9/977: dread dc/df/d27/f9b [0,4194304] 0 2024-04-26T04:53:11.450 INFO:tasks.workunit.client.1.smithi078.stdout:9/978: read dc/df/d4b/dae/ff5 [355674,69011] 0 2024-04-26T04:53:11.451 INFO:tasks.workunit.client.0.smithi077.stdout:0/519: read d0/f1 [3343629,116782] 0 2024-04-26T04:53:11.451 INFO:tasks.workunit.client.0.smithi077.stdout:5/594: creat d1/d42/da4/d25/fd7 x:0 0 0 2024-04-26T04:53:11.453 INFO:tasks.workunit.client.0.smithi077.stdout:0/520: write d0/d19/d69/fa6 [406621,53376] 0 2024-04-26T04:53:11.454 INFO:tasks.workunit.client.1.smithi078.stdout:9/979: mkdir dc/df/d4b/d22/d129 0 2024-04-26T04:53:11.454 INFO:tasks.workunit.client.0.smithi077.stdout:2/597: dread d9/dc/f7a [0,4194304] 0 2024-04-26T04:53:11.458 INFO:tasks.workunit.client.1.smithi078.stdout:9/980: rename dc/df/d4b/fcc to dc/df/d4b/d22/dbe/f12a 0 2024-04-26T04:53:11.460 INFO:tasks.workunit.client.1.smithi078.stdout:9/981: truncate dc/df/d4b/dae/fdc 201434 0 2024-04-26T04:53:11.460 INFO:tasks.workunit.client.0.smithi077.stdout:0/521: dread d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:11.460 INFO:tasks.workunit.client.0.smithi077.stdout:2/598: dwrite d9/f12 [4194304,4194304] 0 2024-04-26T04:53:11.462 INFO:tasks.workunit.client.1.smithi078.stdout:9/982: rmdir dc/df/d4b/d22/d2a/d57/d69/d106 39 2024-04-26T04:53:11.463 INFO:tasks.workunit.client.1.smithi078.stdout:9/983: readlink dc/df/d4b/d58/lc7 0 2024-04-26T04:53:11.464 INFO:tasks.workunit.client.1.smithi078.stdout:9/984: chown dc/df/d4b/d22/d2a/d57/fa4 2 1 2024-04-26T04:53:11.465 INFO:tasks.workunit.client.0.smithi077.stdout:2/599: mkdir d9/dc/d1d/d34/d39/d63/d6a/dc3/dc6/dc9/dd0 0 2024-04-26T04:53:11.466 INFO:tasks.workunit.client.0.smithi077.stdout:6/629: dread d8/f17 [0,4194304] 0 2024-04-26T04:53:11.469 INFO:tasks.workunit.client.0.smithi077.stdout:0/522: dwrite d0/d2c/f2e [0,4194304] 0 2024-04-26T04:53:11.471 INFO:tasks.workunit.client.0.smithi077.stdout:1/531: truncate d9/f42 1011500 0 2024-04-26T04:53:11.472 INFO:tasks.workunit.client.0.smithi077.stdout:1/532: chown d9/d53/d6e/f87 198750 1 2024-04-26T04:53:11.472 INFO:tasks.workunit.client.0.smithi077.stdout:6/630: dread d8/f3f [0,4194304] 0 2024-04-26T04:53:11.473 INFO:tasks.workunit.client.1.smithi078.stdout:9/985: dwrite dc/df/d4b/dae/ff5 [0,4194304] 0 2024-04-26T04:53:11.477 INFO:tasks.workunit.client.0.smithi077.stdout:1/533: rmdir d9/db/dd 39 2024-04-26T04:53:11.480 INFO:tasks.workunit.client.0.smithi077.stdout:6/631: mknod d8/d1c/cd9 0 2024-04-26T04:53:11.481 INFO:tasks.workunit.client.0.smithi077.stdout:4/614: write d4/d17/d18/d43/d51/db4/d15/d1f/f32 [726982,20337] 0 2024-04-26T04:53:11.487 INFO:tasks.workunit.client.0.smithi077.stdout:6/632: symlink d8/d52/d53/d5e/lda 0 2024-04-26T04:53:11.487 INFO:tasks.workunit.client.0.smithi077.stdout:4/615: read d4/d17/d18/f5d [1341214,106613] 0 2024-04-26T04:53:11.487 INFO:tasks.workunit.client.0.smithi077.stdout:0/523: dwrite d0/d47/f16 [0,4194304] 0 2024-04-26T04:53:11.488 INFO:tasks.workunit.client.1.smithi078.stdout:9/986: write dc/df/d27/d81/da8/fac [1031707,102957] 0 2024-04-26T04:53:11.494 INFO:tasks.workunit.client.1.smithi078.stdout:9/987: write dc/df/d4b/d22/dbe/f11f [1413723,126775] 0 2024-04-26T04:53:11.496 INFO:tasks.workunit.client.0.smithi077.stdout:6/633: read - d8/d52/d53/d5e/da6/faa zero size 2024-04-26T04:53:11.497 INFO:tasks.workunit.client.0.smithi077.stdout:6/634: chown d8/c29 1098221 1 2024-04-26T04:53:11.497 INFO:tasks.workunit.client.0.smithi077.stdout:0/524: symlink d0/d19/d1e/d2f/d52/lb5 0 2024-04-26T04:53:11.497 INFO:tasks.workunit.client.0.smithi077.stdout:6/635: dread - d8/d52/d53/da7/fc3 zero size 2024-04-26T04:53:11.499 INFO:tasks.workunit.client.1.smithi078.stdout:9/988: mkdir dc/df/d27/dbc/d12b 0 2024-04-26T04:53:11.501 INFO:tasks.workunit.client.0.smithi077.stdout:2/600: rename d9/dc/d1d/d34/d39/d63/d6a to d9/dc/d1d/d34/d39/d63/d79/d2b/dd1 0 2024-04-26T04:53:11.501 INFO:tasks.workunit.client.0.smithi077.stdout:2/601: fdatasync d9/dc/d1d/d1e/f3b 0 2024-04-26T04:53:11.502 INFO:tasks.workunit.client.0.smithi077.stdout:0/525: symlink d0/d28/d40/lb6 0 2024-04-26T04:53:11.503 INFO:tasks.workunit.client.0.smithi077.stdout:0/526: chown d0/d2c/d4e/f66 15390 1 2024-04-26T04:53:11.503 INFO:tasks.workunit.client.0.smithi077.stdout:6/636: mknod d8/d52/d53/cdb 0 2024-04-26T04:53:11.507 INFO:tasks.workunit.client.0.smithi077.stdout:2/602: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/db8/cd2 0 2024-04-26T04:53:11.508 INFO:tasks.workunit.client.0.smithi077.stdout:6/637: fdatasync d8/d52/d53/d5e/f69 0 2024-04-26T04:53:11.510 INFO:tasks.workunit.client.0.smithi077.stdout:6/638: stat d8/d52/d53/d5e 0 2024-04-26T04:53:11.512 INFO:tasks.workunit.client.0.smithi077.stdout:6/639: fsync d8/d1c/f28 0 2024-04-26T04:53:11.512 INFO:tasks.workunit.client.0.smithi077.stdout:0/527: dwrite d0/d19/d1e/d2f/da8/fab [0,4194304] 0 2024-04-26T04:53:11.514 INFO:tasks.workunit.client.0.smithi077.stdout:2/603: dread d9/dc/d1d/d1e/f9b [0,4194304] 0 2024-04-26T04:53:11.520 INFO:tasks.workunit.client.0.smithi077.stdout:2/604: dwrite d9/dc/d5f/d8e/da6/fb3 [0,4194304] 0 2024-04-26T04:53:11.552 INFO:tasks.workunit.client.1.smithi078.stdout:9/989: link dc/df/d4b/d22/d2a/d103/ddb/d118/l121 dc/df/d4b/d22/d2a/d57/d69/d106/l12c 0 2024-04-26T04:53:11.556 INFO:tasks.workunit.client.0.smithi077.stdout:6/640: creat d8/d52/d53/d5e/d6c/fdc x:0 0 0 2024-04-26T04:53:11.562 INFO:tasks.workunit.client.0.smithi077.stdout:2/605: mknod d9/d15/d56/cd3 0 2024-04-26T04:53:11.562 INFO:tasks.workunit.client.0.smithi077.stdout:1/534: truncate d9/db/d2b/f36 1997869 0 2024-04-26T04:53:11.566 INFO:tasks.workunit.client.0.smithi077.stdout:0/528: rmdir d0/d19/d1e/da7 0 2024-04-26T04:53:11.568 INFO:tasks.workunit.client.0.smithi077.stdout:6/641: mkdir d8/d2a/ddd 0 2024-04-26T04:53:11.569 INFO:tasks.workunit.client.0.smithi077.stdout:2/606: mkdir d9/dc/d1d/d34/dd4 0 2024-04-26T04:53:11.574 INFO:tasks.workunit.client.0.smithi077.stdout:0/529: dwrite d0/d28/d40/f85 [0,4194304] 0 2024-04-26T04:53:11.575 INFO:tasks.workunit.client.1.smithi078.stdout:9/990: truncate dc/df/d27/fe3 4095760 0 2024-04-26T04:53:11.579 INFO:tasks.workunit.client.0.smithi077.stdout:2/607: mknod d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6/dc9/dd0/cd5 0 2024-04-26T04:53:11.583 INFO:tasks.workunit.client.1.smithi078.stdout:9/991: mknod dc/df/d4b/d22/d2a/d57/d69/c12d 0 2024-04-26T04:53:11.589 INFO:tasks.workunit.client.1.smithi078.stdout:9/992: rename dc/df/d4b/d3e/d80/cf4 to dc/c12e 0 2024-04-26T04:53:11.590 INFO:tasks.workunit.client.0.smithi077.stdout:6/642: dwrite d8/d52/f4d [4194304,4194304] 0 2024-04-26T04:53:11.591 INFO:tasks.workunit.client.0.smithi077.stdout:6/643: chown d8/d52/da0/f7d 672415 1 2024-04-26T04:53:11.592 INFO:tasks.workunit.client.0.smithi077.stdout:0/530: dwrite d0/d47/f22 [0,4194304] 0 2024-04-26T04:53:11.595 INFO:tasks.workunit.client.0.smithi077.stdout:6/644: symlink d8/d2a/d86/lde 0 2024-04-26T04:53:11.596 INFO:tasks.workunit.client.0.smithi077.stdout:0/531: getdents d0/d47/d64/d9e 0 2024-04-26T04:53:11.598 INFO:tasks.workunit.client.0.smithi077.stdout:6/645: mkdir d8/d52/d53/d5e/da6/ddf 0 2024-04-26T04:53:11.599 INFO:tasks.workunit.client.0.smithi077.stdout:0/532: symlink d0/d19/d1e/d2f/lb7 0 2024-04-26T04:53:11.600 INFO:tasks.workunit.client.0.smithi077.stdout:1/535: dread d9/f42 [0,4194304] 0 2024-04-26T04:53:11.600 INFO:tasks.workunit.client.0.smithi077.stdout:0/533: chown d0/d19/d1e 26694291 1 2024-04-26T04:53:11.607 INFO:tasks.workunit.client.0.smithi077.stdout:6/646: dread d8/d2a/f32 [4194304,4194304] 0 2024-04-26T04:53:11.607 INFO:tasks.workunit.client.0.smithi077.stdout:6/647: stat d8/fd4 0 2024-04-26T04:53:11.608 INFO:tasks.workunit.client.0.smithi077.stdout:6/648: dread - d8/d52/f87 zero size 2024-04-26T04:53:11.609 INFO:tasks.workunit.client.0.smithi077.stdout:6/649: chown d8/d52/d53/d5e/fb0 974583 1 2024-04-26T04:53:11.610 INFO:tasks.workunit.client.0.smithi077.stdout:6/650: write d8/f50 [3060355,64403] 0 2024-04-26T04:53:11.611 INFO:tasks.workunit.client.0.smithi077.stdout:6/651: write d8/d55/f76 [1081806,101697] 0 2024-04-26T04:53:11.614 INFO:tasks.workunit.client.0.smithi077.stdout:6/652: rename d8/d1c/f8a to d8/d2a/d86/fe0 0 2024-04-26T04:53:11.617 INFO:tasks.workunit.client.0.smithi077.stdout:2/608: dread d9/d15/f37 [0,4194304] 0 2024-04-26T04:53:11.619 INFO:tasks.workunit.client.0.smithi077.stdout:2/609: creat d9/dc/d1d/d34/d39/d63/d79/fd6 x:0 0 0 2024-04-26T04:53:11.620 INFO:tasks.workunit.client.0.smithi077.stdout:2/610: write d9/dc/d1d/f83 [3838469,95653] 0 2024-04-26T04:53:11.621 INFO:tasks.workunit.client.0.smithi077.stdout:2/611: stat d9/dc/d1d/d34/d39/d63 0 2024-04-26T04:53:11.626 INFO:tasks.workunit.client.1.smithi078.stdout:9/993: dread dc/df/d4b/d22/d2a/d57/fbf [0,4194304] 0 2024-04-26T04:53:11.627 INFO:tasks.workunit.client.0.smithi077.stdout:2/612: dwrite d9/dc/d1d/d1e/d77/fc7 [0,4194304] 0 2024-04-26T04:53:11.628 INFO:tasks.workunit.client.1.smithi078.stdout:9/994: fsync dc/df/d4b/d22/fc9 0 2024-04-26T04:53:11.629 INFO:tasks.workunit.client.1.smithi078.stdout:9/995: chown dc/df/d27/l9a 13 1 2024-04-26T04:53:11.630 INFO:tasks.workunit.client.0.smithi077.stdout:1/536: truncate d9/db/d2b/d3f/f52 63507 0 2024-04-26T04:53:11.630 INFO:tasks.workunit.client.1.smithi078.stdout:9/996: chown dc/f3d 0 1 2024-04-26T04:53:11.636 INFO:tasks.workunit.client.0.smithi077.stdout:6/653: getdents d8/d2a/d86 0 2024-04-26T04:53:11.636 INFO:tasks.workunit.client.1.smithi078.stdout:9/997: dwrite dc/df/d4b/dae/fc1 [0,4194304] 0 2024-04-26T04:53:11.636 INFO:tasks.workunit.client.0.smithi077.stdout:2/613: dwrite d9/dc/d1d/d1e/d77/fc7 [0,4194304] 0 2024-04-26T04:53:11.638 INFO:tasks.workunit.client.0.smithi077.stdout:2/614: read d9/d15/f37 [3992226,123410] 0 2024-04-26T04:53:11.638 INFO:tasks.workunit.client.0.smithi077.stdout:2/615: read - d9/d15/fb4 zero size 2024-04-26T04:53:11.642 INFO:tasks.workunit.client.0.smithi077.stdout:0/534: dread d0/d19/d1e/d2f/d52/f76 [0,4194304] 0 2024-04-26T04:53:11.645 INFO:tasks.workunit.client.0.smithi077.stdout:6/654: creat d8/d52/d53/fe1 x:0 0 0 2024-04-26T04:53:11.646 INFO:tasks.workunit.client.0.smithi077.stdout:1/537: link d9/db/dd/l71 d9/db/d2b/d75/l9d 0 2024-04-26T04:53:11.650 INFO:tasks.workunit.client.0.smithi077.stdout:1/538: rmdir d9/d90/d99 39 2024-04-26T04:53:11.651 INFO:tasks.workunit.client.0.smithi077.stdout:1/539: readlink d9/db/dd/l2e 0 2024-04-26T04:53:11.656 INFO:tasks.workunit.client.0.smithi077.stdout:6/655: mknod d8/d2a/ddd/ce2 0 2024-04-26T04:53:11.657 INFO:tasks.workunit.client.0.smithi077.stdout:1/540: dwrite d9/db/dd/d34/f38 [0,4194304] 0 2024-04-26T04:53:11.657 INFO:tasks.workunit.client.0.smithi077.stdout:1/541: chown d9/d53/d8b 5779 1 2024-04-26T04:53:11.659 INFO:tasks.workunit.client.0.smithi077.stdout:2/616: rename d9/dc/d1d/d34/d39/d63/d79/f22 to d9/dc/d1d/d34/fd7 0 2024-04-26T04:53:11.661 INFO:tasks.workunit.client.0.smithi077.stdout:6/656: creat d8/d2a/ddd/fe3 x:0 0 0 2024-04-26T04:53:11.663 INFO:tasks.workunit.client.0.smithi077.stdout:2/617: fsync d9/dc/d1d/d1e/f9b 0 2024-04-26T04:53:11.665 INFO:tasks.workunit.client.0.smithi077.stdout:6/657: mknod d8/d1c/ce4 0 2024-04-26T04:53:11.669 INFO:tasks.workunit.client.1.smithi078.stdout:9/998: truncate dc/df/d27/f5c 21872 0 2024-04-26T04:53:11.670 INFO:tasks.workunit.client.0.smithi077.stdout:6/658: creat d8/d8f/fe5 x:0 0 0 2024-04-26T04:53:11.673 INFO:tasks.workunit.client.0.smithi077.stdout:2/618: symlink d9/dc/d1d/d1e/d77/d9d/ld8 0 2024-04-26T04:53:11.675 INFO:tasks.workunit.client.0.smithi077.stdout:6/659: mkdir d8/d52/d53/d5e/da6/de6 0 2024-04-26T04:53:11.675 INFO:tasks.workunit.client.0.smithi077.stdout:0/535: dwrite d0/d2c/d4e/f4c [0,4194304] 0 2024-04-26T04:53:11.681 INFO:tasks.workunit.client.0.smithi077.stdout:6/660: unlink d8/f43 0 2024-04-26T04:53:11.683 INFO:tasks.workunit.client.0.smithi077.stdout:0/536: chown d0/d19/d1e/d81/l92 0 1 2024-04-26T04:53:11.684 INFO:tasks.workunit.client.0.smithi077.stdout:0/537: fdatasync d0/d19/f2d 0 2024-04-26T04:53:11.692 INFO:tasks.workunit.client.0.smithi077.stdout:0/538: mknod d0/d47/d64/d9e/cb8 0 2024-04-26T04:53:11.696 INFO:tasks.workunit.client.0.smithi077.stdout:0/539: creat d0/d19/d1e/d59/fb9 x:0 0 0 2024-04-26T04:53:11.697 INFO:tasks.workunit.client.0.smithi077.stdout:2/619: dread d9/f17 [0,4194304] 0 2024-04-26T04:53:11.699 INFO:tasks.workunit.client.0.smithi077.stdout:6/661: getdents d8/d1c/db5 0 2024-04-26T04:53:11.702 INFO:tasks.workunit.client.0.smithi077.stdout:0/540: chown d0/d19/d1e/cad 0 1 2024-04-26T04:53:11.704 INFO:tasks.workunit.client.0.smithi077.stdout:2/620: creat d9/dc/d6e/fd9 x:0 0 0 2024-04-26T04:53:11.705 INFO:tasks.workunit.client.0.smithi077.stdout:6/662: dwrite d8/d8f/fe5 [0,4194304] 0 2024-04-26T04:53:11.712 INFO:tasks.workunit.client.0.smithi077.stdout:2/621: mknod d9/dc/d1d/cda 0 2024-04-26T04:53:11.714 INFO:tasks.workunit.client.0.smithi077.stdout:2/622: stat d9/dc/d1d/d34/d39/d63/d79/f1c 0 2024-04-26T04:53:11.716 INFO:tasks.workunit.client.0.smithi077.stdout:0/541: rmdir d0/d47/d64 39 2024-04-26T04:53:11.718 INFO:tasks.workunit.client.0.smithi077.stdout:2/623: symlink d9/dc/d5f/d78/ldb 0 2024-04-26T04:53:11.722 INFO:tasks.workunit.client.0.smithi077.stdout:2/624: symlink d9/dc/d6e/ldc 0 2024-04-26T04:53:11.729 INFO:tasks.workunit.client.0.smithi077.stdout:2/625: dwrite d9/d15/d44/fa8 [0,4194304] 0 2024-04-26T04:53:11.733 INFO:tasks.workunit.client.0.smithi077.stdout:0/542: dread d0/d19/f49 [0,4194304] 0 2024-04-26T04:53:11.735 INFO:tasks.workunit.client.0.smithi077.stdout:2/626: rename d9/d15/d56/f8c to d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6/dc9/fdd 0 2024-04-26T04:53:11.736 INFO:tasks.workunit.client.0.smithi077.stdout:0/543: fsync d0/d19/f87 0 2024-04-26T04:53:11.737 INFO:tasks.workunit.client.0.smithi077.stdout:0/544: readlink d0/d19/d1e/d81/la0 0 2024-04-26T04:53:11.741 INFO:tasks.workunit.client.0.smithi077.stdout:2/627: dwrite d9/d15/f93 [0,4194304] 0 2024-04-26T04:53:11.742 INFO:tasks.workunit.client.0.smithi077.stdout:3/495: sync 2024-04-26T04:53:11.744 INFO:tasks.workunit.client.0.smithi077.stdout:0/545: mkdir d0/d47/d64/d97/dba 0 2024-04-26T04:53:11.746 INFO:tasks.workunit.client.0.smithi077.stdout:2/628: creat d9/d15/d56/fde x:0 0 0 2024-04-26T04:53:11.749 INFO:tasks.workunit.client.0.smithi077.stdout:0/546: dread d0/d19/f32 [0,4194304] 0 2024-04-26T04:53:11.757 INFO:tasks.workunit.client.0.smithi077.stdout:0/547: symlink d0/d19/d1e/lbb 0 2024-04-26T04:53:11.758 INFO:tasks.workunit.client.1.smithi078.stdout:9/999: symlink dc/dd9/de1/l12f 0 2024-04-26T04:53:11.759 INFO:tasks.workunit.client.0.smithi077.stdout:0/548: truncate d0/d47/d93/d94/faa 783622 0 2024-04-26T04:53:11.761 INFO:tasks.workunit.client.0.smithi077.stdout:0/549: mknod d0/d47/d64/cbc 0 2024-04-26T04:53:11.762 INFO:tasks.workunit.client.1.smithi078.stderr:+ rm -rf -- ./tmp.U5lnxyJscE 2024-04-26T04:53:11.773 INFO:tasks.workunit.client.0.smithi077.stdout:0/550: dread d0/d19/d1e/d59/d96/f99 [0,4194304] 0 2024-04-26T04:53:11.774 INFO:tasks.workunit.client.0.smithi077.stdout:0/551: chown d0/l1d 2953 1 2024-04-26T04:53:11.781 INFO:tasks.workunit.client.0.smithi077.stdout:2/629: dread d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6/dc9/fdd [0,4194304] 0 2024-04-26T04:53:11.782 INFO:tasks.workunit.client.0.smithi077.stdout:3/496: write d0/d7/f54 [135514,71586] 0 2024-04-26T04:53:11.782 INFO:tasks.workunit.client.0.smithi077.stdout:2/630: readlink d9/dc/d1d/d34/lb1 0 2024-04-26T04:53:11.788 INFO:tasks.workunit.client.0.smithi077.stdout:0/552: getdents d0/d47/d64 0 2024-04-26T04:53:11.790 INFO:tasks.workunit.client.0.smithi077.stdout:3/497: dwrite d0/f38 [0,4194304] 0 2024-04-26T04:53:11.791 INFO:tasks.workunit.client.0.smithi077.stdout:2/631: dwrite d9/dc/d1d/d34/d39/d63/d79/f1c [0,4194304] 0 2024-04-26T04:53:11.792 INFO:tasks.workunit.client.0.smithi077.stdout:3/498: readlink d0/d9/d32/d2a/l2d 0 2024-04-26T04:53:11.793 INFO:tasks.workunit.client.0.smithi077.stdout:3/499: chown d0/d7/d1f 159 1 2024-04-26T04:53:11.793 INFO:tasks.workunit.client.0.smithi077.stdout:3/500: chown d0/d9/d32/d2a 61334 1 2024-04-26T04:53:11.796 INFO:tasks.workunit.client.0.smithi077.stdout:2/632: fsync d9/f64 0 2024-04-26T04:53:11.799 INFO:tasks.workunit.client.0.smithi077.stdout:3/501: mknod d0/d7/d1f/d76/c9c 0 2024-04-26T04:53:11.802 INFO:tasks.workunit.client.0.smithi077.stdout:0/553: rename d0/d19/d8d/lac to d0/d19/d1e/d2f/d52/lbd 0 2024-04-26T04:53:11.805 INFO:tasks.workunit.client.0.smithi077.stdout:2/633: truncate d9/dc/d1d/d34/d39/d63/d79/f3f 2411140 0 2024-04-26T04:53:11.806 INFO:tasks.workunit.client.0.smithi077.stdout:2/634: fdatasync d9/d15/d44/fa8 0 2024-04-26T04:53:11.807 INFO:tasks.workunit.client.0.smithi077.stdout:0/554: mknod d0/d19/d1e/cbe 0 2024-04-26T04:53:11.808 INFO:tasks.workunit.client.0.smithi077.stdout:0/555: chown d0/d2c/f88 2950 1 2024-04-26T04:53:11.809 INFO:tasks.workunit.client.0.smithi077.stdout:0/556: read d0/d19/d1e/d2f/da8/fab [3943259,25226] 0 2024-04-26T04:53:11.810 INFO:tasks.workunit.client.0.smithi077.stdout:0/557: write d0/d47/f22 [3190578,121086] 0 2024-04-26T04:53:11.811 INFO:tasks.workunit.client.0.smithi077.stdout:3/502: creat d0/f9d x:0 0 0 2024-04-26T04:53:11.812 INFO:tasks.workunit.client.0.smithi077.stdout:0/558: readlink d0/d28/d40/lb6 0 2024-04-26T04:53:11.817 INFO:tasks.workunit.client.0.smithi077.stdout:2/635: dwrite d9/dc/d1d/d34/fd7 [0,4194304] 0 2024-04-26T04:53:11.819 INFO:tasks.workunit.client.0.smithi077.stdout:0/559: rename d0/d28/c75 to d0/d19/d1e/cbf 0 2024-04-26T04:53:11.820 INFO:tasks.workunit.client.0.smithi077.stdout:3/503: dwrite d0/f19 [0,4194304] 0 2024-04-26T04:53:11.821 INFO:tasks.workunit.client.0.smithi077.stdout:2/636: unlink d9/d15/d44/d4e/fa7 0 2024-04-26T04:53:11.827 INFO:tasks.workunit.client.0.smithi077.stdout:0/560: mknod d0/d47/d93/cc0 0 2024-04-26T04:53:11.829 INFO:tasks.workunit.client.0.smithi077.stdout:3/504: mknod d0/d7/d9a/c9e 0 2024-04-26T04:53:11.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/561: chown d0/c23 0 1 2024-04-26T04:53:11.836 INFO:tasks.workunit.client.0.smithi077.stdout:3/505: dwrite d0/d7/d1f/d76/f7e [0,4194304] 0 2024-04-26T04:53:11.837 INFO:tasks.workunit.client.0.smithi077.stdout:3/506: readlink d0/d9/l8f 0 2024-04-26T04:53:11.844 INFO:tasks.workunit.client.0.smithi077.stdout:0/562: write d0/d47/d68/fa5 [1030862,106057] 0 2024-04-26T04:53:11.845 INFO:tasks.workunit.client.0.smithi077.stdout:3/507: creat d0/d7/d1f/d76/f9f x:0 0 0 2024-04-26T04:53:11.852 INFO:tasks.workunit.client.0.smithi077.stdout:0/563: dwrite d0/d19/d69/fa6 [0,4194304] 0 2024-04-26T04:53:11.855 INFO:tasks.workunit.client.0.smithi077.stdout:0/564: symlink d0/d47/d93/lc1 0 2024-04-26T04:53:11.855 INFO:tasks.workunit.client.0.smithi077.stdout:3/508: dwrite d0/d7/f34 [0,4194304] 0 2024-04-26T04:53:11.857 INFO:tasks.workunit.client.0.smithi077.stdout:0/565: write d0/d19/d1e/d2f/d52/f62 [4923539,73441] 0 2024-04-26T04:53:11.857 INFO:tasks.workunit.client.0.smithi077.stdout:3/509: mknod d0/d7/d1f/d76/ca0 0 2024-04-26T04:53:11.858 INFO:tasks.workunit.client.0.smithi077.stdout:3/510: write d0/f38 [1067786,118445] 0 2024-04-26T04:53:11.865 INFO:tasks.workunit.client.0.smithi077.stdout:3/511: symlink d0/d7/d15/la1 0 2024-04-26T04:53:11.866 INFO:tasks.workunit.client.0.smithi077.stdout:0/566: mknod d0/d28/cc2 0 2024-04-26T04:53:11.868 INFO:tasks.workunit.client.0.smithi077.stdout:3/512: symlink d0/d7/d1f/d76/la2 0 2024-04-26T04:53:11.870 INFO:tasks.workunit.client.0.smithi077.stdout:2/637: truncate d9/dc/d1d/f24 3213107 0 2024-04-26T04:53:11.871 INFO:tasks.workunit.client.0.smithi077.stdout:0/567: creat d0/d47/d64/d9e/fc3 x:0 0 0 2024-04-26T04:53:11.872 INFO:tasks.workunit.client.0.smithi077.stdout:3/513: truncate d0/d9/f2b 1432824 0 2024-04-26T04:53:11.873 INFO:tasks.workunit.client.0.smithi077.stdout:3/514: write d0/d7/d1f/d4d/f92 [864155,92531] 0 2024-04-26T04:53:11.875 INFO:tasks.workunit.client.0.smithi077.stdout:0/568: rename d0/d19/d1e/d59/d5a to d0/d47/d64/d97/dba/dc4 0 2024-04-26T04:53:11.876 INFO:tasks.workunit.client.0.smithi077.stdout:0/569: write d0/d2c/f88 [3562784,32978] 0 2024-04-26T04:53:11.877 INFO:tasks.workunit.client.0.smithi077.stdout:3/515: write d0/d7/d15/f50 [1538587,19650] 0 2024-04-26T04:53:11.877 INFO:tasks.workunit.client.0.smithi077.stdout:2/638: creat d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/fdf x:0 0 0 2024-04-26T04:53:11.878 INFO:tasks.workunit.client.0.smithi077.stdout:2/639: chown d9/fbb 19487 1 2024-04-26T04:53:11.881 INFO:tasks.workunit.client.0.smithi077.stdout:3/516: rename d0/d9/f26 to d0/d7/d1f/d81/fa3 0 2024-04-26T04:53:11.885 INFO:tasks.workunit.client.0.smithi077.stdout:2/640: mkdir d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc4/de0 0 2024-04-26T04:53:11.885 INFO:tasks.workunit.client.0.smithi077.stdout:0/570: link d0/d19/d69/c7e d0/d19/d1e/d59/d96/cc5 0 2024-04-26T04:53:11.888 INFO:tasks.workunit.client.0.smithi077.stdout:2/641: symlink d9/dc/d6e/le1 0 2024-04-26T04:53:11.888 INFO:tasks.workunit.client.0.smithi077.stdout:3/517: dwrite d0/f1b [0,4194304] 0 2024-04-26T04:53:11.889 INFO:tasks.workunit.client.0.smithi077.stdout:3/518: fsync d0/f19 0 2024-04-26T04:53:11.899 INFO:tasks.workunit.client.0.smithi077.stdout:8/676: sync 2024-04-26T04:53:11.903 INFO:tasks.workunit.client.0.smithi077.stdout:0/571: dwrite d0/d19/d1e/d2f/d52/f62 [0,4194304] 0 2024-04-26T04:53:11.909 INFO:tasks.workunit.client.0.smithi077.stdout:3/519: creat d0/d9/d32/d2a/d85/fa4 x:0 0 0 2024-04-26T04:53:11.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/677: dwrite d4/df/d22/d2d/d2b/f7b [0,4194304] 0 2024-04-26T04:53:11.920 INFO:tasks.workunit.client.0.smithi077.stdout:2/642: dread d9/dc/d1d/d34/f36 [0,4194304] 0 2024-04-26T04:53:11.921 INFO:tasks.workunit.client.0.smithi077.stdout:2/643: fdatasync d9/dc/d1d/d1e/f13 0 2024-04-26T04:53:11.923 INFO:tasks.workunit.client.0.smithi077.stdout:3/520: link d0/d7/l83 d0/d7/d1f/d76/la5 0 2024-04-26T04:53:11.924 INFO:tasks.workunit.client.0.smithi077.stdout:3/521: fsync d0/d7/d1f/d4d/f82 0 2024-04-26T04:53:11.925 INFO:tasks.workunit.client.0.smithi077.stdout:2/644: truncate d9/dc/d1d/d34/d39/d63/f9c 912303 0 2024-04-26T04:53:11.927 INFO:tasks.workunit.client.0.smithi077.stdout:2/645: dread d9/dc/f7a [0,4194304] 0 2024-04-26T04:53:11.932 INFO:tasks.workunit.client.0.smithi077.stdout:8/678: fsync d4/df/d22/d2d/d2b/f7b 0 2024-04-26T04:53:11.936 INFO:tasks.workunit.client.0.smithi077.stdout:8/679: chown d4/df/d22/d2d/d2b/l77 3096642 1 2024-04-26T04:53:11.936 INFO:tasks.workunit.client.0.smithi077.stdout:2/646: dread d9/dc/d1d/d34/d39/d63/d79/f3f [0,4194304] 0 2024-04-26T04:53:11.939 INFO:tasks.workunit.client.0.smithi077.stdout:2/647: chown d9/dc/d1d/d1e/d77/fa2 4 1 2024-04-26T04:53:11.943 INFO:tasks.workunit.client.0.smithi077.stdout:0/572: dwrite d0/d47/f55 [0,4194304] 0 2024-04-26T04:53:11.944 INFO:tasks.workunit.client.0.smithi077.stdout:3/522: dread d0/d9/f30 [4194304,4194304] 0 2024-04-26T04:53:11.945 INFO:tasks.workunit.client.0.smithi077.stdout:2/648: dread - d9/d15/d44/f58 zero size 2024-04-26T04:53:11.950 INFO:tasks.workunit.client.0.smithi077.stdout:3/523: mknod d0/d7/d9a/ca6 0 2024-04-26T04:53:11.955 INFO:tasks.workunit.client.0.smithi077.stdout:0/573: dread d0/d47/f30 [0,4194304] 0 2024-04-26T04:53:11.961 INFO:tasks.workunit.client.0.smithi077.stdout:0/574: unlink d0/d47/l6b 0 2024-04-26T04:53:11.962 INFO:tasks.workunit.client.0.smithi077.stdout:8/680: dwrite d4/df/d18/d56/f72 [0,4194304] 0 2024-04-26T04:53:11.966 INFO:tasks.workunit.client.0.smithi077.stdout:8/681: creat d4/df/d22/d2d/d2b/d33/dd8/fe6 x:0 0 0 2024-04-26T04:53:11.967 INFO:tasks.workunit.client.0.smithi077.stdout:8/682: fdatasync d4/f5 0 2024-04-26T04:53:11.970 INFO:tasks.workunit.client.0.smithi077.stdout:0/575: dread d0/d19/d1e/d2f/d52/f76 [0,4194304] 0 2024-04-26T04:53:11.971 INFO:tasks.workunit.client.0.smithi077.stdout:3/524: truncate d0/d7/f54 2985935 0 2024-04-26T04:53:11.972 INFO:tasks.workunit.client.0.smithi077.stdout:3/525: read - d0/d9/f78 zero size 2024-04-26T04:53:11.972 INFO:tasks.workunit.client.0.smithi077.stdout:0/576: symlink d0/d19/d1e/d2f/da8/lc6 0 2024-04-26T04:53:11.974 INFO:tasks.workunit.client.0.smithi077.stdout:8/683: read d4/df/d22/d2d/fc1 [2290183,92689] 0 2024-04-26T04:53:11.976 INFO:tasks.workunit.client.0.smithi077.stdout:3/526: mknod d0/d9/d32/d29/ca7 0 2024-04-26T04:53:11.978 INFO:tasks.workunit.client.0.smithi077.stdout:0/577: symlink d0/d19/d1e/d2f/d90/lc7 0 2024-04-26T04:53:11.979 INFO:tasks.workunit.client.0.smithi077.stdout:3/527: symlink d0/d7/d15/la8 0 2024-04-26T04:53:11.980 INFO:tasks.workunit.client.0.smithi077.stdout:2/649: dread d9/dc/d1d/d34/d39/d63/d79/f30 [0,4194304] 0 2024-04-26T04:53:11.980 INFO:tasks.workunit.client.0.smithi077.stdout:3/528: dread - d0/d9/f78 zero size 2024-04-26T04:53:11.984 INFO:tasks.workunit.client.0.smithi077.stdout:2/650: truncate d9/dc/d1d/d34/f69 2815992 0 2024-04-26T04:53:11.999 INFO:tasks.workunit.client.0.smithi077.stdout:8/684: write d4/fd [3165925,21582] 0 2024-04-26T04:53:12.003 INFO:tasks.workunit.client.0.smithi077.stdout:0/578: write d0/fe [1313795,57414] 0 2024-04-26T04:53:12.007 INFO:tasks.workunit.client.0.smithi077.stdout:8/685: dwrite d4/df/d22/d2d/f1b [0,4194304] 0 2024-04-26T04:53:12.013 INFO:tasks.workunit.client.0.smithi077.stdout:0/579: dwrite d0/d47/f30 [0,4194304] 0 2024-04-26T04:53:12.018 INFO:tasks.workunit.client.0.smithi077.stdout:3/529: dread d0/d7/d15/f65 [0,4194304] 0 2024-04-26T04:53:12.018 INFO:tasks.workunit.client.0.smithi077.stdout:2/651: dwrite d9/dc/d1d/d1e/d77/fa2 [0,4194304] 0 2024-04-26T04:53:12.027 INFO:tasks.workunit.client.0.smithi077.stdout:2/652: dread d9/dc/d1d/d1e/d77/fa2 [0,4194304] 0 2024-04-26T04:53:12.069 INFO:tasks.workunit.client.0.smithi077.stdout:2/653: rename d9/dc/d1d/d34/d39/d63/d79/f94 to d9/dc/d1d/d1e/fe2 0 2024-04-26T04:53:12.071 INFO:tasks.workunit.client.0.smithi077.stdout:8/686: mknod d4/df/d6f/da3/dc7/dde/ce7 0 2024-04-26T04:53:12.076 INFO:tasks.workunit.client.0.smithi077.stdout:0/580: dwrite d0/d19/d1e/f20 [0,4194304] 0 2024-04-26T04:53:12.076 INFO:tasks.workunit.client.0.smithi077.stdout:7/669: sync 2024-04-26T04:53:12.076 INFO:tasks.workunit.client.0.smithi077.stdout:9/570: sync 2024-04-26T04:53:12.079 INFO:tasks.workunit.client.0.smithi077.stdout:8/687: rename d4/df/dce/fcb to d4/df/d22/d80/d86/fe8 0 2024-04-26T04:53:12.083 INFO:tasks.workunit.client.0.smithi077.stdout:2/654: link d9/dc/d1d/d34/d39/d63/d79/d5b/fcf d9/d15/d44/fe3 0 2024-04-26T04:53:12.089 INFO:tasks.workunit.client.0.smithi077.stdout:9/571: chown d9/d3a/d79/d50/l93 7 1 2024-04-26T04:53:12.091 INFO:tasks.workunit.client.0.smithi077.stdout:3/530: link d0/d9/d32/d2a/c5b d0/d9/ca9 0 2024-04-26T04:53:12.093 INFO:tasks.workunit.client.0.smithi077.stdout:8/688: symlink d4/df/dc6/le9 0 2024-04-26T04:53:12.094 INFO:tasks.workunit.client.0.smithi077.stdout:9/572: truncate d9/dd/d1b/d2b/f57 60943 0 2024-04-26T04:53:12.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/670: getdents da/d13/d22/d5d/d9f/da7/de0 0 2024-04-26T04:53:12.095 INFO:tasks.workunit.client.0.smithi077.stdout:9/573: chown d9/d7d/fa1 544808 1 2024-04-26T04:53:12.096 INFO:tasks.workunit.client.0.smithi077.stdout:8/689: mknod d4/df/dce/d5b/cea 0 2024-04-26T04:53:12.097 INFO:tasks.workunit.client.0.smithi077.stdout:0/581: creat d0/d47/fc8 x:0 0 0 2024-04-26T04:53:12.101 INFO:tasks.workunit.client.0.smithi077.stdout:6/663: sync 2024-04-26T04:53:12.101 INFO:tasks.workunit.client.0.smithi077.stdout:5/595: sync 2024-04-26T04:53:12.101 INFO:tasks.workunit.client.0.smithi077.stdout:1/542: sync 2024-04-26T04:53:12.102 INFO:tasks.workunit.client.0.smithi077.stdout:4/616: sync 2024-04-26T04:53:12.103 INFO:tasks.workunit.client.0.smithi077.stdout:9/574: rmdir d9/d7d/d94 39 2024-04-26T04:53:12.134 INFO:tasks.workunit.client.0.smithi077.stdout:4/617: dread d4/d17/d18/d43/d51/db4/d15/f40 [0,4194304] 0 2024-04-26T04:53:12.134 INFO:tasks.workunit.client.0.smithi077.stdout:4/618: chown d4/d17/d18/d43/d51/db4/d15/d1f/d76/f87 5 1 2024-04-26T04:53:12.184 INFO:tasks.workunit.client.0.smithi077.stdout:0/582: rename d0/f9 to d0/d47/d68/d9f/fc9 0 2024-04-26T04:53:12.186 INFO:tasks.workunit.client.0.smithi077.stdout:0/583: write d0/d19/d1e/d2f/f44 [1743307,129758] 0 2024-04-26T04:53:12.187 INFO:tasks.workunit.client.0.smithi077.stdout:3/531: truncate d0/d9/d32/d2a/f40 1555128 0 2024-04-26T04:53:12.188 INFO:tasks.workunit.client.0.smithi077.stdout:4/619: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/d25/fcb [0,4194304] 0 2024-04-26T04:53:12.189 INFO:tasks.workunit.client.0.smithi077.stdout:6/664: mkdir d8/d1c/da4/de7 0 2024-04-26T04:53:12.190 INFO:tasks.workunit.client.0.smithi077.stdout:6/665: chown d8/fa1 49 1 2024-04-26T04:53:12.191 INFO:tasks.workunit.client.0.smithi077.stdout:5/596: fdatasync d1/d42/da4/fa0 0 2024-04-26T04:53:12.194 INFO:tasks.workunit.client.0.smithi077.stdout:2/655: link d9/dc/d1d/d1e/c4d d9/dc/d1d/d1e/d77/ce4 0 2024-04-26T04:53:12.201 INFO:tasks.workunit.client.0.smithi077.stdout:4/620: write f2 [2566259,87991] 0 2024-04-26T04:53:12.203 INFO:tasks.workunit.client.0.smithi077.stdout:1/543: dread d9/f1d [0,4194304] 0 2024-04-26T04:53:12.203 INFO:tasks.workunit.client.0.smithi077.stdout:8/690: write d4/df/d22/d2d/f30 [904761,118509] 0 2024-04-26T04:53:12.208 INFO:tasks.workunit.client.0.smithi077.stdout:2/656: write d9/dc/d1d/d34/d39/d63/fca [853487,43127] 0 2024-04-26T04:53:12.211 INFO:tasks.workunit.client.0.smithi077.stdout:3/532: link d0/d9/f30 d0/d9/d32/d2a/faa 0 2024-04-26T04:53:12.215 INFO:tasks.workunit.client.0.smithi077.stdout:2/657: dwrite d9/dc/f41 [0,4194304] 0 2024-04-26T04:53:12.216 INFO:tasks.workunit.client.0.smithi077.stdout:5/597: rename d1/d12/d28/d38/l9b to d1/d8d/ld8 0 2024-04-26T04:53:12.219 INFO:tasks.workunit.client.0.smithi077.stdout:0/584: rmdir d0/d47/d93/d94/da9 0 2024-04-26T04:53:12.227 INFO:tasks.workunit.client.0.smithi077.stdout:5/598: dwrite d1/d42/d62/f9e [0,4194304] 0 2024-04-26T04:53:12.227 INFO:tasks.workunit.client.0.smithi077.stdout:8/691: write d4/d69/fa2 [6304014,21322] 0 2024-04-26T04:53:12.228 INFO:tasks.workunit.client.0.smithi077.stdout:6/666: dwrite d8/d52/da0/f77 [4194304,4194304] 0 2024-04-26T04:53:12.232 INFO:tasks.workunit.client.0.smithi077.stdout:3/533: dread d0/d7/d1f/f72 [0,4194304] 0 2024-04-26T04:53:12.233 INFO:tasks.workunit.client.0.smithi077.stdout:2/658: unlink d9/d15/c71 0 2024-04-26T04:53:12.237 INFO:tasks.workunit.client.0.smithi077.stdout:6/667: dread d8/d52/d53/f80 [0,4194304] 0 2024-04-26T04:53:12.238 INFO:tasks.workunit.client.0.smithi077.stdout:6/668: chown d8/d1c/f24 40253299 1 2024-04-26T04:53:12.238 INFO:tasks.workunit.client.0.smithi077.stdout:0/585: creat d0/d47/d64/d97/dba/fca x:0 0 0 2024-04-26T04:53:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:11 smithi077 ceph-mon[26311]: pgmap v25: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 198 MiB/s wr, 418 op/s 2024-04-26T04:53:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:11 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:12.244 INFO:tasks.workunit.client.0.smithi077.stdout:8/692: rmdir d4/df/d22/d2d/d2b/d33 39 2024-04-26T04:53:12.244 INFO:tasks.workunit.client.0.smithi077.stdout:1/544: dwrite d9/db/f13 [0,4194304] 0 2024-04-26T04:53:12.249 INFO:tasks.workunit.client.0.smithi077.stdout:5/599: unlink d1/d42/faf 0 2024-04-26T04:53:12.251 INFO:tasks.workunit.client.0.smithi077.stdout:2/659: rename d9/dc/d5f/d8e/da6/fb3 to d9/dc/d5f/d8e/db9/fe5 0 2024-04-26T04:53:12.251 INFO:tasks.workunit.client.0.smithi077.stdout:6/669: dread - d8/d2a/d86/fe0 zero size 2024-04-26T04:53:12.252 INFO:tasks.workunit.client.0.smithi077.stdout:0/586: symlink d0/d19/d1e/d2f/d90/lcb 0 2024-04-26T04:53:12.253 INFO:tasks.workunit.client.0.smithi077.stdout:1/545: chown d9/db/dd/l17 1686 1 2024-04-26T04:53:12.254 INFO:tasks.workunit.client.0.smithi077.stdout:1/546: read - d9/db/dd/d34/d4a/f92 zero size 2024-04-26T04:53:12.259 INFO:tasks.workunit.client.0.smithi077.stdout:0/587: dwrite d0/d19/d1e/d2f/f3b [0,4194304] 0 2024-04-26T04:53:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:11 smithi078 ceph-mon[33746]: pgmap v25: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 198 MiB/s wr, 418 op/s 2024-04-26T04:53:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:11 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:12.268 INFO:tasks.workunit.client.0.smithi077.stdout:2/660: dread d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:53:12.269 INFO:tasks.workunit.client.0.smithi077.stdout:2/661: chown d9/dc/d1d/d34/d39/d63/d79/d5b/db8 52 1 2024-04-26T04:53:12.273 INFO:tasks.workunit.client.0.smithi077.stdout:8/693: dread d4/df/d18/d56/f7c [0,4194304] 0 2024-04-26T04:53:12.305 INFO:tasks.workunit.client.0.smithi077.stdout:5/600: unlink d1/d30/d33/l6e 0 2024-04-26T04:53:12.313 INFO:tasks.workunit.client.0.smithi077.stdout:0/588: mknod d0/d19/ccc 0 2024-04-26T04:53:12.314 INFO:tasks.workunit.client.0.smithi077.stdout:2/662: mkdir d9/d15/d56/de6 0 2024-04-26T04:53:12.317 INFO:tasks.workunit.client.0.smithi077.stdout:8/694: stat d4/df/d22/d2d/f4a 0 2024-04-26T04:53:12.318 INFO:tasks.workunit.client.0.smithi077.stdout:3/534: write d0/d7/d1f/f60 [360839,80700] 0 2024-04-26T04:53:12.318 INFO:tasks.workunit.client.0.smithi077.stdout:8/695: readlink d4/d17/l45 0 2024-04-26T04:53:12.319 INFO:tasks.workunit.client.0.smithi077.stdout:8/696: stat d4/df/dce/la4 0 2024-04-26T04:53:12.322 INFO:tasks.workunit.client.0.smithi077.stdout:1/547: write d9/db/d2b/f61 [3179851,129061] 0 2024-04-26T04:53:12.323 INFO:tasks.workunit.client.0.smithi077.stdout:5/601: dread d1/d30/d35/f6c [0,4194304] 0 2024-04-26T04:53:12.326 INFO:tasks.workunit.client.0.smithi077.stdout:0/589: unlink d0/d47/f16 0 2024-04-26T04:53:12.326 INFO:tasks.workunit.client.0.smithi077.stdout:5/602: dread - d1/d12/d28/d38/f7c zero size 2024-04-26T04:53:12.326 INFO:tasks.workunit.client.0.smithi077.stdout:6/670: dwrite d8/d52/f87 [0,4194304] 0 2024-04-26T04:53:12.327 INFO:tasks.workunit.client.0.smithi077.stdout:5/603: readlink d1/d12/d28/d38/l91 0 2024-04-26T04:53:12.329 INFO:tasks.workunit.client.0.smithi077.stdout:8/697: fdatasync d4/df/d22/d80/fe4 0 2024-04-26T04:53:12.333 INFO:tasks.workunit.client.0.smithi077.stdout:2/663: dwrite d9/dc/d1d/d34/d39/d63/d79/f54 [0,4194304] 0 2024-04-26T04:53:12.335 INFO:tasks.workunit.client.0.smithi077.stdout:6/671: dwrite d8/d52/da0/f77 [4194304,4194304] 0 2024-04-26T04:53:12.341 INFO:tasks.workunit.client.0.smithi077.stdout:2/664: dwrite d9/dc/d1d/d34/d39/d63/d79/f54 [0,4194304] 0 2024-04-26T04:53:12.342 INFO:tasks.workunit.client.0.smithi077.stdout:2/665: fdatasync d9/d15/d44/fa8 0 2024-04-26T04:53:12.343 INFO:tasks.workunit.client.0.smithi077.stdout:1/548: creat d9/db/dd/d34/d63/f9e x:0 0 0 2024-04-26T04:53:12.345 INFO:tasks.workunit.client.0.smithi077.stdout:2/666: read d9/dc/d1d/d1e/f5e [28962,50391] 0 2024-04-26T04:53:12.354 INFO:tasks.workunit.client.0.smithi077.stdout:3/535: symlink d0/d7/d1f/lab 0 2024-04-26T04:53:12.355 INFO:tasks.workunit.client.0.smithi077.stdout:5/604: creat d1/d42/d62/dc8/fd9 x:0 0 0 2024-04-26T04:53:12.360 INFO:tasks.workunit.client.0.smithi077.stdout:1/549: mknod d9/d53/d8b/c9f 0 2024-04-26T04:53:12.363 INFO:tasks.workunit.client.0.smithi077.stdout:2/667: mknod d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/ce7 0 2024-04-26T04:53:12.367 INFO:tasks.workunit.client.0.smithi077.stdout:8/698: rename d4/df/f46 to d4/df/d22/d2d/d2b/d33/dbc/feb 0 2024-04-26T04:53:12.369 INFO:tasks.workunit.client.0.smithi077.stdout:1/550: fdatasync d9/db/f60 0 2024-04-26T04:53:12.369 INFO:tasks.workunit.client.0.smithi077.stdout:1/551: readlink d9/db/dd/l26 0 2024-04-26T04:53:12.370 INFO:tasks.workunit.client.0.smithi077.stdout:3/536: symlink d0/d9/d32/lac 0 2024-04-26T04:53:12.372 INFO:tasks.workunit.client.0.smithi077.stdout:5/605: unlink d1/c47 0 2024-04-26T04:53:12.378 INFO:tasks.workunit.client.0.smithi077.stdout:2/668: rename d9/dc/d5f/l6d to d9/dc/d5f/d8e/dcd/le8 0 2024-04-26T04:53:12.378 INFO:tasks.workunit.client.0.smithi077.stdout:2/669: chown d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/l7c 1 1 2024-04-26T04:53:12.379 INFO:tasks.workunit.client.0.smithi077.stdout:3/537: rmdir d0/d7/d9a 39 2024-04-26T04:53:12.386 INFO:tasks.workunit.client.0.smithi077.stdout:0/590: dwrite d0/d47/f31 [0,4194304] 0 2024-04-26T04:53:12.387 INFO:tasks.workunit.client.0.smithi077.stdout:1/552: link d9/d53/d6e/l78 d9/db/d2b/d75/la0 0 2024-04-26T04:53:12.389 INFO:tasks.workunit.client.0.smithi077.stdout:7/671: sync 2024-04-26T04:53:12.390 INFO:tasks.workunit.client.0.smithi077.stdout:2/670: mkdir d9/dc/d1d/d34/d39/d63/de9 0 2024-04-26T04:53:12.392 INFO:tasks.workunit.client.0.smithi077.stdout:8/699: getdents d4/d69/d73 0 2024-04-26T04:53:12.393 INFO:tasks.workunit.client.0.smithi077.stdout:0/591: creat d0/d47/d93/d94/fcd x:0 0 0 2024-04-26T04:53:12.394 INFO:tasks.workunit.client.0.smithi077.stdout:0/592: truncate d0/d47/fc8 595465 0 2024-04-26T04:53:12.395 INFO:tasks.workunit.client.0.smithi077.stdout:0/593: readlink d0/d19/d1e/d2f/d52/l58 0 2024-04-26T04:53:12.396 INFO:tasks.workunit.client.0.smithi077.stdout:6/672: write f4 [3301896,9626] 0 2024-04-26T04:53:12.397 INFO:tasks.workunit.client.0.smithi077.stdout:6/673: chown d8/d1c/c44 0 1 2024-04-26T04:53:12.399 INFO:tasks.workunit.client.0.smithi077.stdout:8/700: write d4/df/d22/d2d/fc1 [4119643,129965] 0 2024-04-26T04:53:12.399 INFO:tasks.workunit.client.0.smithi077.stdout:0/594: rmdir d0/d47/d64 39 2024-04-26T04:53:12.401 INFO:tasks.workunit.client.0.smithi077.stdout:2/671: creat d9/dc/d1d/d34/d39/d63/de9/fea x:0 0 0 2024-04-26T04:53:12.402 INFO:tasks.workunit.client.0.smithi077.stdout:7/672: dread da/d15/d1a/fb5 [0,4194304] 0 2024-04-26T04:53:12.402 INFO:tasks.workunit.client.0.smithi077.stdout:6/674: mkdir d8/d52/d53/d5e/da6/de8 0 2024-04-26T04:53:12.404 INFO:tasks.workunit.client.0.smithi077.stdout:7/673: chown da/d13/d22/d5d/d9f/lca 132985330 1 2024-04-26T04:53:12.404 INFO:tasks.workunit.client.0.smithi077.stdout:8/701: unlink d4/df/d6f/fb4 0 2024-04-26T04:53:12.405 INFO:tasks.workunit.client.0.smithi077.stdout:8/702: fdatasync d4/df/dce/d5b/fa5 0 2024-04-26T04:53:12.406 INFO:tasks.workunit.client.0.smithi077.stdout:2/672: chown d9/dc/d1d/d34/d39/d63/d79/d2b/c3e 112 1 2024-04-26T04:53:12.408 INFO:tasks.workunit.client.0.smithi077.stdout:1/553: getdents d9/db/dd/d34 0 2024-04-26T04:53:12.409 INFO:tasks.workunit.client.0.smithi077.stdout:1/554: readlink d9/d53/d6e/d7c/l8c 0 2024-04-26T04:53:12.410 INFO:tasks.workunit.client.0.smithi077.stdout:2/673: dread d9/f17 [0,4194304] 0 2024-04-26T04:53:12.410 INFO:tasks.workunit.client.0.smithi077.stdout:6/675: mkdir d8/d52/d53/d5e/da6/de9 0 2024-04-26T04:53:12.411 INFO:tasks.workunit.client.0.smithi077.stdout:2/674: write d9/f2c [7270840,71753] 0 2024-04-26T04:53:12.415 INFO:tasks.workunit.client.0.smithi077.stdout:8/703: rmdir d4/df/d22/d80/d86 39 2024-04-26T04:53:12.417 INFO:tasks.workunit.client.0.smithi077.stdout:1/555: mkdir d9/db/d2b/d75/da1 0 2024-04-26T04:53:12.419 INFO:tasks.workunit.client.0.smithi077.stdout:1/556: chown d9/db/d2b/d75/f91 3425225 1 2024-04-26T04:53:12.420 INFO:tasks.workunit.client.0.smithi077.stdout:3/538: write d0/d7/d15/f20 [5232162,34494] 0 2024-04-26T04:53:12.422 INFO:tasks.workunit.client.0.smithi077.stdout:6/676: creat d8/d52/d53/da7/fea x:0 0 0 2024-04-26T04:53:12.423 INFO:tasks.workunit.client.0.smithi077.stdout:5/606: dwrite d1/d12/d28/d38/f7c [0,4194304] 0 2024-04-26T04:53:12.425 INFO:tasks.workunit.client.0.smithi077.stdout:7/674: symlink da/d13/le1 0 2024-04-26T04:53:12.427 INFO:tasks.workunit.client.0.smithi077.stdout:2/675: symlink d9/d15/d91/leb 0 2024-04-26T04:53:12.432 INFO:tasks.workunit.client.0.smithi077.stdout:0/595: rename d0/d19/d1e/d2f/d90 to d0/d47/dce 0 2024-04-26T04:53:12.433 INFO:tasks.workunit.client.0.smithi077.stdout:4/621: sync 2024-04-26T04:53:12.433 INFO:tasks.workunit.client.0.smithi077.stdout:9/575: sync 2024-04-26T04:53:12.441 INFO:tasks.workunit.client.0.smithi077.stdout:6/677: fsync d8/d1c/f4c 0 2024-04-26T04:53:12.445 INFO:tasks.workunit.client.0.smithi077.stdout:5/607: rmdir d1/d12/d28 39 2024-04-26T04:53:12.452 INFO:tasks.workunit.client.0.smithi077.stdout:1/557: link d9/db/dd/d68/f89 d9/d53/d8b/fa2 0 2024-04-26T04:53:12.453 INFO:tasks.workunit.client.0.smithi077.stdout:8/704: dread d4/df/f15 [0,4194304] 0 2024-04-26T04:53:12.454 INFO:tasks.workunit.client.0.smithi077.stdout:8/705: chown d4/df/d22/d2d/d2b/f5d 723 1 2024-04-26T04:53:12.454 INFO:tasks.workunit.client.0.smithi077.stdout:8/706: dread - d4/d17/d90/fd1 zero size 2024-04-26T04:53:12.455 INFO:tasks.workunit.client.0.smithi077.stdout:6/678: rename d8/d52/f6e to d8/d52/d53/d5e/d6c/feb 0 2024-04-26T04:53:12.456 INFO:tasks.workunit.client.0.smithi077.stdout:8/707: write d4/d69/d73/fb5 [126280,30678] 0 2024-04-26T04:53:12.458 INFO:tasks.workunit.client.0.smithi077.stdout:5/608: creat d1/d42/da4/d25/fda x:0 0 0 2024-04-26T04:53:12.459 INFO:tasks.workunit.client.0.smithi077.stdout:5/609: write d1/d8/d7e/fc6 [879230,107945] 0 2024-04-26T04:53:12.460 INFO:tasks.workunit.client.0.smithi077.stdout:7/675: link da/d13/d22/f46 da/d15/fe2 0 2024-04-26T04:53:12.461 INFO:tasks.workunit.client.0.smithi077.stdout:9/576: mknod d9/dd/d1b/cc6 0 2024-04-26T04:53:12.462 INFO:tasks.workunit.client.0.smithi077.stdout:0/596: mknod d0/d47/d64/d97/dba/dc4/ccf 0 2024-04-26T04:53:12.466 INFO:tasks.workunit.client.0.smithi077.stdout:1/558: symlink d9/d53/d6e/d7c/la3 0 2024-04-26T04:53:12.471 INFO:tasks.workunit.client.0.smithi077.stdout:6/679: chown d8/d2a/l2d 3466 1 2024-04-26T04:53:12.472 INFO:tasks.workunit.client.0.smithi077.stdout:5/610: dread d1/d12/d45/f63 [0,4194304] 0 2024-04-26T04:53:12.473 INFO:tasks.workunit.client.0.smithi077.stdout:5/611: write d1/d42/da4/f34 [4416738,74583] 0 2024-04-26T04:53:12.475 INFO:tasks.workunit.client.0.smithi077.stdout:9/577: chown d9/dd/c2a 18 1 2024-04-26T04:53:12.478 INFO:tasks.workunit.client.0.smithi077.stdout:0/597: symlink d0/d47/d64/d97/dba/dc4/ld0 0 2024-04-26T04:53:12.479 INFO:tasks.workunit.client.0.smithi077.stdout:0/598: write d0/d47/d68/fa5 [2114215,63343] 0 2024-04-26T04:53:12.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/559: creat d9/db/d2b/d3f/fa4 x:0 0 0 2024-04-26T04:53:12.486 INFO:tasks.workunit.client.0.smithi077.stdout:3/539: getdents d0/d9/d32/d2a/d85 0 2024-04-26T04:53:12.493 INFO:tasks.workunit.client.0.smithi077.stdout:4/622: truncate d4/d17/d18/d43/d51/db4/d15/d1f/f34 1430910 0 2024-04-26T04:53:12.501 INFO:tasks.workunit.client.0.smithi077.stdout:1/560: creat d9/d53/d6e/d7c/fa5 x:0 0 0 2024-04-26T04:53:12.503 INFO:tasks.workunit.client.0.smithi077.stdout:9/578: dwrite d9/d3a/d79/d50/f9a [0,4194304] 0 2024-04-26T04:53:12.504 INFO:tasks.workunit.client.0.smithi077.stdout:8/708: rename d4/df/d6f/da3/lb7 to d4/df/dce/lec 0 2024-04-26T04:53:12.505 INFO:tasks.workunit.client.0.smithi077.stdout:5/612: mkdir d1/d42/d62/dc8/dd5/ddb 0 2024-04-26T04:53:12.515 INFO:tasks.workunit.client.0.smithi077.stdout:6/680: creat d8/d2a/fec x:0 0 0 2024-04-26T04:53:12.517 INFO:tasks.workunit.client.0.smithi077.stdout:8/709: mknod d4/d17/d90/ced 0 2024-04-26T04:53:12.518 INFO:tasks.workunit.client.0.smithi077.stdout:9/579: unlink d9/dd/d3d/f47 0 2024-04-26T04:53:12.523 INFO:tasks.workunit.client.0.smithi077.stdout:3/540: link d0/d7/d15/f44 d0/d7/fad 0 2024-04-26T04:53:12.523 INFO:tasks.workunit.client.0.smithi077.stdout:8/710: dwrite d4/df/d22/d2d/d2b/f5d [4194304,4194304] 0 2024-04-26T04:53:12.524 INFO:tasks.workunit.client.0.smithi077.stdout:0/599: truncate d0/d2c/f88 3509643 0 2024-04-26T04:53:12.525 INFO:tasks.workunit.client.0.smithi077.stdout:3/541: write d0/d7/d1f/f51 [67409,77884] 0 2024-04-26T04:53:12.527 INFO:tasks.workunit.client.0.smithi077.stdout:6/681: mkdir d8/ded 0 2024-04-26T04:53:12.529 INFO:tasks.workunit.client.0.smithi077.stdout:5/613: read d1/d12/d28/f2e [2038146,4023] 0 2024-04-26T04:53:12.530 INFO:tasks.workunit.client.0.smithi077.stdout:5/614: fdatasync d1/d30/f36 0 2024-04-26T04:53:12.530 INFO:tasks.workunit.client.0.smithi077.stdout:4/623: creat d4/d17/d18/d43/d51/db4/d15/d1f/fd3 x:0 0 0 2024-04-26T04:53:12.532 INFO:tasks.workunit.client.0.smithi077.stdout:5/615: stat d1/d8/l2f 0 2024-04-26T04:53:12.532 INFO:tasks.workunit.client.0.smithi077.stdout:8/711: dwrite d4/d87/fb2 [0,4194304] 0 2024-04-26T04:53:12.536 INFO:tasks.workunit.client.0.smithi077.stdout:0/600: mknod d0/d19/d8d/cd1 0 2024-04-26T04:53:12.537 INFO:tasks.workunit.client.0.smithi077.stdout:0/601: chown d0/d28/l33 131045 1 2024-04-26T04:53:12.539 INFO:tasks.workunit.client.0.smithi077.stdout:1/561: dwrite d9/db/d2b/f3d [0,4194304] 0 2024-04-26T04:53:12.542 INFO:tasks.workunit.client.0.smithi077.stdout:7/676: rename da/d13/fd0 to da/d6b/d63/fe3 0 2024-04-26T04:53:12.542 INFO:tasks.workunit.client.0.smithi077.stdout:1/562: chown d9/db/dd/d34/d4a/f59 1630753 1 2024-04-26T04:53:12.544 INFO:tasks.workunit.client.0.smithi077.stdout:6/682: rmdir d8/d8f 39 2024-04-26T04:53:12.547 INFO:tasks.workunit.client.0.smithi077.stdout:3/542: dwrite d0/d7/d1f/f72 [0,4194304] 0 2024-04-26T04:53:12.549 INFO:tasks.workunit.client.0.smithi077.stdout:0/602: dwrite d0/d2c/d4e/f4c [0,4194304] 0 2024-04-26T04:53:12.550 INFO:tasks.workunit.client.0.smithi077.stdout:4/624: stat d4/l5f 0 2024-04-26T04:53:12.556 INFO:tasks.workunit.client.0.smithi077.stdout:8/712: creat d4/df/dce/d5b/d8d/fee x:0 0 0 2024-04-26T04:53:12.557 INFO:tasks.workunit.client.0.smithi077.stdout:8/713: stat d4/d69/d73/fdb 0 2024-04-26T04:53:12.559 INFO:tasks.workunit.client.0.smithi077.stdout:8/714: write d4/df/f32 [291379,37710] 0 2024-04-26T04:53:12.561 INFO:tasks.workunit.client.0.smithi077.stdout:8/715: chown d4/df/d22/d2d/f4a 4 1 2024-04-26T04:53:12.562 INFO:tasks.workunit.client.0.smithi077.stdout:6/683: fsync d8/d52/fb3 0 2024-04-26T04:53:12.569 INFO:tasks.workunit.client.0.smithi077.stdout:9/580: rename d9/dd/la4 to d9/d35/d99/d9f/lc7 0 2024-04-26T04:53:12.575 INFO:tasks.workunit.client.0.smithi077.stdout:0/603: mkdir d0/d19/d1e/dd2 0 2024-04-26T04:53:12.578 INFO:tasks.workunit.client.0.smithi077.stdout:9/581: chown d9/dd/le 442 1 2024-04-26T04:53:12.582 INFO:tasks.workunit.client.0.smithi077.stdout:1/563: rename d9/d53/d8b/f8e to d9/db/d2b/d75/fa6 0 2024-04-26T04:53:12.585 INFO:tasks.workunit.client.0.smithi077.stdout:0/604: rmdir d0/d28/d40 39 2024-04-26T04:53:12.586 INFO:tasks.workunit.client.0.smithi077.stdout:7/677: dread da/d15/d42/f77 [0,4194304] 0 2024-04-26T04:53:12.586 INFO:tasks.workunit.client.0.smithi077.stdout:5/616: dwrite d1/d12/d28/d38/f8b [0,4194304] 0 2024-04-26T04:53:12.589 INFO:tasks.workunit.client.0.smithi077.stdout:9/582: mkdir d9/dd/d3d/d7c/dc8 0 2024-04-26T04:53:12.589 INFO:tasks.workunit.client.0.smithi077.stdout:5/617: dread d1/f5 [4194304,4194304] 0 2024-04-26T04:53:12.592 INFO:tasks.workunit.client.0.smithi077.stdout:5/618: stat d1/d42/da4/fa0 0 2024-04-26T04:53:12.593 INFO:tasks.workunit.client.0.smithi077.stdout:8/716: dread d4/d87/fbe [0,4194304] 0 2024-04-26T04:53:12.595 INFO:tasks.workunit.client.0.smithi077.stdout:1/564: unlink d9/db/dd/l2e 0 2024-04-26T04:53:12.597 INFO:tasks.workunit.client.0.smithi077.stdout:8/717: write d4/df/dce/d5b/d8d/fd7 [318093,112428] 0 2024-04-26T04:53:12.598 INFO:tasks.workunit.client.0.smithi077.stdout:6/684: dread d8/d2a/f66 [0,4194304] 0 2024-04-26T04:53:12.599 INFO:tasks.workunit.client.0.smithi077.stdout:1/565: chown d9/d53/d6e/d7c/la3 1 1 2024-04-26T04:53:12.599 INFO:tasks.workunit.client.0.smithi077.stdout:8/718: read d4/df/f15 [1445893,37723] 0 2024-04-26T04:53:12.600 INFO:tasks.workunit.client.0.smithi077.stdout:7/678: dwrite da/d13/fcb [0,4194304] 0 2024-04-26T04:53:12.602 INFO:tasks.workunit.client.0.smithi077.stdout:1/566: read - d9/db/dd/d68/d81/f88 zero size 2024-04-26T04:53:12.602 INFO:tasks.workunit.client.0.smithi077.stdout:7/679: fsync da/d15/faa 0 2024-04-26T04:53:12.603 INFO:tasks.workunit.client.0.smithi077.stdout:3/543: write d0/d9/d32/d2a/f40 [2028731,130080] 0 2024-04-26T04:53:12.605 INFO:tasks.workunit.client.0.smithi077.stdout:4/625: rename d4/d17/f3d to d4/d4e/fd4 0 2024-04-26T04:53:12.610 INFO:tasks.workunit.client.0.smithi077.stdout:5/619: unlink d1/d30/d33/cc9 0 2024-04-26T04:53:12.617 INFO:tasks.workunit.client.0.smithi077.stdout:7/680: unlink da/d15/d42/l6e 0 2024-04-26T04:53:12.623 INFO:tasks.workunit.client.0.smithi077.stdout:9/583: write d9/dd/d1b/d2b/f72 [609419,56991] 0 2024-04-26T04:53:12.625 INFO:tasks.workunit.client.0.smithi077.stdout:0/605: dwrite d0/d47/f8c [0,4194304] 0 2024-04-26T04:53:12.626 INFO:tasks.workunit.client.0.smithi077.stdout:5/620: rename d1/d30/d35/f6c to d1/d42/d62/dc8/fdc 0 2024-04-26T04:53:12.628 INFO:tasks.workunit.client.0.smithi077.stdout:5/621: read d1/d42/d62/f6f [1002885,33220] 0 2024-04-26T04:53:12.628 INFO:tasks.workunit.client.0.smithi077.stdout:6/685: creat d8/d52/d53/d5e/da6/ddf/fee x:0 0 0 2024-04-26T04:53:12.631 INFO:tasks.workunit.client.0.smithi077.stdout:5/622: chown d1/d12/d28/d38/fb5 15233113 1 2024-04-26T04:53:12.632 INFO:tasks.workunit.client.0.smithi077.stdout:7/681: mknod da/d15/ce4 0 2024-04-26T04:53:12.634 INFO:tasks.workunit.client.0.smithi077.stdout:1/567: mkdir d9/db/d2b/d75/da1/da7 0 2024-04-26T04:53:12.636 INFO:tasks.workunit.client.0.smithi077.stdout:4/626: fdatasync d4/d17/d18/d43/d53/fbe 0 2024-04-26T04:53:12.637 INFO:tasks.workunit.client.0.smithi077.stdout:3/544: write d0/d9/f30 [2042204,94267] 0 2024-04-26T04:53:12.642 INFO:tasks.workunit.client.0.smithi077.stdout:0/606: creat d0/d47/d64/d97/dba/fd3 x:0 0 0 2024-04-26T04:53:12.644 INFO:tasks.workunit.client.0.smithi077.stdout:3/545: dwrite d0/d9/f78 [0,4194304] 0 2024-04-26T04:53:12.649 INFO:tasks.workunit.client.0.smithi077.stdout:2/676: sync 2024-04-26T04:53:12.651 INFO:tasks.workunit.client.0.smithi077.stdout:5/623: dread d1/d12/d28/f9f [0,4194304] 0 2024-04-26T04:53:12.652 INFO:tasks.workunit.client.0.smithi077.stdout:7/682: write da/f59 [1993206,78446] 0 2024-04-26T04:53:12.653 INFO:tasks.workunit.client.0.smithi077.stdout:8/719: dwrite d4/df/d22/d2d/d2b/f54 [0,4194304] 0 2024-04-26T04:53:12.654 INFO:tasks.workunit.client.0.smithi077.stdout:7/683: fdatasync da/d6b/fc3 0 2024-04-26T04:53:12.655 INFO:tasks.workunit.client.0.smithi077.stdout:1/568: write d9/db/d2b/d75/fa6 [48809,128195] 0 2024-04-26T04:53:12.658 INFO:tasks.workunit.client.0.smithi077.stdout:7/684: write da/d15/d42/d8c/f9b [3347578,25547] 0 2024-04-26T04:53:12.660 INFO:tasks.workunit.client.0.smithi077.stdout:0/607: mkdir d0/d19/d1e/d2f/da8/dd4 0 2024-04-26T04:53:12.663 INFO:tasks.workunit.client.0.smithi077.stdout:6/686: dwrite d8/d55/f9b [0,4194304] 0 2024-04-26T04:53:12.668 INFO:tasks.workunit.client.0.smithi077.stdout:2/677: fdatasync d9/dc/d1d/d34/f36 0 2024-04-26T04:53:12.673 INFO:tasks.workunit.client.0.smithi077.stdout:5/624: write d1/d42/da4/d25/f4b [2164012,91513] 0 2024-04-26T04:53:12.678 INFO:tasks.workunit.client.0.smithi077.stdout:1/569: rename d9/db/d2b/d75/da1 to d9/d53/d8b/d93/da8 0 2024-04-26T04:53:12.681 INFO:tasks.workunit.client.0.smithi077.stdout:7/685: fdatasync da/d13/d22/d5d/f66 0 2024-04-26T04:53:12.683 INFO:tasks.workunit.client.0.smithi077.stdout:3/546: mkdir d0/dae 0 2024-04-26T04:53:12.685 INFO:tasks.workunit.client.0.smithi077.stdout:0/608: unlink d0/d19/f49 0 2024-04-26T04:53:12.690 INFO:tasks.workunit.client.0.smithi077.stdout:4/627: creat d4/d17/d18/fd5 x:0 0 0 2024-04-26T04:53:12.691 INFO:tasks.workunit.client.0.smithi077.stdout:4/628: write d4/d17/d18/fd5 [575992,82759] 0 2024-04-26T04:53:12.692 INFO:tasks.workunit.client.0.smithi077.stdout:4/629: chown d4/d17/d18/l73 460829 1 2024-04-26T04:53:12.693 INFO:tasks.workunit.client.0.smithi077.stdout:5/625: fdatasync d1/d12/d28/d38/f90 0 2024-04-26T04:53:12.694 INFO:tasks.workunit.client.0.smithi077.stdout:5/626: fdatasync d1/d8/f95 0 2024-04-26T04:53:12.696 INFO:tasks.workunit.client.0.smithi077.stdout:9/584: getdents d9/dd 0 2024-04-26T04:53:12.697 INFO:tasks.workunit.client.0.smithi077.stdout:2/678: rename d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/ce7 to d9/dc/d6e/cec 0 2024-04-26T04:53:12.707 INFO:tasks.workunit.client.0.smithi077.stdout:6/687: creat d8/d52/d53/d5e/da6/de6/fef x:0 0 0 2024-04-26T04:53:12.714 INFO:tasks.workunit.client.0.smithi077.stdout:4/630: creat d4/d17/d18/d43/d6e/d6c/fd6 x:0 0 0 2024-04-26T04:53:12.716 INFO:tasks.workunit.client.0.smithi077.stdout:0/609: write d0/f3 [5822923,12536] 0 2024-04-26T04:53:12.717 INFO:tasks.workunit.client.0.smithi077.stdout:7/686: dread da/d13/d22/f2e [0,4194304] 0 2024-04-26T04:53:12.718 INFO:tasks.workunit.client.0.smithi077.stdout:5/627: chown d1/d12/d28/l97 1 1 2024-04-26T04:53:12.720 INFO:tasks.workunit.client.0.smithi077.stdout:9/585: fsync d9/d7d/fa1 0 2024-04-26T04:53:12.723 INFO:tasks.workunit.client.0.smithi077.stdout:2/679: mknod d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6/dc9/dd0/ced 0 2024-04-26T04:53:12.723 INFO:tasks.workunit.client.0.smithi077.stdout:9/586: dread d9/d3a/d79/f6b [0,4194304] 0 2024-04-26T04:53:12.729 INFO:tasks.workunit.client.0.smithi077.stdout:6/688: creat d8/d52/d53/ff0 x:0 0 0 2024-04-26T04:53:12.737 INFO:tasks.workunit.client.0.smithi077.stdout:5/628: chown d1/d42/da4/d84/ccc 214 1 2024-04-26T04:53:12.737 INFO:tasks.workunit.client.0.smithi077.stdout:5/629: chown d1/d42/da4/d84/c86 1076552200 1 2024-04-26T04:53:12.738 INFO:tasks.workunit.client.0.smithi077.stdout:5/630: chown d1/d30/d35/l54 802 1 2024-04-26T04:53:12.744 INFO:tasks.workunit.client.0.smithi077.stdout:4/631: write d4/f9c [2653919,28279] 0 2024-04-26T04:53:12.745 INFO:tasks.workunit.client.0.smithi077.stdout:4/632: read - d4/d17/d18/d43/d53/f79 zero size 2024-04-26T04:53:12.745 INFO:tasks.workunit.client.0.smithi077.stdout:1/570: link d9/db/d2b/d75/c79 d9/d53/d8b/d93/ca9 0 2024-04-26T04:53:12.746 INFO:tasks.workunit.client.0.smithi077.stdout:7/687: write da/f89 [1908354,117156] 0 2024-04-26T04:53:12.746 INFO:tasks.workunit.client.0.smithi077.stdout:4/633: stat d4/d4e/dad 0 2024-04-26T04:53:12.749 INFO:tasks.workunit.client.0.smithi077.stdout:4/634: truncate d4/d17/d18/d43/d53/d75/f7e 5135398 0 2024-04-26T04:53:12.752 INFO:tasks.workunit.client.0.smithi077.stdout:2/680: write d9/d15/f31 [2872917,8227] 0 2024-04-26T04:53:12.753 INFO:tasks.workunit.client.0.smithi077.stdout:0/610: dwrite d0/d19/d1e/d2f/d52/f76 [0,4194304] 0 2024-04-26T04:53:12.754 INFO:tasks.workunit.client.0.smithi077.stdout:2/681: stat d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6 0 2024-04-26T04:53:12.755 INFO:tasks.workunit.client.0.smithi077.stdout:5/631: mknod d1/d42/da4/cdd 0 2024-04-26T04:53:12.757 INFO:tasks.workunit.client.0.smithi077.stdout:9/587: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/fc9 x:0 0 0 2024-04-26T04:53:12.758 INFO:tasks.workunit.client.0.smithi077.stdout:1/571: truncate d9/db/dd/f74 375020 0 2024-04-26T04:53:12.759 INFO:tasks.workunit.client.0.smithi077.stdout:1/572: chown d9/f1b 746665298 1 2024-04-26T04:53:12.760 INFO:tasks.workunit.client.0.smithi077.stdout:1/573: chown d9/d53/d6e/d7c/l8c 1447 1 2024-04-26T04:53:12.763 INFO:tasks.workunit.client.0.smithi077.stdout:6/689: mknod d8/d2a/cf1 0 2024-04-26T04:53:12.764 INFO:tasks.workunit.client.0.smithi077.stdout:6/690: write d8/ff [5429027,75981] 0 2024-04-26T04:53:12.765 INFO:tasks.workunit.client.0.smithi077.stdout:5/632: dread d1/d42/da4/d25/d31/d3c/f8e [0,4194304] 0 2024-04-26T04:53:12.767 INFO:tasks.workunit.client.0.smithi077.stdout:5/633: write d1/d8/d7e/fc6 [1809806,71032] 0 2024-04-26T04:53:12.784 INFO:tasks.workunit.client.0.smithi077.stdout:7/688: dwrite da/d13/d22/f46 [0,4194304] 0 2024-04-26T04:53:12.790 INFO:tasks.workunit.client.0.smithi077.stdout:2/682: mknod d9/d15/d56/de6/cee 0 2024-04-26T04:53:12.792 INFO:tasks.workunit.client.0.smithi077.stdout:9/588: link d9/f8d d9/d35/d99/fca 0 2024-04-26T04:53:12.795 INFO:tasks.workunit.client.0.smithi077.stdout:1/574: symlink d9/laa 0 2024-04-26T04:53:12.798 INFO:tasks.workunit.client.0.smithi077.stdout:4/635: write d4/d17/d7f/f9e [922840,34751] 0 2024-04-26T04:53:12.800 INFO:tasks.workunit.client.0.smithi077.stdout:7/689: truncate da/d13/d22/d34/fac 1026544 0 2024-04-26T04:53:12.805 INFO:tasks.workunit.client.0.smithi077.stdout:5/634: write d1/d12/f60 [3289106,19161] 0 2024-04-26T04:53:12.813 INFO:tasks.workunit.client.0.smithi077.stdout:0/611: getdents d0 0 2024-04-26T04:53:12.817 INFO:tasks.workunit.client.0.smithi077.stdout:2/683: fdatasync d9/dc/d1d/d34/d39/d63/f9c 0 2024-04-26T04:53:12.822 INFO:tasks.workunit.client.0.smithi077.stdout:1/575: mkdir d9/db/dd/d9c/dab 0 2024-04-26T04:53:12.824 INFO:tasks.workunit.client.0.smithi077.stdout:1/576: fdatasync d9/d53/d6e/d7c/f82 0 2024-04-26T04:53:12.825 INFO:tasks.workunit.client.0.smithi077.stdout:9/589: dwrite d9/dd/d1b/d2b/d32/f39 [0,4194304] 0 2024-04-26T04:53:12.829 INFO:tasks.workunit.client.0.smithi077.stdout:5/635: mknod d1/d42/d62/dc8/dd5/ddb/cde 0 2024-04-26T04:53:12.830 INFO:tasks.workunit.client.0.smithi077.stdout:4/636: dwrite d4/d17/d18/d43/d53/f79 [0,4194304] 0 2024-04-26T04:53:12.837 INFO:tasks.workunit.client.0.smithi077.stdout:2/684: write d9/dc/d1d/d34/f61 [2323474,2215] 0 2024-04-26T04:53:12.843 INFO:tasks.workunit.client.0.smithi077.stdout:9/590: creat d9/dd/d1b/d2b/d32/d36/fcb x:0 0 0 2024-04-26T04:53:12.845 INFO:tasks.workunit.client.0.smithi077.stdout:5/636: creat d1/d42/d62/dc8/fdf x:0 0 0 2024-04-26T04:53:12.847 INFO:tasks.workunit.client.0.smithi077.stdout:1/577: dwrite f7 [0,4194304] 0 2024-04-26T04:53:12.849 INFO:tasks.workunit.client.0.smithi077.stdout:7/690: getdents da/d6b/d63/d64 0 2024-04-26T04:53:12.849 INFO:tasks.workunit.client.0.smithi077.stdout:7/691: chown da/d13/d22/d5d/cab 13275454 1 2024-04-26T04:53:12.851 INFO:tasks.workunit.client.0.smithi077.stdout:7/692: fdatasync da/d13/d22/d5d/d9f/fc0 0 2024-04-26T04:53:12.851 INFO:tasks.workunit.client.0.smithi077.stdout:2/685: read d9/dc/d1d/d34/fa9 [2410772,16629] 0 2024-04-26T04:53:12.854 INFO:tasks.workunit.client.0.smithi077.stdout:5/637: symlink d1/d42/d62/dc8/le0 0 2024-04-26T04:53:12.855 INFO:tasks.workunit.client.0.smithi077.stdout:5/638: chown d1/d12/d28/d38/cb2 483 1 2024-04-26T04:53:12.856 INFO:tasks.workunit.client.0.smithi077.stdout:1/578: stat d9/db/d2b/c31 0 2024-04-26T04:53:12.857 INFO:tasks.workunit.client.0.smithi077.stdout:9/591: creat d9/dd/d3d/d7c/dc8/fcc x:0 0 0 2024-04-26T04:53:12.857 INFO:tasks.workunit.client.0.smithi077.stdout:1/579: fsync d9/db/d2b/d75/fa6 0 2024-04-26T04:53:12.865 INFO:tasks.workunit.client.0.smithi077.stdout:7/693: mknod da/ce5 0 2024-04-26T04:53:12.866 INFO:tasks.workunit.client.0.smithi077.stdout:2/686: dwrite d9/dc/f7a [0,4194304] 0 2024-04-26T04:53:12.867 INFO:tasks.workunit.client.0.smithi077.stdout:1/580: rename d9/f42 to d9/d53/d8b/d93/da8/da7/fac 0 2024-04-26T04:53:12.870 INFO:tasks.workunit.client.0.smithi077.stdout:4/637: getdents d4/d17/d18/d43/d51/db4/d15/d1f 0 2024-04-26T04:53:12.872 INFO:tasks.workunit.client.0.smithi077.stdout:4/638: chown d4/d17/d18/d43/d51/db4/d15/d1f/d25/ca2 1807811 1 2024-04-26T04:53:12.873 INFO:tasks.workunit.client.0.smithi077.stdout:4/639: readlink d4/d17/d18/d43/d6e/l9d 0 2024-04-26T04:53:12.875 INFO:tasks.workunit.client.0.smithi077.stdout:9/592: dread d9/d35/f69 [0,4194304] 0 2024-04-26T04:53:12.877 INFO:tasks.workunit.client.0.smithi077.stdout:5/639: truncate d1/d42/d62/f89 1522037 0 2024-04-26T04:53:12.882 INFO:tasks.workunit.client.0.smithi077.stdout:9/593: dwrite d9/dd/d1b/d2b/d32/d36/fb6 [0,4194304] 0 2024-04-26T04:53:12.890 INFO:tasks.workunit.client.0.smithi077.stdout:4/640: creat d4/d4e/dad/fd7 x:0 0 0 2024-04-26T04:53:12.894 INFO:tasks.workunit.client.0.smithi077.stdout:5/640: creat d1/d8f/fe1 x:0 0 0 2024-04-26T04:53:12.935 INFO:tasks.workunit.client.0.smithi077.stdout:5/641: read d1/d42/f69 [474735,12409] 0 2024-04-26T04:53:12.936 INFO:tasks.workunit.client.0.smithi077.stdout:7/694: dwrite da/d15/d1a/f20 [0,4194304] 0 2024-04-26T04:53:12.936 INFO:tasks.workunit.client.0.smithi077.stdout:1/581: dread d9/db/dd/f2f [0,4194304] 0 2024-04-26T04:53:12.938 INFO:tasks.workunit.client.0.smithi077.stdout:9/594: dread d9/dd/d1b/f73 [0,4194304] 0 2024-04-26T04:53:12.938 INFO:tasks.workunit.client.0.smithi077.stdout:2/687: dread d9/d15/d91/f96 [4194304,4194304] 0 2024-04-26T04:53:12.941 INFO:tasks.workunit.client.0.smithi077.stdout:5/642: symlink d1/d42/d62/d6b/le2 0 2024-04-26T04:53:12.945 INFO:tasks.workunit.client.0.smithi077.stdout:9/595: mkdir d9/d45/dcd 0 2024-04-26T04:53:12.947 INFO:tasks.workunit.client.0.smithi077.stdout:4/641: dread d4/d17/d18/f5d [0,4194304] 0 2024-04-26T04:53:12.949 INFO:tasks.workunit.client.0.smithi077.stdout:9/596: dread d9/dd/d1b/fab [0,4194304] 0 2024-04-26T04:53:12.950 INFO:tasks.workunit.client.0.smithi077.stdout:1/582: creat d9/d90/fad x:0 0 0 2024-04-26T04:53:12.952 INFO:tasks.workunit.client.0.smithi077.stdout:2/688: rename d9/dc/d6e/fae to d9/dc/fef 0 2024-04-26T04:53:12.953 INFO:tasks.workunit.client.0.smithi077.stdout:1/583: mknod d9/d53/d8b/cae 0 2024-04-26T04:53:12.955 INFO:tasks.workunit.client.0.smithi077.stdout:5/643: creat d1/fe3 x:0 0 0 2024-04-26T04:53:12.955 INFO:tasks.workunit.client.0.smithi077.stdout:1/584: read d9/d53/d6e/f87 [3599983,28244] 0 2024-04-26T04:53:12.956 INFO:tasks.workunit.client.0.smithi077.stdout:2/689: fdatasync d9/dc/d1d/d1e/f13 0 2024-04-26T04:53:12.958 INFO:tasks.workunit.client.0.smithi077.stdout:5/644: creat d1/d30/d33/fe4 x:0 0 0 2024-04-26T04:53:12.961 INFO:tasks.workunit.client.0.smithi077.stdout:5/645: creat d1/d12/d87/fe5 x:0 0 0 2024-04-26T04:53:12.962 INFO:tasks.workunit.client.0.smithi077.stdout:1/585: creat d9/db/faf x:0 0 0 2024-04-26T04:53:12.965 INFO:tasks.workunit.client.0.smithi077.stdout:1/586: symlink d9/d53/d6e/d7c/lb0 0 2024-04-26T04:53:12.966 INFO:tasks.workunit.client.0.smithi077.stdout:5/646: dread d1/d42/d62/f89 [0,4194304] 0 2024-04-26T04:53:12.967 INFO:tasks.workunit.client.0.smithi077.stdout:1/587: creat d9/db/dd/d34/d63/fb1 x:0 0 0 2024-04-26T04:53:12.968 INFO:tasks.workunit.client.0.smithi077.stdout:5/647: dread - d1/d42/da4/f9c zero size 2024-04-26T04:53:12.970 INFO:tasks.workunit.client.0.smithi077.stdout:1/588: mkdir d9/db/dd/d34/db2 0 2024-04-26T04:53:12.971 INFO:tasks.workunit.client.0.smithi077.stdout:4/642: dread d4/d17/d18/d43/d51/db4/d15/d1f/f32 [0,4194304] 0 2024-04-26T04:53:12.976 INFO:tasks.workunit.client.0.smithi077.stdout:1/589: getdents d9/d53/d8b/d93/da8 0 2024-04-26T04:53:12.978 INFO:tasks.workunit.client.0.smithi077.stdout:1/590: unlink d9/laa 0 2024-04-26T04:53:12.981 INFO:tasks.workunit.client.0.smithi077.stdout:1/591: write d9/db/dd/f2f [819795,38133] 0 2024-04-26T04:53:12.989 INFO:tasks.workunit.client.0.smithi077.stdout:5/648: dread d1/d8f/fbb [0,4194304] 0 2024-04-26T04:53:12.990 INFO:tasks.workunit.client.0.smithi077.stdout:4/643: dread d4/f9c [0,4194304] 0 2024-04-26T04:53:12.996 INFO:tasks.workunit.client.0.smithi077.stdout:5/649: dwrite d1/d30/f53 [0,4194304] 0 2024-04-26T04:53:12.997 INFO:tasks.workunit.client.0.smithi077.stdout:4/644: dwrite d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:12.998 INFO:tasks.workunit.client.0.smithi077.stdout:5/650: write d1/d8/d7e/fc6 [2319734,40290] 0 2024-04-26T04:53:13.006 INFO:tasks.workunit.client.0.smithi077.stdout:1/592: dread d9/f20 [0,4194304] 0 2024-04-26T04:53:13.021 INFO:tasks.workunit.client.0.smithi077.stdout:4/645: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/dd8 0 2024-04-26T04:53:13.022 INFO:tasks.workunit.client.0.smithi077.stdout:1/593: write d9/db/dd/d34/f9b [1073730,53356] 0 2024-04-26T04:53:13.023 INFO:tasks.workunit.client.0.smithi077.stdout:1/594: fsync d9/d53/d6e/d7c/fa5 0 2024-04-26T04:53:13.029 INFO:tasks.workunit.client.0.smithi077.stdout:5/651: dwrite d1/d30/d35/f5a [0,4194304] 0 2024-04-26T04:53:13.032 INFO:tasks.workunit.client.0.smithi077.stdout:5/652: mknod d1/d8d/da7/ce6 0 2024-04-26T04:53:13.033 INFO:tasks.workunit.client.0.smithi077.stdout:5/653: chown d1/d42/da4/d25/d31/d3c/l56 32206 1 2024-04-26T04:53:13.036 INFO:tasks.workunit.client.0.smithi077.stdout:5/654: mkdir d1/d30/d35/de7 0 2024-04-26T04:53:13.049 INFO:tasks.workunit.client.0.smithi077.stdout:4/646: symlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/ld9 0 2024-04-26T04:53:13.051 INFO:tasks.workunit.client.0.smithi077.stdout:1/595: dwrite d9/f7a [0,4194304] 0 2024-04-26T04:53:13.058 INFO:tasks.workunit.client.0.smithi077.stdout:5/655: dwrite d1/d42/da4/fa0 [0,4194304] 0 2024-04-26T04:53:13.059 INFO:tasks.workunit.client.0.smithi077.stdout:4/647: dwrite d4/d17/f7a [0,4194304] 0 2024-04-26T04:53:13.062 INFO:tasks.workunit.client.0.smithi077.stdout:5/656: write d1/d42/d62/f6f [2638521,59121] 0 2024-04-26T04:53:13.064 INFO:tasks.workunit.client.0.smithi077.stdout:5/657: dread - d1/d42/dac/fcf zero size 2024-04-26T04:53:13.066 INFO:tasks.workunit.client.0.smithi077.stdout:4/648: mkdir d4/d17/d18/d43/d6e/d6c/dda 0 2024-04-26T04:53:13.068 INFO:tasks.workunit.client.0.smithi077.stdout:5/658: chown d1/d8c/l93 51 1 2024-04-26T04:53:13.073 INFO:tasks.workunit.client.0.smithi077.stdout:4/649: write d4/d17/d18/d43/d51/db4/d15/d1f/fcf [292335,65012] 0 2024-04-26T04:53:13.073 INFO:tasks.workunit.client.0.smithi077.stdout:5/659: fdatasync d1/d8f/fbb 0 2024-04-26T04:53:13.074 INFO:tasks.workunit.client.0.smithi077.stdout:1/596: link d9/db/dd/f74 d9/d53/d8b/d93/da8/da7/fb3 0 2024-04-26T04:53:13.076 INFO:tasks.workunit.client.0.smithi077.stdout:1/597: write d9/db/dd/d34/d63/f8a [1128157,44997] 0 2024-04-26T04:53:13.084 INFO:tasks.workunit.client.0.smithi077.stdout:4/650: link d4/lbf d4/d17/d18/d43/d63/ldb 0 2024-04-26T04:53:13.088 INFO:tasks.workunit.client.0.smithi077.stdout:4/651: rename d4/d17/d18/d43/d6e/d6c/dae/dc5/dd0 to d4/d4e/dad/d8b/ddc 0 2024-04-26T04:53:13.090 INFO:tasks.workunit.client.0.smithi077.stdout:4/652: fsync d4/d4e/fce 0 2024-04-26T04:53:13.091 INFO:tasks.workunit.client.0.smithi077.stdout:5/660: truncate d1/d42/d62/f6f 1554267 0 2024-04-26T04:53:13.092 INFO:tasks.workunit.client.0.smithi077.stdout:5/661: dread - d1/d42/da4/d25/fd7 zero size 2024-04-26T04:53:13.097 INFO:tasks.workunit.client.0.smithi077.stdout:5/662: mkdir d1/d42/d62/d6b/de8 0 2024-04-26T04:53:13.098 INFO:tasks.workunit.client.0.smithi077.stdout:1/598: dread d9/f21 [0,4194304] 0 2024-04-26T04:53:13.100 INFO:tasks.workunit.client.0.smithi077.stdout:1/599: read d9/d53/d6e/d7c/f84 [661533,37050] 0 2024-04-26T04:53:13.102 INFO:tasks.workunit.client.0.smithi077.stdout:4/653: link d4/d17/d18/d43/d51/db4/d15/d1f/d25/c66 d4/d17/d18/d43/d63/d80/db6/cdd 0 2024-04-26T04:53:13.104 INFO:tasks.workunit.client.0.smithi077.stdout:4/654: write d4/d4e/dad/d8b/dba/fbb [302675,31822] 0 2024-04-26T04:53:13.105 INFO:tasks.workunit.client.0.smithi077.stdout:4/655: read - d4/d4e/dad/fd7 zero size 2024-04-26T04:53:13.110 INFO:tasks.workunit.client.0.smithi077.stdout:5/663: truncate d1/d30/f53 918201 0 2024-04-26T04:53:13.112 INFO:tasks.workunit.client.0.smithi077.stdout:1/600: getdents d9/db/dd/d68 0 2024-04-26T04:53:13.114 INFO:tasks.workunit.client.0.smithi077.stdout:1/601: read d9/f1d [334294,57893] 0 2024-04-26T04:53:13.117 INFO:tasks.workunit.client.0.smithi077.stdout:1/602: rmdir d9/d53/d6e 39 2024-04-26T04:53:13.117 INFO:tasks.workunit.client.0.smithi077.stdout:5/664: dwrite d1/d8d/da7/fb4 [0,4194304] 0 2024-04-26T04:53:13.123 INFO:tasks.workunit.client.0.smithi077.stdout:1/603: rmdir d9/db/dd 39 2024-04-26T04:53:13.130 INFO:tasks.workunit.client.0.smithi077.stdout:1/604: fsync d9/db/d2b/d3f/f55 0 2024-04-26T04:53:13.132 INFO:tasks.workunit.client.0.smithi077.stdout:4/656: link d4/d17/d18/d43/d51/db4/d15/d1f/d35/l9b d4/d17/d18/d43/d51/db4/lde 0 2024-04-26T04:53:13.139 INFO:tasks.workunit.client.0.smithi077.stdout:4/657: dwrite d4/d4e/dad/fd7 [0,4194304] 0 2024-04-26T04:53:13.140 INFO:tasks.workunit.client.0.smithi077.stdout:5/665: dwrite d1/d12/d28/f2e [0,4194304] 0 2024-04-26T04:53:13.141 INFO:tasks.workunit.client.0.smithi077.stdout:4/658: chown d4/d17/d7f/cd2 6926077 1 2024-04-26T04:53:13.141 INFO:tasks.workunit.client.0.smithi077.stdout:1/605: rename d9/db/dd/l71 to d9/db/dd/d9c/lb4 0 2024-04-26T04:53:13.142 INFO:tasks.workunit.client.0.smithi077.stdout:4/659: chown d4/d17/d18/d43/d6e/d6c 0 1 2024-04-26T04:53:13.146 INFO:tasks.workunit.client.0.smithi077.stdout:5/666: mknod d1/d42/d62/d6b/de8/ce9 0 2024-04-26T04:53:13.146 INFO:tasks.workunit.client.0.smithi077.stdout:4/660: readlink d4/d17/d18/d43/d51/db4/d15/d1f/la9 0 2024-04-26T04:53:13.148 INFO:tasks.workunit.client.0.smithi077.stdout:4/661: stat d4/d17 0 2024-04-26T04:53:13.148 INFO:tasks.workunit.client.0.smithi077.stdout:5/667: write d1/fe3 [537452,51712] 0 2024-04-26T04:53:13.152 INFO:tasks.workunit.client.0.smithi077.stdout:1/606: dwrite d9/db/dd/d68/d81/f88 [0,4194304] 0 2024-04-26T04:53:13.153 INFO:tasks.workunit.client.0.smithi077.stdout:8/720: sync 2024-04-26T04:53:13.153 INFO:tasks.workunit.client.0.smithi077.stdout:4/662: read d4/d17/d18/d43/d63/fb3 [363595,66867] 0 2024-04-26T04:53:13.159 INFO:tasks.workunit.client.0.smithi077.stdout:1/607: write d9/db/dd/d34/d63/f77 [2624439,114526] 0 2024-04-26T04:53:13.163 INFO:tasks.workunit.client.0.smithi077.stdout:8/721: creat d4/d69/d73/fef x:0 0 0 2024-04-26T04:53:13.164 INFO:tasks.workunit.client.0.smithi077.stdout:4/663: unlink d4/d17/d18/d43/d51/db4/f93 0 2024-04-26T04:53:13.167 INFO:tasks.workunit.client.0.smithi077.stdout:1/608: creat d9/db/dd/d68/d81/fb5 x:0 0 0 2024-04-26T04:53:13.171 INFO:tasks.workunit.client.0.smithi077.stdout:4/664: dwrite d4/d17/d18/d43/d53/f79 [0,4194304] 0 2024-04-26T04:53:13.173 INFO:tasks.workunit.client.0.smithi077.stdout:1/609: dread d9/d53/d6e/f30 [0,4194304] 0 2024-04-26T04:53:13.175 INFO:tasks.workunit.client.0.smithi077.stdout:4/665: mknod d4/d17/d18/d43/d51/db4/d15/d1f/dd8/cdf 0 2024-04-26T04:53:13.178 INFO:tasks.workunit.client.0.smithi077.stdout:5/668: truncate d1/d30/d33/f76 289650 0 2024-04-26T04:53:13.179 INFO:tasks.workunit.client.0.smithi077.stdout:5/669: write d1/d12/f19 [1994299,50123] 0 2024-04-26T04:53:13.186 INFO:tasks.workunit.client.0.smithi077.stdout:4/666: rename d4/d17/d18/d43/d53/d75 to d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0 0 2024-04-26T04:53:13.188 INFO:tasks.workunit.client.0.smithi077.stdout:5/670: truncate d1/d12/d28/f9f 948630 0 2024-04-26T04:53:13.188 INFO:tasks.workunit.client.0.smithi077.stdout:5/671: dread - d1/fc1 zero size 2024-04-26T04:53:13.193 INFO:tasks.workunit.client.0.smithi077.stdout:8/722: dread d4/df/d22/d2d/f3d [0,4194304] 0 2024-04-26T04:53:13.194 INFO:tasks.workunit.client.0.smithi077.stdout:8/723: stat d4/df/d18/d56 0 2024-04-26T04:53:13.194 INFO:tasks.workunit.client.0.smithi077.stdout:5/672: symlink d1/d12/d45/lea 0 2024-04-26T04:53:13.197 INFO:tasks.workunit.client.0.smithi077.stdout:4/667: creat d4/d17/d18/d43/d51/db4/d15/d1f/d76/fe1 x:0 0 0 2024-04-26T04:53:13.198 INFO:tasks.workunit.client.0.smithi077.stdout:4/668: fsync d4/d17/d18/d43/d51/db4/d15/d1f/d76/f82 0 2024-04-26T04:53:13.203 INFO:tasks.workunit.client.0.smithi077.stdout:5/673: chown d1/d12/d45/cd6 0 1 2024-04-26T04:53:13.203 INFO:tasks.workunit.client.0.smithi077.stdout:5/674: chown d1/d8/d72 0 1 2024-04-26T04:53:13.206 INFO:tasks.workunit.client.0.smithi077.stdout:1/610: getdents d9/d90/d99 0 2024-04-26T04:53:13.207 INFO:tasks.workunit.client.0.smithi077.stdout:4/669: fsync d4/d17/d18/d43/d51/db4/d15/d1f/d76/fe1 0 2024-04-26T04:53:13.212 INFO:tasks.workunit.client.0.smithi077.stdout:1/611: rename d9/d53/d6e/l4e to d9/db/dd/d34/lb6 0 2024-04-26T04:53:13.219 INFO:tasks.workunit.client.0.smithi077.stdout:8/724: link d4/df/dce/c6b d4/df/cf0 0 2024-04-26T04:53:13.221 INFO:tasks.workunit.client.0.smithi077.stdout:5/675: dwrite d1/d42/da4/d25/d31/f71 [4194304,4194304] 0 2024-04-26T04:53:13.224 INFO:tasks.workunit.client.0.smithi077.stdout:1/612: creat d9/db/dd/d9c/dab/fb7 x:0 0 0 2024-04-26T04:53:13.228 INFO:tasks.workunit.client.0.smithi077.stdout:5/676: mkdir d1/d42/d62/d6b/deb 0 2024-04-26T04:53:13.229 INFO:tasks.workunit.client.0.smithi077.stdout:1/613: chown d9/l29 126909 1 2024-04-26T04:53:13.230 INFO:tasks.workunit.client.0.smithi077.stdout:1/614: chown d9/d53/d6e 44466621 1 2024-04-26T04:53:13.232 INFO:tasks.workunit.client.0.smithi077.stdout:4/670: getdents d4/d17/d18/d43/d51/db4/d15/d1f/dd8 0 2024-04-26T04:53:13.233 INFO:tasks.workunit.client.0.smithi077.stdout:5/677: rmdir d1/d8c 39 2024-04-26T04:53:13.236 INFO:tasks.workunit.client.0.smithi077.stdout:5/678: mkdir d1/d42/da4/d25/d31/d3c/dec 0 2024-04-26T04:53:13.237 INFO:tasks.workunit.client.0.smithi077.stdout:1/615: truncate d9/db/f4c 541731 0 2024-04-26T04:53:13.239 INFO:tasks.workunit.client.0.smithi077.stdout:4/671: creat d4/d17/d18/fe2 x:0 0 0 2024-04-26T04:53:13.240 INFO:tasks.workunit.client.0.smithi077.stdout:1/616: mkdir d9/d53/d8b/d93/da8/db8 0 2024-04-26T04:53:13.241 INFO:tasks.workunit.client.0.smithi077.stdout:1/617: fdatasync d9/db/dd/d34/d63/f8a 0 2024-04-26T04:53:13.243 INFO:tasks.workunit.client.0.smithi077.stdout:5/679: creat d1/d12/fed x:0 0 0 2024-04-26T04:53:13.244 INFO:tasks.workunit.client.0.smithi077.stdout:5/680: write d1/d12/d28/f2e [1691118,82473] 0 2024-04-26T04:53:13.251 INFO:tasks.workunit.client.0.smithi077.stdout:1/618: mkdir d9/d53/db9 0 2024-04-26T04:53:13.252 INFO:tasks.workunit.client.0.smithi077.stdout:1/619: chown d9/db/d2b/d75 212992 1 2024-04-26T04:53:13.256 INFO:tasks.workunit.client.0.smithi077.stdout:4/672: dwrite d4/d4e/dad/f8c [4194304,4194304] 0 2024-04-26T04:53:13.256 INFO:tasks.workunit.client.0.smithi077.stdout:5/681: rmdir d1/d42/d62/dc8 39 2024-04-26T04:53:13.258 INFO:tasks.workunit.client.0.smithi077.stdout:4/673: truncate d4/d17/d18/fa7 311946 0 2024-04-26T04:53:13.264 INFO:tasks.workunit.client.0.smithi077.stdout:5/682: dwrite d1/d8f/fbb [0,4194304] 0 2024-04-26T04:53:13.264 INFO:tasks.workunit.client.0.smithi077.stdout:5/683: chown d1/d8/lad 27397 1 2024-04-26T04:53:13.276 INFO:tasks.workunit.client.0.smithi077.stdout:1/620: mkdir d9/d53/dba 0 2024-04-26T04:53:13.277 INFO:tasks.workunit.client.0.smithi077.stdout:1/621: chown d9/db/d2b/d75/l98 6 1 2024-04-26T04:53:13.278 INFO:tasks.workunit.client.0.smithi077.stdout:5/684: rename d1/d8/f95 to d1/d12/d28/d38/fee 0 2024-04-26T04:53:13.279 INFO:tasks.workunit.client.0.smithi077.stdout:1/622: write d9/db/d2b/d75/fa6 [363565,31897] 0 2024-04-26T04:53:13.280 INFO:tasks.workunit.client.0.smithi077.stdout:5/685: write d1/d12/f19 [4483458,127121] 0 2024-04-26T04:53:13.280 INFO:tasks.workunit.client.0.smithi077.stdout:4/674: unlink d4/d17/d18/d43/d51/db4/d15/d1f/c71 0 2024-04-26T04:53:13.283 INFO:tasks.workunit.client.0.smithi077.stdout:1/623: mkdir d9/db/dd/d34/d4a/dbb 0 2024-04-26T04:53:13.286 INFO:tasks.workunit.client.0.smithi077.stdout:5/686: dwrite d1/d12/f50 [0,4194304] 0 2024-04-26T04:53:13.287 INFO:tasks.workunit.client.0.smithi077.stdout:4/675: creat d4/d17/d18/d43/d53/fe3 x:0 0 0 2024-04-26T04:53:13.288 INFO:tasks.workunit.client.0.smithi077.stdout:1/624: creat d9/d53/d8b/fbc x:0 0 0 2024-04-26T04:53:13.292 INFO:tasks.workunit.client.0.smithi077.stdout:4/676: symlink d4/d4e/dad/d8b/dba/le4 0 2024-04-26T04:53:13.295 INFO:tasks.workunit.client.0.smithi077.stdout:1/625: rename d9/d53/d6e/f28 to d9/db/dd/d34/fbd 0 2024-04-26T04:53:13.295 INFO:tasks.workunit.client.0.smithi077.stdout:1/626: dread - d9/db/dd/d68/f89 zero size 2024-04-26T04:53:13.296 INFO:tasks.workunit.client.0.smithi077.stdout:5/687: unlink d1/c1d 0 2024-04-26T04:53:13.297 INFO:tasks.workunit.client.0.smithi077.stdout:5/688: readlink d1/d42/da4/l78 0 2024-04-26T04:53:13.301 INFO:tasks.workunit.client.0.smithi077.stdout:5/689: unlink d1/d42/d62/d6b/f77 0 2024-04-26T04:53:13.302 INFO:tasks.workunit.client.0.smithi077.stdout:1/627: symlink d9/db/dd/d34/d4a/dbb/lbe 0 2024-04-26T04:53:13.305 INFO:tasks.workunit.client.0.smithi077.stdout:1/628: creat d9/d90/fbf x:0 0 0 2024-04-26T04:53:13.308 INFO:tasks.workunit.client.0.smithi077.stdout:1/629: mkdir d9/d53/d6e/d7c/dc0 0 2024-04-26T04:53:13.311 INFO:tasks.workunit.client.0.smithi077.stdout:1/630: mkdir d9/d53/dc1 0 2024-04-26T04:53:13.314 INFO:tasks.workunit.client.0.smithi077.stdout:1/631: rename d9/db/d2b/d3f/fa4 to d9/db/d2b/d3f/fc2 0 2024-04-26T04:53:13.316 INFO:tasks.workunit.client.0.smithi077.stdout:1/632: write d9/db/dd/d34/d63/f9e [352688,94320] 0 2024-04-26T04:53:13.320 INFO:tasks.workunit.client.0.smithi077.stdout:1/633: creat d9/db/d2b/d3f/fc3 x:0 0 0 2024-04-26T04:53:13.321 INFO:tasks.workunit.client.0.smithi077.stdout:1/634: chown d9/db/dd/d68/d81/f88 53287 1 2024-04-26T04:53:13.322 INFO:tasks.workunit.client.0.smithi077.stdout:1/635: readlink d9/l56 0 2024-04-26T04:53:13.324 INFO:tasks.workunit.client.0.smithi077.stdout:4/677: dwrite d4/d17/d18/d43/d53/fbe [0,4194304] 0 2024-04-26T04:53:13.332 INFO:tasks.workunit.client.0.smithi077.stdout:1/636: mknod d9/db/dd/d34/cc4 0 2024-04-26T04:53:13.333 INFO:tasks.workunit.client.0.smithi077.stdout:4/678: dwrite d4/d17/d18/d43/d6e/f95 [0,4194304] 0 2024-04-26T04:53:13.336 INFO:tasks.workunit.client.0.smithi077.stdout:1/637: chown d9/c14 32 1 2024-04-26T04:53:13.338 INFO:tasks.workunit.client.0.smithi077.stdout:1/638: write d9/db/dd/d34/d63/f77 [3540473,23070] 0 2024-04-26T04:53:13.359 INFO:tasks.workunit.client.0.smithi077.stdout:1/639: dread d9/d53/f86 [0,4194304] 0 2024-04-26T04:53:13.366 INFO:tasks.workunit.client.0.smithi077.stdout:4/679: dread d4/d17/d18/d43/d51/db4/d15/f40 [0,4194304] 0 2024-04-26T04:53:13.374 INFO:tasks.workunit.client.0.smithi077.stdout:4/680: getdents d4/d17/d18/d43/d6e/d6c/dae 0 2024-04-26T04:53:13.375 INFO:tasks.workunit.client.0.smithi077.stdout:4/681: chown d4/c9a 758017656 1 2024-04-26T04:53:13.378 INFO:tasks.workunit.client.0.smithi077.stdout:4/682: mkdir d4/d4e/dad/d8b/dba/de5 0 2024-04-26T04:53:13.382 INFO:tasks.workunit.client.0.smithi077.stdout:4/683: dwrite d4/d17/f57 [0,4194304] 0 2024-04-26T04:53:13.384 INFO:tasks.workunit.client.0.smithi077.stdout:4/684: fdatasync d4/d17/d18/d43/d51/db4/d15/f50 0 2024-04-26T04:53:13.389 INFO:tasks.workunit.client.0.smithi077.stdout:4/685: symlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/le6 0 2024-04-26T04:53:13.390 INFO:tasks.workunit.client.0.smithi077.stdout:4/686: write d4/d17/f96 [556450,104221] 0 2024-04-26T04:53:13.391 INFO:tasks.workunit.client.0.smithi077.stdout:4/687: read d4/d17/d18/f5d [99789,50773] 0 2024-04-26T04:53:13.393 INFO:tasks.workunit.client.0.smithi077.stdout:4/688: truncate d4/d17/d18/fa7 1089567 0 2024-04-26T04:53:13.393 INFO:tasks.workunit.client.0.smithi077.stdout:1/640: dread d9/db/d2b/f61 [4194304,4194304] 0 2024-04-26T04:53:13.396 INFO:tasks.workunit.client.0.smithi077.stdout:4/689: write d4/d17/d18/d43/d51/db4/d15/d1f/d25/fcb [3045922,27589] 0 2024-04-26T04:53:13.397 INFO:tasks.workunit.client.0.smithi077.stdout:4/690: chown d4/d17/d18/d43/d63/fb3 28102189 1 2024-04-26T04:53:13.401 INFO:tasks.workunit.client.0.smithi077.stdout:4/691: write d4/d4e/f91 [4514899,69028] 0 2024-04-26T04:53:13.402 INFO:tasks.workunit.client.0.smithi077.stdout:1/641: creat d9/d53/d8b/d93/da8/db8/fc5 x:0 0 0 2024-04-26T04:53:13.403 INFO:tasks.workunit.client.0.smithi077.stdout:4/692: dread d4/d17/d18/d43/d53/fbe [0,4194304] 0 2024-04-26T04:53:13.404 INFO:tasks.workunit.client.0.smithi077.stdout:4/693: stat d4/d4e 0 2024-04-26T04:53:13.406 INFO:tasks.workunit.client.0.smithi077.stdout:1/642: symlink d9/db/dd/lc6 0 2024-04-26T04:53:13.412 INFO:tasks.workunit.client.0.smithi077.stdout:1/643: mknod d9/d53/db9/cc7 0 2024-04-26T04:53:13.412 INFO:tasks.workunit.client.0.smithi077.stdout:1/644: dread - d9/db/dd/d68/d81/fb5 zero size 2024-04-26T04:53:13.417 INFO:tasks.workunit.client.0.smithi077.stdout:4/694: dread d4/d17/d18/d43/d51/db4/d15/d1f/d25/fcb [0,4194304] 0 2024-04-26T04:53:13.424 INFO:tasks.workunit.client.0.smithi077.stdout:4/695: creat d4/d17/d18/d43/d51/db4/d15/d1f/fe7 x:0 0 0 2024-04-26T04:53:13.428 INFO:tasks.workunit.client.0.smithi077.stdout:1/645: dwrite d9/db/dd/f69 [0,4194304] 0 2024-04-26T04:53:13.430 INFO:tasks.workunit.client.0.smithi077.stdout:1/646: fdatasync d9/db/faf 0 2024-04-26T04:53:13.434 INFO:tasks.workunit.client.0.smithi077.stdout:4/696: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/fe7 [0,4194304] 0 2024-04-26T04:53:13.434 INFO:tasks.workunit.client.0.smithi077.stdout:1/647: truncate d9/db/dd/d34/d63/f76 4075300 0 2024-04-26T04:53:13.435 INFO:tasks.workunit.client.0.smithi077.stdout:1/648: stat d9/db/d2b/d3f/f55 0 2024-04-26T04:53:13.440 INFO:tasks.workunit.client.0.smithi077.stdout:4/697: symlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/le8 0 2024-04-26T04:53:13.441 INFO:tasks.workunit.client.0.smithi077.stdout:1/649: dread d9/db/dd/d34/d63/f9e [0,4194304] 0 2024-04-26T04:53:13.463 INFO:tasks.workunit.client.0.smithi077.stdout:1/650: dwrite d9/d53/f86 [0,4194304] 0 2024-04-26T04:53:13.464 INFO:tasks.workunit.client.0.smithi077.stdout:1/651: write d9/db/dd/d34/f9b [2611098,123440] 0 2024-04-26T04:53:13.471 INFO:tasks.workunit.client.0.smithi077.stdout:1/652: symlink d9/d53/dc1/lc8 0 2024-04-26T04:53:13.481 INFO:tasks.workunit.client.0.smithi077.stdout:3/547: sync 2024-04-26T04:53:13.485 INFO:tasks.workunit.client.0.smithi077.stdout:3/548: getdents d0/d9/d32 0 2024-04-26T04:53:13.487 INFO:tasks.workunit.client.0.smithi077.stdout:3/549: unlink d0/d9/d32/d93/f97 0 2024-04-26T04:53:13.489 INFO:tasks.workunit.client.0.smithi077.stdout:3/550: creat d0/d9/d32/d29/faf x:0 0 0 2024-04-26T04:53:13.503 INFO:tasks.workunit.client.0.smithi077.stdout:3/551: dread d0/d7/d1f/f72 [0,4194304] 0 2024-04-26T04:53:13.504 INFO:tasks.workunit.client.0.smithi077.stdout:3/552: dread - d0/d9/d32/d2a/f4e zero size 2024-04-26T04:53:13.609 INFO:tasks.workunit.client.0.smithi077.stdout:3/553: unlink d0/d7/f90 0 2024-04-26T04:53:13.611 INFO:tasks.workunit.client.0.smithi077.stdout:3/554: unlink d0/d7/d15/f50 0 2024-04-26T04:53:13.617 INFO:tasks.workunit.client.0.smithi077.stdout:3/555: dwrite d0/d9/f84 [0,4194304] 0 2024-04-26T04:53:13.624 INFO:tasks.workunit.client.0.smithi077.stdout:3/556: dwrite d0/f38 [4194304,4194304] 0 2024-04-26T04:53:13.625 INFO:tasks.workunit.client.0.smithi077.stdout:3/557: stat d0/d7/d1f/l55 0 2024-04-26T04:53:13.627 INFO:tasks.workunit.client.0.smithi077.stdout:3/558: read d0/d9/d32/d29/f31 [8736480,6543] 0 2024-04-26T04:53:13.627 INFO:tasks.workunit.client.0.smithi077.stdout:3/559: chown d0 28176273 1 2024-04-26T04:53:13.628 INFO:tasks.workunit.client.0.smithi077.stdout:6/691: sync 2024-04-26T04:53:13.631 INFO:tasks.workunit.client.0.smithi077.stdout:6/692: chown d8/d52/c99 12 1 2024-04-26T04:53:13.632 INFO:tasks.workunit.client.0.smithi077.stdout:3/560: dread d0/d9/d32/d2a/faa [8388608,4194304] 0 2024-04-26T04:53:13.637 INFO:tasks.workunit.client.0.smithi077.stdout:3/561: symlink d0/d7/d1f/d81/lb0 0 2024-04-26T04:53:13.639 INFO:tasks.workunit.client.0.smithi077.stdout:6/693: mkdir d8/d2a/df2 0 2024-04-26T04:53:13.642 INFO:tasks.workunit.client.0.smithi077.stdout:6/694: truncate d8/d1c/f3e 804072 0 2024-04-26T04:53:13.643 INFO:tasks.workunit.client.0.smithi077.stdout:3/562: dwrite d0/d9/d32/f8a [0,4194304] 0 2024-04-26T04:53:13.655 INFO:tasks.workunit.client.0.smithi077.stdout:6/695: write d8/fe [2160880,80429] 0 2024-04-26T04:53:13.656 INFO:tasks.workunit.client.0.smithi077.stdout:3/563: rmdir d0/d9/d32/d93 0 2024-04-26T04:53:13.663 INFO:tasks.workunit.client.0.smithi077.stdout:3/564: dwrite d0/d7/d1f/d81/f89 [0,4194304] 0 2024-04-26T04:53:13.670 INFO:tasks.workunit.client.0.smithi077.stdout:6/696: dwrite d8/d52/d53/f80 [0,4194304] 0 2024-04-26T04:53:13.675 INFO:tasks.workunit.client.0.smithi077.stdout:6/697: creat d8/d52/da0/dad/ff3 x:0 0 0 2024-04-26T04:53:13.677 INFO:tasks.workunit.client.0.smithi077.stdout:3/565: write d0/d9/d32/d2a/faa [4684739,66841] 0 2024-04-26T04:53:13.680 INFO:tasks.workunit.client.0.smithi077.stdout:6/698: mkdir d8/d52/d53/da7/df4 0 2024-04-26T04:53:13.685 INFO:tasks.workunit.client.0.smithi077.stdout:6/699: rename d8/l1e to d8/d52/d53/da7/df4/lf5 0 2024-04-26T04:53:13.685 INFO:tasks.workunit.client.0.smithi077.stdout:6/700: chown d8/d2a/lc6 1 1 2024-04-26T04:53:13.688 INFO:tasks.workunit.client.0.smithi077.stdout:6/701: fsync d8/f6d 0 2024-04-26T04:53:13.689 INFO:tasks.workunit.client.0.smithi077.stdout:3/566: dread d0/d9/d32/d29/f71 [0,4194304] 0 2024-04-26T04:53:13.694 INFO:tasks.workunit.client.0.smithi077.stdout:6/702: fsync d8/d52/fb3 0 2024-04-26T04:53:13.700 INFO:tasks.workunit.client.0.smithi077.stdout:3/567: dread d0/d9/f2b [0,4194304] 0 2024-04-26T04:53:13.705 INFO:tasks.workunit.client.0.smithi077.stdout:3/568: dread d0/d9/d32/d29/f6a [0,4194304] 0 2024-04-26T04:53:13.715 INFO:tasks.workunit.client.0.smithi077.stdout:6/703: dwrite d8/d52/d53/f72 [0,4194304] 0 2024-04-26T04:53:13.720 INFO:tasks.workunit.client.0.smithi077.stdout:6/704: symlink d8/d52/d53/d5e/d6c/lf6 0 2024-04-26T04:53:13.721 INFO:tasks.workunit.client.0.smithi077.stdout:3/569: write d0/d7/d15/f44 [65288,18882] 0 2024-04-26T04:53:13.723 INFO:tasks.workunit.client.0.smithi077.stdout:6/705: creat d8/d52/da0/ff7 x:0 0 0 2024-04-26T04:53:13.725 INFO:tasks.workunit.client.0.smithi077.stdout:3/570: write d0/f1 [6185908,130250] 0 2024-04-26T04:53:13.727 INFO:tasks.workunit.client.0.smithi077.stdout:6/706: rename d8/d52/d53/da7/fd2 to d8/d52/d53/d5e/da6/ddf/ff8 0 2024-04-26T04:53:13.731 INFO:tasks.workunit.client.0.smithi077.stdout:0/612: sync 2024-04-26T04:53:13.732 INFO:tasks.workunit.client.0.smithi077.stdout:6/707: rename d8/d55/f90 to d8/d52/ff9 0 2024-04-26T04:53:13.736 INFO:tasks.workunit.client.0.smithi077.stdout:6/708: mkdir d8/d52/d53/da7/dfa 0 2024-04-26T04:53:13.739 INFO:tasks.workunit.client.0.smithi077.stdout:6/709: creat d8/d1c/db5/ffb x:0 0 0 2024-04-26T04:53:13.739 INFO:tasks.workunit.client.0.smithi077.stdout:6/710: stat d8/d52/d53/fcc 0 2024-04-26T04:53:13.739 INFO:tasks.workunit.client.0.smithi077.stdout:3/571: link d0/d7/d1f/d76/c77 d0/d7/d9a/cb1 0 2024-04-26T04:53:13.741 INFO:tasks.workunit.client.0.smithi077.stdout:6/711: mknod d8/d52/d53/d5e/cfc 0 2024-04-26T04:53:13.742 INFO:tasks.workunit.client.0.smithi077.stdout:6/712: chown d8/d1c/f79 22 1 2024-04-26T04:53:13.743 INFO:tasks.workunit.client.0.smithi077.stdout:3/572: fdatasync d0/d9/d32/d29/f71 0 2024-04-26T04:53:13.749 INFO:tasks.workunit.client.0.smithi077.stdout:3/573: dread d0/d7/fad [0,4194304] 0 2024-04-26T04:53:13.750 INFO:tasks.workunit.client.0.smithi077.stdout:0/613: dread d0/d47/f22 [0,4194304] 0 2024-04-26T04:53:13.755 INFO:tasks.workunit.client.0.smithi077.stdout:3/574: truncate d0/d9/d32/d29/f4b 661550 0 2024-04-26T04:53:13.755 INFO:tasks.workunit.client.0.smithi077.stdout:3/575: stat d0 0 2024-04-26T04:53:13.757 INFO:tasks.workunit.client.0.smithi077.stdout:0/614: dwrite d0/d19/f2d [4194304,4194304] 0 2024-04-26T04:53:13.758 INFO:tasks.workunit.client.0.smithi077.stdout:0/615: write d0/d47/d64/d9e/fc3 [1010770,52202] 0 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:7/695: sync 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:8/725: sync 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:1/653: sync 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:2/690: sync 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:9/597: sync 2024-04-26T04:53:13.766 INFO:tasks.workunit.client.0.smithi077.stdout:5/690: sync 2024-04-26T04:53:13.767 INFO:tasks.workunit.client.0.smithi077.stdout:4/698: sync 2024-04-26T04:53:13.770 INFO:tasks.workunit.client.0.smithi077.stdout:3/576: rename d0/d9/d32/d29/c74 to d0/d9/cb2 0 2024-04-26T04:53:13.770 INFO:tasks.workunit.client.0.smithi077.stdout:5/691: write d1/d30/d33/fe4 [665159,6482] 0 2024-04-26T04:53:13.770 INFO:tasks.workunit.client.0.smithi077.stdout:7/696: chown da/d13/d22/d5d/dbd/cc2 5810 1 2024-04-26T04:53:13.775 INFO:tasks.workunit.client.0.smithi077.stdout:4/699: dwrite d4/d4e/f91 [0,4194304] 0 2024-04-26T04:53:13.775 INFO:tasks.workunit.client.0.smithi077.stdout:1/654: creat d9/d53/d8b/d93/da8/db8/fc9 x:0 0 0 2024-04-26T04:53:13.777 INFO:tasks.workunit.client.0.smithi077.stdout:0/616: rename d0/c5b to d0/d47/d64/cd5 0 2024-04-26T04:53:13.779 INFO:tasks.workunit.client.0.smithi077.stdout:2/691: truncate d9/dc/d1d/f4a 4180005 0 2024-04-26T04:53:13.784 INFO:tasks.workunit.client.0.smithi077.stdout:4/700: creat d4/d4e/dad/d8b/ddc/fe9 x:0 0 0 2024-04-26T04:53:13.784 INFO:tasks.workunit.client.0.smithi077.stdout:4/701: fsync d4/d4e/f8d 0 2024-04-26T04:53:13.786 INFO:tasks.workunit.client.0.smithi077.stdout:6/713: write d8/f60 [5226409,107447] 0 2024-04-26T04:53:13.787 INFO:tasks.workunit.client.0.smithi077.stdout:4/702: write d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/f8e [1173823,26317] 0 2024-04-26T04:53:13.788 INFO:tasks.workunit.client.0.smithi077.stdout:6/714: truncate d8/d52/d53/d5e/fab 894834 0 2024-04-26T04:53:13.789 INFO:tasks.workunit.client.0.smithi077.stdout:8/726: rename d4/df/dce/c88 to d4/df/d6f/da3/dc7/dde/cf1 0 2024-04-26T04:53:13.791 INFO:tasks.workunit.client.0.smithi077.stdout:9/598: creat d9/dd/d1b/d2b/d32/fce x:0 0 0 2024-04-26T04:53:13.792 INFO:tasks.workunit.client.0.smithi077.stdout:9/599: dread - d9/d7d/fbf zero size 2024-04-26T04:53:13.793 INFO:tasks.workunit.client.0.smithi077.stdout:2/692: mknod d9/dc/d1d/d34/d39/d63/d79/cf0 0 2024-04-26T04:53:13.794 INFO:tasks.workunit.client.0.smithi077.stdout:1/655: creat d9/d53/dba/fca x:0 0 0 2024-04-26T04:53:13.795 INFO:tasks.workunit.client.0.smithi077.stdout:2/693: read d9/dc/d1d/d1e/d77/fa2 [665875,7732] 0 2024-04-26T04:53:13.796 INFO:tasks.workunit.client.0.smithi077.stdout:1/656: write d9/d53/d8b/d93/da8/db8/fc9 [736627,48999] 0 2024-04-26T04:53:13.797 INFO:tasks.workunit.client.0.smithi077.stdout:1/657: stat d9/db/dd/f23 0 2024-04-26T04:53:13.798 INFO:tasks.workunit.client.0.smithi077.stdout:2/694: chown d9/dc/d1d/d34/d39/d63/d79/d5b/fb6 0 1 2024-04-26T04:53:13.798 INFO:tasks.workunit.client.0.smithi077.stdout:1/658: chown d9/db/dd/c57 73321 1 2024-04-26T04:53:13.799 INFO:tasks.workunit.client.0.smithi077.stdout:4/703: symlink d4/d17/d18/d43/lea 0 2024-04-26T04:53:13.800 INFO:tasks.workunit.client.0.smithi077.stdout:2/695: read d9/dc/d1d/d34/d39/d63/d79/f1c [1632283,55634] 0 2024-04-26T04:53:13.801 INFO:tasks.workunit.client.0.smithi077.stdout:6/715: creat d8/d52/d53/d5e/da6/ddf/ffd x:0 0 0 2024-04-26T04:53:13.804 INFO:tasks.workunit.client.0.smithi077.stdout:3/577: rename d0/d7/d15 to d0/d7/d9a/db3 0 2024-04-26T04:53:13.807 INFO:tasks.workunit.client.0.smithi077.stdout:8/727: unlink d4/df/dce/d5b/f63 0 2024-04-26T04:53:13.808 INFO:tasks.workunit.client.0.smithi077.stdout:9/600: dread d9/dd/d1b/d2b/d32/f39 [0,4194304] 0 2024-04-26T04:53:13.809 INFO:tasks.workunit.client.0.smithi077.stdout:2/696: dwrite d9/d15/f9f [0,4194304] 0 2024-04-26T04:53:13.814 INFO:tasks.workunit.client.0.smithi077.stdout:1/659: symlink d9/db/dd/d34/d4a/dbb/lcb 0 2024-04-26T04:53:13.822 INFO:tasks.workunit.client.0.smithi077.stdout:4/704: creat d4/d17/d7f/feb x:0 0 0 2024-04-26T04:53:13.828 INFO:tasks.workunit.client.0.smithi077.stdout:3/578: chown d0/d7/c80 0 1 2024-04-26T04:53:13.831 INFO:tasks.workunit.client.0.smithi077.stdout:8/728: mkdir d4/df/d6f/da3/dc7/df2 0 2024-04-26T04:53:13.831 INFO:tasks.workunit.client.0.smithi077.stdout:8/729: chown d4/df/d22 0 1 2024-04-26T04:53:13.833 INFO:tasks.workunit.client.0.smithi077.stdout:2/697: chown d9/d15/c59 14 1 2024-04-26T04:53:13.836 INFO:tasks.workunit.client.0.smithi077.stdout:1/660: symlink d9/db/dd/d68/d81/lcc 0 2024-04-26T04:53:13.840 INFO:tasks.workunit.client.0.smithi077.stdout:4/705: mkdir d4/d4e/dad/d8b/ddc/dec 0 2024-04-26T04:53:13.847 INFO:tasks.workunit.client.0.smithi077.stdout:3/579: unlink d0/d9/c57 0 2024-04-26T04:53:13.848 INFO:tasks.workunit.client.0.smithi077.stdout:3/580: fsync d0/d7/d1f/d4d/f82 0 2024-04-26T04:53:13.854 INFO:tasks.workunit.client.0.smithi077.stdout:0/617: creat d0/d47/dce/fd6 x:0 0 0 2024-04-26T04:53:13.859 INFO:tasks.workunit.client.0.smithi077.stdout:6/716: link d8/d2a/cb8 d8/d2a/ddd/cfe 0 2024-04-26T04:53:13.860 INFO:tasks.workunit.client.0.smithi077.stdout:6/717: write d8/d55/f9b [484129,92351] 0 2024-04-26T04:53:13.861 INFO:tasks.workunit.client.0.smithi077.stdout:2/698: dwrite d9/dc/d1d/d34/d39/d63/d79/f3f [0,4194304] 0 2024-04-26T04:53:13.862 INFO:tasks.workunit.client.0.smithi077.stdout:5/692: rename d1/d42/da4/cdd to d1/d42/d62/cef 0 2024-04-26T04:53:13.868 INFO:tasks.workunit.client.0.smithi077.stdout:1/661: rename d9/db/dd/d34/fbd to d9/db/dd/d34/fcd 0 2024-04-26T04:53:13.870 INFO:tasks.workunit.client.0.smithi077.stdout:4/706: symlink d4/d4e/dad/d8b/ddc/dec/led 0 2024-04-26T04:53:13.871 INFO:tasks.workunit.client.0.smithi077.stdout:4/707: stat d4/d17/d18/d43/d51/db4/d15/f50 0 2024-04-26T04:53:13.872 INFO:tasks.workunit.client.0.smithi077.stdout:4/708: fdatasync d4/d17/d18/d43/d6e/d6c/fd6 0 2024-04-26T04:53:13.879 INFO:tasks.workunit.client.0.smithi077.stdout:2/699: truncate d9/dc/d5f/d8e/db9/fbc 945402 0 2024-04-26T04:53:13.881 INFO:tasks.workunit.client.0.smithi077.stdout:7/697: rename da/d13/d22/d34/l3f to da/d15/d42/d8c/le6 0 2024-04-26T04:53:13.883 INFO:tasks.workunit.client.0.smithi077.stdout:7/698: write da/ff [2064840,126524] 0 2024-04-26T04:53:13.885 INFO:tasks.workunit.client.0.smithi077.stdout:8/730: creat d4/df/d22/d80/ff3 x:0 0 0 2024-04-26T04:53:13.889 INFO:tasks.workunit.client.0.smithi077.stdout:0/618: rmdir d0/d19/d69 39 2024-04-26T04:53:13.891 INFO:tasks.workunit.client.0.smithi077.stdout:1/662: dread d9/f43 [0,4194304] 0 2024-04-26T04:53:13.892 INFO:tasks.workunit.client.0.smithi077.stdout:6/718: truncate d8/d1c/f4c 243595 0 2024-04-26T04:53:13.894 INFO:tasks.workunit.client.0.smithi077.stdout:9/601: rename d9/d35/d99/lb0 to d9/d43/lcf 0 2024-04-26T04:53:13.895 INFO:tasks.workunit.client.0.smithi077.stdout:7/699: truncate da/d13/d22/d5d/f9a 919787 0 2024-04-26T04:53:13.897 INFO:tasks.workunit.client.0.smithi077.stdout:8/731: mknod d4/d17/d90/cf4 0 2024-04-26T04:53:13.898 INFO:tasks.workunit.client.0.smithi077.stdout:4/709: write d4/d17/d18/d43/d51/db4/d15/d1f/f34 [692390,119574] 0 2024-04-26T04:53:13.898 INFO:tasks.workunit.client.0.smithi077.stdout:8/732: chown d4/df/d6f/db8 14 1 2024-04-26T04:53:13.901 INFO:tasks.workunit.client.0.smithi077.stdout:2/700: creat d9/dc/d1d/d34/d39/d63/d79/dc2/ff1 x:0 0 0 2024-04-26T04:53:13.903 INFO:tasks.workunit.client.0.smithi077.stdout:2/701: chown d9/dc/d1d/d34/d39/d63/d79/fd6 17288897 1 2024-04-26T04:53:13.905 INFO:tasks.workunit.client.0.smithi077.stdout:5/693: rename d1/d42/d62/f9e to d1/d42/dac/ff0 0 2024-04-26T04:53:13.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/733: creat d4/d69/d73/ff5 x:0 0 0 2024-04-26T04:53:13.910 INFO:tasks.workunit.client.0.smithi077.stdout:0/619: mkdir d0/d19/d1e/dd2/dd7 0 2024-04-26T04:53:13.911 INFO:tasks.workunit.client.0.smithi077.stdout:7/700: dread da/d13/d22/d34/f5a [4194304,4194304] 0 2024-04-26T04:53:13.912 INFO:tasks.workunit.client.0.smithi077.stdout:7/701: write da/fc [1548002,78381] 0 2024-04-26T04:53:13.913 INFO:tasks.workunit.client.0.smithi077.stdout:1/663: truncate d9/db/f39 1653302 0 2024-04-26T04:53:13.913 INFO:tasks.workunit.client.0.smithi077.stdout:1/664: chown f7 36 1 2024-04-26T04:53:13.914 INFO:tasks.workunit.client.0.smithi077.stdout:7/702: fsync da/d15/d42/d8c/f9b 0 2024-04-26T04:53:13.915 INFO:tasks.workunit.client.0.smithi077.stdout:6/719: creat d8/d2a/df2/fff x:0 0 0 2024-04-26T04:53:13.916 INFO:tasks.workunit.client.0.smithi077.stdout:6/720: write d8/d52/d53/d5e/f8c [5936925,68564] 0 2024-04-26T04:53:13.920 INFO:tasks.workunit.client.0.smithi077.stdout:2/702: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/cf2 0 2024-04-26T04:53:13.920 INFO:tasks.workunit.client.0.smithi077.stdout:2/703: readlink d9/dc/d1d/d34/l5a 0 2024-04-26T04:53:13.926 INFO:tasks.workunit.client.0.smithi077.stdout:9/602: rename d9/dd/d1b/d2b/d32/fc2 to d9/d3a/d79/d95/dad/fd0 0 2024-04-26T04:53:13.927 INFO:tasks.workunit.client.0.smithi077.stdout:9/603: fdatasync d9/dd/d1b/d2b/d32/fce 0 2024-04-26T04:53:13.928 INFO:tasks.workunit.client.0.smithi077.stdout:9/604: chown d9/d3a/f6e 2033 1 2024-04-26T04:53:13.931 INFO:tasks.workunit.client.0.smithi077.stdout:4/710: dwrite d4/d4e/fce [0,4194304] 0 2024-04-26T04:53:13.932 INFO:tasks.workunit.client.0.smithi077.stdout:4/711: dread - d4/d17/d18/d43/d6e/d6c/fd6 zero size 2024-04-26T04:53:13.938 INFO:tasks.workunit.client.0.smithi077.stdout:4/712: dwrite d4/d17/f57 [0,4194304] 0 2024-04-26T04:53:13.939 INFO:tasks.workunit.client.0.smithi077.stdout:6/721: symlink d8/d52/d53/da7/l100 0 2024-04-26T04:53:13.945 INFO:tasks.workunit.client.0.smithi077.stdout:0/620: rename d0/d19/d1e/dd2 to d0/d47/d64/d97/dba/dd8 0 2024-04-26T04:53:13.946 INFO:tasks.workunit.client.0.smithi077.stdout:3/581: link d0/c12 d0/d9/d32/d2a/d85/cb4 0 2024-04-26T04:53:13.947 INFO:tasks.workunit.client.0.smithi077.stdout:4/713: dwrite d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:13.953 INFO:tasks.workunit.client.0.smithi077.stdout:5/694: write d1/d12/d28/d38/fee [3504937,49522] 0 2024-04-26T04:53:13.953 INFO:tasks.workunit.client.0.smithi077.stdout:9/605: mkdir d9/d7d/dd1 0 2024-04-26T04:53:13.957 INFO:tasks.workunit.client.0.smithi077.stdout:3/582: symlink d0/d9/d32/d29/lb5 0 2024-04-26T04:53:13.958 INFO:tasks.workunit.client.0.smithi077.stdout:3/583: write d0/f87 [1647361,58640] 0 2024-04-26T04:53:13.958 INFO:tasks.workunit.client.0.smithi077.stdout:4/714: dread d4/d17/f96 [0,4194304] 0 2024-04-26T04:53:13.961 INFO:tasks.workunit.client.0.smithi077.stdout:6/722: rename d8/d52/c3a to d8/d8f/c101 0 2024-04-26T04:53:13.962 INFO:tasks.workunit.client.0.smithi077.stdout:2/704: creat d9/ff3 x:0 0 0 2024-04-26T04:53:13.964 INFO:tasks.workunit.client.0.smithi077.stdout:5/695: mknod d1/d8/d72/cf1 0 2024-04-26T04:53:13.981 INFO:tasks.workunit.client.0.smithi077.stdout:9/606: symlink d9/dd/d3d/d7c/ld2 0 2024-04-26T04:53:13.986 INFO:tasks.workunit.client.0.smithi077.stdout:8/734: getdents d4/df/d6f/da3/dc7 0 2024-04-26T04:53:13.986 INFO:tasks.workunit.client.0.smithi077.stdout:1/665: getdents d9/d53/d8b/d93/da8/da7 0 2024-04-26T04:53:13.988 INFO:tasks.workunit.client.0.smithi077.stdout:8/735: truncate d4/df/d22/d2d/d2b/d33/dbc/feb 4751334 0 2024-04-26T04:53:13.988 INFO:tasks.workunit.client.0.smithi077.stdout:7/703: truncate da/d15/d1a/f79 1242714 0 2024-04-26T04:53:13.989 INFO:tasks.workunit.client.0.smithi077.stdout:7/704: write da/d6b/fc3 [283747,113981] 0 2024-04-26T04:53:13.991 INFO:tasks.workunit.client.0.smithi077.stdout:6/723: mkdir d8/d52/d53/d5e/d6c/d102 0 2024-04-26T04:53:13.992 INFO:tasks.workunit.client.0.smithi077.stdout:6/724: write d8/d52/da0/dad/ff3 [388432,120524] 0 2024-04-26T04:53:13.994 INFO:tasks.workunit.client.0.smithi077.stdout:2/705: creat d9/dc/d5f/ff4 x:0 0 0 2024-04-26T04:53:13.999 INFO:tasks.workunit.client.0.smithi077.stdout:9/607: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3 0 2024-04-26T04:53:14.000 INFO:tasks.workunit.client.0.smithi077.stdout:9/608: chown d9/d3a/d79/f6b 6364 1 2024-04-26T04:53:14.008 INFO:tasks.workunit.client.0.smithi077.stdout:3/584: mkdir d0/dae/db6 0 2024-04-26T04:53:14.008 INFO:tasks.workunit.client.0.smithi077.stdout:8/736: symlink d4/df/d6f/lf6 0 2024-04-26T04:53:14.010 INFO:tasks.workunit.client.0.smithi077.stdout:5/696: mknod d1/d42/da4/dd3/cf2 0 2024-04-26T04:53:14.010 INFO:tasks.workunit.client.0.smithi077.stdout:8/737: write d4/df/dce/d5b/fa5 [1085038,58155] 0 2024-04-26T04:53:14.011 INFO:tasks.workunit.client.0.smithi077.stdout:7/705: dread da/d13/d22/d34/fac [0,4194304] 0 2024-04-26T04:53:14.011 INFO:tasks.workunit.client.0.smithi077.stdout:8/738: stat d4/df/d22/d80 0 2024-04-26T04:53:14.012 INFO:tasks.workunit.client.0.smithi077.stdout:2/706: creat d9/dc/d1d/d34/d39/d63/d79/d2b/dd1/dc3/dc6/dc9/dd0/ff5 x:0 0 0 2024-04-26T04:53:14.014 INFO:tasks.workunit.client.0.smithi077.stdout:0/621: creat d0/d28/fd9 x:0 0 0 2024-04-26T04:53:14.019 INFO:tasks.workunit.client.0.smithi077.stdout:8/739: dwrite d4/df/d22/d2d/d2b/d33/fe1 [0,4194304] 0 2024-04-26T04:53:14.042 INFO:tasks.workunit.client.0.smithi077.stdout:3/585: mkdir d0/d7/d9a/db7 0 2024-04-26T04:53:14.044 INFO:tasks.workunit.client.0.smithi077.stdout:7/706: truncate da/d13/d22/d34/f51 4932589 0 2024-04-26T04:53:14.044 INFO:tasks.workunit.client.0.smithi077.stdout:9/609: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/dd4 0 2024-04-26T04:53:14.046 INFO:tasks.workunit.client.0.smithi077.stdout:0/622: symlink d0/d19/d1e/d2f/d52/lda 0 2024-04-26T04:53:14.046 INFO:tasks.workunit.client.0.smithi077.stdout:9/610: dread d9/d43/f56 [0,4194304] 0 2024-04-26T04:53:14.048 INFO:tasks.workunit.client.0.smithi077.stdout:0/623: stat d0/d47/d64/d97/dba/dd8/dd7 0 2024-04-26T04:53:14.048 INFO:tasks.workunit.client.0.smithi077.stdout:9/611: write d9/f22 [1922963,60969] 0 2024-04-26T04:53:14.048 INFO:tasks.workunit.client.0.smithi077.stdout:1/666: link d9/d53/d8b/c9f d9/d53/dc1/cce 0 2024-04-26T04:53:14.049 INFO:tasks.workunit.client.0.smithi077.stdout:8/740: symlink d4/df/dce/d5b/lf7 0 2024-04-26T04:53:14.050 INFO:tasks.workunit.client.0.smithi077.stdout:9/612: write d9/d45/db1/fc0 [600684,105684] 0 2024-04-26T04:53:14.052 INFO:tasks.workunit.client.0.smithi077.stdout:8/741: chown d4/df/d22/d2d/d2b/l6d 8 1 2024-04-26T04:53:14.054 INFO:tasks.workunit.client.0.smithi077.stdout:8/742: stat d4/df/d22/d2d/d2b/d33/dbc/feb 0 2024-04-26T04:53:14.056 INFO:tasks.workunit.client.0.smithi077.stdout:2/707: dread d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:53:14.057 INFO:tasks.workunit.client.0.smithi077.stdout:9/613: dwrite d9/d3a/d79/f33 [0,4194304] 0 2024-04-26T04:53:14.059 INFO:tasks.workunit.client.0.smithi077.stdout:5/697: symlink d1/d12/lf3 0 2024-04-26T04:53:14.060 INFO:tasks.workunit.client.0.smithi077.stdout:5/698: fdatasync d1/d42/da4/d25/d31/d3c/f68 0 2024-04-26T04:53:14.062 INFO:tasks.workunit.client.0.smithi077.stdout:7/707: rename da/l12 to da/d13/d22/d5d/d9f/da7/de0/le7 0 2024-04-26T04:53:14.069 INFO:tasks.workunit.client.0.smithi077.stdout:7/708: dwrite da/d15/d42/f50 [0,4194304] 0 2024-04-26T04:53:14.163 INFO:tasks.workunit.client.0.smithi077.stdout:6/725: getdents d8/d52/d53/d5e/da6/ddf 0 2024-04-26T04:53:14.179 INFO:tasks.workunit.client.0.smithi077.stdout:8/743: symlink d4/df/d22/lf8 0 2024-04-26T04:53:14.179 INFO:tasks.workunit.client.0.smithi077.stdout:8/744: read d4/d17/f6a [574817,81189] 0 2024-04-26T04:53:14.181 INFO:tasks.workunit.client.0.smithi077.stdout:5/699: dread d1/d42/da4/f17 [0,4194304] 0 2024-04-26T04:53:14.182 INFO:tasks.workunit.client.0.smithi077.stdout:2/708: mkdir d9/dc/d1d/d34/d39/d63/df6 0 2024-04-26T04:53:14.184 INFO:tasks.workunit.client.0.smithi077.stdout:0/624: creat d0/fdb x:0 0 0 2024-04-26T04:53:14.189 INFO:tasks.workunit.client.0.smithi077.stdout:8/745: read d4/df/d18/f4c [119711,31038] 0 2024-04-26T04:53:14.192 INFO:tasks.workunit.client.0.smithi077.stdout:2/709: truncate d9/dc/d1d/d34/f36 1861160 0 2024-04-26T04:53:14.193 INFO:tasks.workunit.client.0.smithi077.stdout:2/710: dread - d9/dc/d1d/d1e/d77/fb5 zero size 2024-04-26T04:53:14.195 INFO:tasks.workunit.client.0.smithi077.stdout:1/667: link d9/db/d2b/d3f/f52 d9/d53/d6e/d7c/fcf 0 2024-04-26T04:53:14.196 INFO:tasks.workunit.client.0.smithi077.stdout:1/668: chown d9/db/dd/d34/d4a/dbb 5393244 1 2024-04-26T04:53:14.199 INFO:tasks.workunit.client.0.smithi077.stdout:2/711: dwrite d9/dc/d1d/d34/d39/d63/d79/f54 [0,4194304] 0 2024-04-26T04:53:14.200 INFO:tasks.workunit.client.0.smithi077.stdout:8/746: creat d4/df/dce/d5b/d8d/ff9 x:0 0 0 2024-04-26T04:53:14.203 INFO:tasks.workunit.client.0.smithi077.stdout:5/700: creat d1/d8c/ff4 x:0 0 0 2024-04-26T04:53:14.210 INFO:tasks.workunit.client.0.smithi077.stdout:0/625: rmdir d0/d47 39 2024-04-26T04:53:14.212 INFO:tasks.workunit.client.0.smithi077.stdout:1/669: rmdir d9/d90/d99 39 2024-04-26T04:53:14.218 INFO:tasks.workunit.client.0.smithi077.stdout:2/712: fsync d9/dc/d1d/d1e/fe2 0 2024-04-26T04:53:14.220 INFO:tasks.workunit.client.0.smithi077.stdout:8/747: unlink d4/df/d18/d56/c65 0 2024-04-26T04:53:14.231 INFO:tasks.workunit.client.0.smithi077.stdout:2/713: creat d9/dc/d5f/d8e/db9/ff7 x:0 0 0 2024-04-26T04:53:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:13 smithi077 ceph-mon[26311]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 104 MiB/s rd, 267 MiB/s wr, 524 op/s 2024-04-26T04:53:14.244 INFO:tasks.workunit.client.0.smithi077.stdout:0/626: mkdir d0/d19/d1e/d2f/ddc 0 2024-04-26T04:53:14.245 INFO:tasks.workunit.client.0.smithi077.stdout:1/670: dread d9/db/dd/d34/f38 [0,4194304] 0 2024-04-26T04:53:14.246 INFO:tasks.workunit.client.0.smithi077.stdout:1/671: stat d9/db/d2b/d3f/f50 0 2024-04-26T04:53:14.247 INFO:tasks.workunit.client.0.smithi077.stdout:1/672: stat d9/d53/d8b/d93/c64 0 2024-04-26T04:53:14.250 INFO:tasks.workunit.client.0.smithi077.stdout:0/627: creat d0/d47/d64/d97/dba/fdd x:0 0 0 2024-04-26T04:53:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:13 smithi078 ceph-mon[33746]: pgmap v26: 65 pgs: 65 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 104 MiB/s rd, 267 MiB/s wr, 524 op/s 2024-04-26T04:53:14.324 INFO:tasks.workunit.client.0.smithi077.stdout:5/701: dwrite d1/d12/d45/f63 [0,4194304] 0 2024-04-26T04:53:14.331 INFO:tasks.workunit.client.0.smithi077.stdout:0/628: truncate d0/d47/d93/d94/faa 468618 0 2024-04-26T04:53:14.335 INFO:tasks.workunit.client.0.smithi077.stdout:1/673: dwrite d9/db/dd/d68/f7b [0,4194304] 0 2024-04-26T04:53:14.339 INFO:tasks.workunit.client.0.smithi077.stdout:1/674: mknod d9/db/dd/d34/cd0 0 2024-04-26T04:53:14.339 INFO:tasks.workunit.client.0.smithi077.stdout:0/629: dwrite d0/d47/f30 [8388608,4194304] 0 2024-04-26T04:53:14.347 INFO:tasks.workunit.client.0.smithi077.stdout:5/702: dread d1/d8/ff [0,4194304] 0 2024-04-26T04:53:14.348 INFO:tasks.workunit.client.0.smithi077.stdout:5/703: chown d1/fc1 379940 1 2024-04-26T04:53:14.352 INFO:tasks.workunit.client.0.smithi077.stdout:1/675: creat d9/db/fd1 x:0 0 0 2024-04-26T04:53:14.352 INFO:tasks.workunit.client.0.smithi077.stdout:0/630: link d0/d19/d1e/d2f/fa4 d0/d19/d1e/d59/fde 0 2024-04-26T04:53:14.362 INFO:tasks.workunit.client.0.smithi077.stdout:9/614: rename d9/d3a/f89 to d9/d3a/d79/d95/dad/fd5 0 2024-04-26T04:53:14.363 INFO:tasks.workunit.client.0.smithi077.stdout:9/615: truncate d9/dd/d3d/faf 587209 0 2024-04-26T04:53:14.364 INFO:tasks.workunit.client.0.smithi077.stdout:0/631: dread d0/d47/d68/fa5 [0,4194304] 0 2024-04-26T04:53:14.365 INFO:tasks.workunit.client.0.smithi077.stdout:7/709: rename da/d13/d22/d34/ca2 to da/d15/ce8 0 2024-04-26T04:53:14.365 INFO:tasks.workunit.client.0.smithi077.stdout:0/632: chown d0/d47/d64/d97/dba/dc4/f73 2727791 1 2024-04-26T04:53:14.367 INFO:tasks.workunit.client.0.smithi077.stdout:9/616: fsync d9/d3a/d79/d95/fb8 0 2024-04-26T04:53:14.369 INFO:tasks.workunit.client.0.smithi077.stdout:6/726: rename d8/d52/d53/f65 to d8/d52/d53/da7/dfa/f103 0 2024-04-26T04:53:14.372 INFO:tasks.workunit.client.0.smithi077.stdout:0/633: creat d0/d47/d64/d97/dba/dc4/fdf x:0 0 0 2024-04-26T04:53:14.377 INFO:tasks.workunit.client.0.smithi077.stdout:6/727: dwrite d8/d52/d53/d5e/da6/ddf/fee [0,4194304] 0 2024-04-26T04:53:14.379 INFO:tasks.workunit.client.0.smithi077.stdout:9/617: creat d9/d3a/fd6 x:0 0 0 2024-04-26T04:53:14.382 INFO:tasks.workunit.client.0.smithi077.stdout:8/748: rename d4/df/d22/d2d/d2b/d33/dbc/feb to d4/df/d6f/da3/dc7/ffa 0 2024-04-26T04:53:14.386 INFO:tasks.workunit.client.0.smithi077.stdout:0/634: truncate d0/d19/d1e/d2f/da8/fab 1223377 0 2024-04-26T04:53:14.390 INFO:tasks.workunit.client.0.smithi077.stdout:6/728: unlink d8/d2a/d86/lde 0 2024-04-26T04:53:14.394 INFO:tasks.workunit.client.0.smithi077.stdout:8/749: unlink d4/df/d18/l4b 0 2024-04-26T04:53:14.398 INFO:tasks.workunit.client.0.smithi077.stdout:9/618: creat d9/d7d/dd1/fd7 x:0 0 0 2024-04-26T04:53:14.399 INFO:tasks.workunit.client.0.smithi077.stdout:6/729: creat d8/d2a/ddd/f104 x:0 0 0 2024-04-26T04:53:14.399 INFO:tasks.workunit.client.0.smithi077.stdout:9/619: dread - d9/d7d/dd1/fd7 zero size 2024-04-26T04:53:14.403 INFO:tasks.workunit.client.0.smithi077.stdout:0/635: creat d0/d19/d69/fe0 x:0 0 0 2024-04-26T04:53:14.404 INFO:tasks.workunit.client.0.smithi077.stdout:0/636: dread - d0/d19/f87 zero size 2024-04-26T04:53:14.404 INFO:tasks.workunit.client.0.smithi077.stdout:0/637: write d0/d47/f30 [12659964,103354] 0 2024-04-26T04:53:14.406 INFO:tasks.workunit.client.0.smithi077.stdout:6/730: truncate d8/d1c/f5c 1612697 0 2024-04-26T04:53:14.410 INFO:tasks.workunit.client.0.smithi077.stdout:2/714: rename d9/dc/d1d/d34/d39/d63/d79/d2b to d9/dc/d1d/d34/df8 0 2024-04-26T04:53:14.410 INFO:tasks.workunit.client.0.smithi077.stdout:5/704: rename d1 to d1/d42/dac/df5 22 2024-04-26T04:53:14.414 INFO:tasks.workunit.client.0.smithi077.stdout:9/620: dread d9/f2d [0,4194304] 0 2024-04-26T04:53:14.415 INFO:tasks.workunit.client.0.smithi077.stdout:6/731: rmdir d8/d52 39 2024-04-26T04:53:14.416 INFO:tasks.workunit.client.0.smithi077.stdout:2/715: mkdir d9/dc/d1d/d1e/d77/df9 0 2024-04-26T04:53:14.419 INFO:tasks.workunit.client.0.smithi077.stdout:9/621: mknod d9/d45/cd8 0 2024-04-26T04:53:14.420 INFO:tasks.workunit.client.0.smithi077.stdout:9/622: chown d9/dd/d3d/d7c/dc8 25680602 1 2024-04-26T04:53:14.422 INFO:tasks.workunit.client.0.smithi077.stdout:6/732: write d8/d52/d53/d5e/fae [4328103,32802] 0 2024-04-26T04:53:14.425 INFO:tasks.workunit.client.0.smithi077.stdout:5/705: dread d1/d42/da4/f34 [0,4194304] 0 2024-04-26T04:53:14.426 INFO:tasks.workunit.client.0.smithi077.stdout:9/623: rename d9/dd/l13 to d9/d43/ld9 0 2024-04-26T04:53:14.429 INFO:tasks.workunit.client.0.smithi077.stdout:6/733: creat d8/d52/d53/d5e/d6c/d102/f105 x:0 0 0 2024-04-26T04:53:14.430 INFO:tasks.workunit.client.0.smithi077.stdout:5/706: mkdir d1/df6 0 2024-04-26T04:53:14.431 INFO:tasks.workunit.client.0.smithi077.stdout:5/707: readlink d1/d12/d87/lc0 0 2024-04-26T04:53:14.432 INFO:tasks.workunit.client.0.smithi077.stdout:6/734: creat d8/d2a/d86/f106 x:0 0 0 2024-04-26T04:53:14.432 INFO:tasks.workunit.client.0.smithi077.stdout:5/708: chown d1/d8/c6a 133841387 1 2024-04-26T04:53:14.433 INFO:tasks.workunit.client.0.smithi077.stdout:2/716: dread d9/dc/d1d/d34/d39/d63/d79/f30 [0,4194304] 0 2024-04-26T04:53:14.433 INFO:tasks.workunit.client.0.smithi077.stdout:6/735: chown d8/d52/d53/d5e/f69 9918928 1 2024-04-26T04:53:14.435 INFO:tasks.workunit.client.0.smithi077.stdout:2/717: symlink d9/d15/d44/d4e/lfa 0 2024-04-26T04:53:14.439 INFO:tasks.workunit.client.0.smithi077.stdout:2/718: unlink d9/d15/d56/fde 0 2024-04-26T04:53:14.440 INFO:tasks.workunit.client.0.smithi077.stdout:2/719: chown d9/dc/d1d/d34/df8/dd1/dc3/fa0 3 1 2024-04-26T04:53:14.442 INFO:tasks.workunit.client.0.smithi077.stdout:9/624: dread d9/d3a/d79/f33 [8388608,4194304] 0 2024-04-26T04:53:14.443 INFO:tasks.workunit.client.0.smithi077.stdout:6/736: link d8/d52/da0/dad/fbe d8/d52/d53/f107 0 2024-04-26T04:53:14.444 INFO:tasks.workunit.client.0.smithi077.stdout:2/720: creat d9/dc/d1d/d34/d39/d63/d79/d5b/ffb x:0 0 0 2024-04-26T04:53:14.447 INFO:tasks.workunit.client.0.smithi077.stdout:6/737: mkdir d8/d1c/d108 0 2024-04-26T04:53:14.449 INFO:tasks.workunit.client.0.smithi077.stdout:6/738: symlink d8/d52/d53/d5e/l109 0 2024-04-26T04:53:14.450 INFO:tasks.workunit.client.0.smithi077.stdout:6/739: fdatasync d8/d1c/f24 0 2024-04-26T04:53:14.454 INFO:tasks.workunit.client.0.smithi077.stdout:2/721: creat d9/dc/d1d/d34/ffc x:0 0 0 2024-04-26T04:53:14.455 INFO:tasks.workunit.client.0.smithi077.stdout:2/722: fdatasync d9/dc/d1d/d34/fd7 0 2024-04-26T04:53:14.459 INFO:tasks.workunit.client.0.smithi077.stdout:2/723: chown d9/dc/f5c 5438755 1 2024-04-26T04:53:14.459 INFO:tasks.workunit.client.0.smithi077.stdout:0/638: truncate d0/f1 2369371 0 2024-04-26T04:53:14.506 INFO:tasks.workunit.client.0.smithi077.stdout:0/639: truncate d0/d47/d64/f6d 762031 0 2024-04-26T04:53:14.506 INFO:tasks.workunit.client.0.smithi077.stdout:6/740: getdents d8/d8f 0 2024-04-26T04:53:14.511 INFO:tasks.workunit.client.0.smithi077.stdout:9/625: rmdir d9/dd/d1b/d2b/d32 39 2024-04-26T04:53:14.518 INFO:tasks.workunit.client.0.smithi077.stdout:2/724: dwrite d9/dc/d1d/d34/d39/d63/f9c [0,4194304] 0 2024-04-26T04:53:14.519 INFO:tasks.workunit.client.0.smithi077.stdout:0/640: dwrite d0/d19/d1e/f20 [0,4194304] 0 2024-04-26T04:53:14.519 INFO:tasks.workunit.client.0.smithi077.stdout:6/741: dwrite d8/d52/d53/da7/fea [0,4194304] 0 2024-04-26T04:53:14.525 INFO:tasks.workunit.client.0.smithi077.stdout:2/725: rmdir d9/dc/d5f/d8e/dcd 39 2024-04-26T04:53:14.526 INFO:tasks.workunit.client.0.smithi077.stdout:2/726: read - d9/dc/fef zero size 2024-04-26T04:53:14.527 INFO:tasks.workunit.client.0.smithi077.stdout:2/727: readlink d9/dc/d1d/d34/d39/d63/l48 0 2024-04-26T04:53:14.528 INFO:tasks.workunit.client.0.smithi077.stdout:9/626: rename d9/dd/d1b/d2b/f72 to d9/dd/d3d/d7c/dc8/fda 0 2024-04-26T04:53:14.534 INFO:tasks.workunit.client.0.smithi077.stdout:0/641: symlink d0/d28/le1 0 2024-04-26T04:53:14.535 INFO:tasks.workunit.client.0.smithi077.stdout:2/728: mknod d9/dc/d1d/d34/d39/d63/cfd 0 2024-04-26T04:53:14.536 INFO:tasks.workunit.client.0.smithi077.stdout:0/642: readlink d0/d19/d1e/d2f/d52/lb5 0 2024-04-26T04:53:14.537 INFO:tasks.workunit.client.0.smithi077.stdout:2/729: stat d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0 0 2024-04-26T04:53:14.538 INFO:tasks.workunit.client.0.smithi077.stdout:0/643: truncate d0/d47/d93/d94/fcd 482427 0 2024-04-26T04:53:14.538 INFO:tasks.workunit.client.0.smithi077.stdout:6/742: rmdir d8/d52/d53/d5e/da6/de8 0 2024-04-26T04:53:14.538 INFO:tasks.workunit.client.0.smithi077.stdout:0/644: stat d0/d47/l8b 0 2024-04-26T04:53:14.539 INFO:tasks.workunit.client.0.smithi077.stdout:0/645: write d0/d47/d64/d97/dba/fdd [164746,116504] 0 2024-04-26T04:53:14.542 INFO:tasks.workunit.client.0.smithi077.stdout:9/627: rename d9/d3a/d79/d50 to d9/dd/d1b/d2b/ddb 0 2024-04-26T04:53:14.553 INFO:tasks.workunit.client.0.smithi077.stdout:0/646: fdatasync d0/d2c/f88 0 2024-04-26T04:53:14.554 INFO:tasks.workunit.client.0.smithi077.stdout:9/628: mknod d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/dd4/cdc 0 2024-04-26T04:53:14.555 INFO:tasks.workunit.client.0.smithi077.stdout:9/629: chown d9/d3a/d79/d95/dad/db2 16304 1 2024-04-26T04:53:14.560 INFO:tasks.workunit.client.0.smithi077.stdout:9/630: rename d9/dd/d3d/ca9 to d9/d45/dc4/cdd 0 2024-04-26T04:53:14.563 INFO:tasks.workunit.client.0.smithi077.stdout:2/730: write f5 [1130715,71245] 0 2024-04-26T04:53:14.564 INFO:tasks.workunit.client.0.smithi077.stdout:2/731: write d9/f17 [2196015,49087] 0 2024-04-26T04:53:14.565 INFO:tasks.workunit.client.0.smithi077.stdout:6/743: write d8/d52/fb3 [918056,13826] 0 2024-04-26T04:53:14.569 INFO:tasks.workunit.client.0.smithi077.stdout:2/732: truncate d9/dc/d1d/f24 3085193 0 2024-04-26T04:53:14.569 INFO:tasks.workunit.client.0.smithi077.stdout:2/733: chown d9/dc/d1d 21 1 2024-04-26T04:53:14.572 INFO:tasks.workunit.client.0.smithi077.stdout:2/734: creat d9/dc/d5f/ffe x:0 0 0 2024-04-26T04:53:14.572 INFO:tasks.workunit.client.0.smithi077.stdout:2/735: stat d9/d15/d56/c8b 0 2024-04-26T04:53:14.575 INFO:tasks.workunit.client.0.smithi077.stdout:9/631: write d9/dd/d1b/f73 [4186,29295] 0 2024-04-26T04:53:14.576 INFO:tasks.workunit.client.0.smithi077.stdout:2/736: symlink d9/dc/d1d/d34/d39/d63/d79/d5b/lff 0 2024-04-26T04:53:14.576 INFO:tasks.workunit.client.0.smithi077.stdout:6/744: link d8/f5a d8/d8f/f10a 0 2024-04-26T04:53:14.579 INFO:tasks.workunit.client.0.smithi077.stdout:0/647: dwrite d0/d19/d1e/d59/fde [0,4194304] 0 2024-04-26T04:53:14.583 INFO:tasks.workunit.client.0.smithi077.stdout:9/632: dread d9/d45/db1/fc0 [0,4194304] 0 2024-04-26T04:53:14.583 INFO:tasks.workunit.client.0.smithi077.stdout:0/648: chown d0/d47/d64 1010337528 1 2024-04-26T04:53:14.585 INFO:tasks.workunit.client.0.smithi077.stdout:2/737: creat d9/dc/d1d/d34/df8/dd1/dc3/dc6/f100 x:0 0 0 2024-04-26T04:53:14.591 INFO:tasks.workunit.client.0.smithi077.stdout:9/633: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/fde x:0 0 0 2024-04-26T04:53:14.595 INFO:tasks.workunit.client.0.smithi077.stdout:6/745: mkdir d8/d2a/d10b 0 2024-04-26T04:53:14.596 INFO:tasks.workunit.client.0.smithi077.stdout:6/746: readlink d8/d52/da0/l8b 0 2024-04-26T04:53:14.596 INFO:tasks.workunit.client.0.smithi077.stdout:2/738: stat d9/dc/c35 0 2024-04-26T04:53:14.597 INFO:tasks.workunit.client.0.smithi077.stdout:9/634: dwrite d9/d35/d99/fa3 [0,4194304] 0 2024-04-26T04:53:14.599 INFO:tasks.workunit.client.0.smithi077.stdout:9/635: write d9/dd/d1b/f73 [907953,9240] 0 2024-04-26T04:53:14.604 INFO:tasks.workunit.client.0.smithi077.stdout:0/649: write d0/d19/d1e/d2f/f44 [1764161,72855] 0 2024-04-26T04:53:14.605 INFO:tasks.workunit.client.0.smithi077.stdout:9/636: symlink d9/d3a/d79/d95/dad/db2/ldf 0 2024-04-26T04:53:14.613 INFO:tasks.workunit.client.0.smithi077.stdout:0/650: mknod d0/d2c/ce2 0 2024-04-26T04:53:14.613 INFO:tasks.workunit.client.0.smithi077.stdout:2/739: link d9/d15/d44/c8f d9/dc/d6e/c101 0 2024-04-26T04:53:14.617 INFO:tasks.workunit.client.0.smithi077.stdout:6/747: dwrite d8/d52/da0/f98 [0,4194304] 0 2024-04-26T04:53:14.619 INFO:tasks.workunit.client.0.smithi077.stdout:2/740: rmdir d9/dc/d1d/d1e 39 2024-04-26T04:53:14.621 INFO:tasks.workunit.client.0.smithi077.stdout:0/651: readlink d0/d19/d1e/d81/la0 0 2024-04-26T04:53:14.624 INFO:tasks.workunit.client.0.smithi077.stdout:9/637: dread d9/d35/f69 [0,4194304] 0 2024-04-26T04:53:14.625 INFO:tasks.workunit.client.0.smithi077.stdout:6/748: rename c2 to d8/d52/d53/da7/dfa/c10c 0 2024-04-26T04:53:14.625 INFO:tasks.workunit.client.0.smithi077.stdout:6/749: chown d8/d52/d53/da7 1251481 1 2024-04-26T04:53:14.627 INFO:tasks.workunit.client.0.smithi077.stdout:0/652: symlink d0/d47/d68/le3 0 2024-04-26T04:53:14.628 INFO:tasks.workunit.client.0.smithi077.stdout:9/638: creat d9/d35/d99/fe0 x:0 0 0 2024-04-26T04:53:14.629 INFO:tasks.workunit.client.0.smithi077.stdout:9/639: stat d9/dd/d1b/d2b/d32/d36/c77 0 2024-04-26T04:53:14.630 INFO:tasks.workunit.client.0.smithi077.stdout:2/741: mknod d9/dc/d5f/d8e/dcd/c102 0 2024-04-26T04:53:14.632 INFO:tasks.workunit.client.0.smithi077.stdout:6/750: creat d8/d55/dc8/f10d x:0 0 0 2024-04-26T04:53:14.633 INFO:tasks.workunit.client.0.smithi077.stdout:6/751: truncate d8/d52/d53/fe1 486947 0 2024-04-26T04:53:14.634 INFO:tasks.workunit.client.0.smithi077.stdout:9/640: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/fe1 x:0 0 0 2024-04-26T04:53:14.634 INFO:tasks.workunit.client.0.smithi077.stdout:6/752: truncate d8/d2a/d86/f106 442289 0 2024-04-26T04:53:14.635 INFO:tasks.workunit.client.0.smithi077.stdout:0/653: symlink d0/d47/le4 0 2024-04-26T04:53:14.638 INFO:tasks.workunit.client.0.smithi077.stdout:2/742: creat d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/f103 x:0 0 0 2024-04-26T04:53:14.639 INFO:tasks.workunit.client.0.smithi077.stdout:9/641: creat d9/d35/d99/d9f/fe2 x:0 0 0 2024-04-26T04:53:14.639 INFO:tasks.workunit.client.0.smithi077.stdout:6/753: dread d8/d52/da0/f98 [0,4194304] 0 2024-04-26T04:53:14.640 INFO:tasks.workunit.client.0.smithi077.stdout:2/743: write d9/dc/d5f/ffe [507227,122076] 0 2024-04-26T04:53:14.642 INFO:tasks.workunit.client.0.smithi077.stdout:6/754: write d8/d2a/d86/f106 [680517,52442] 0 2024-04-26T04:53:14.643 INFO:tasks.workunit.client.0.smithi077.stdout:6/755: write d8/d8f/fe5 [3391948,76744] 0 2024-04-26T04:53:14.650 INFO:tasks.workunit.client.0.smithi077.stdout:6/756: mknod d8/d1c/db5/c10e 0 2024-04-26T04:53:14.650 INFO:tasks.workunit.client.0.smithi077.stdout:2/744: dwrite d9/dc/f7a [4194304,4194304] 0 2024-04-26T04:53:14.652 INFO:tasks.workunit.client.0.smithi077.stdout:6/757: truncate d8/d52/d53/ff0 505970 0 2024-04-26T04:53:14.659 INFO:tasks.workunit.client.0.smithi077.stdout:2/745: dwrite d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/f103 [0,4194304] 0 2024-04-26T04:53:14.661 INFO:tasks.workunit.client.0.smithi077.stdout:6/758: mkdir d8/d1c/da4/d10f 0 2024-04-26T04:53:14.662 INFO:tasks.workunit.client.0.smithi077.stdout:6/759: chown d8 48549020 1 2024-04-26T04:53:14.663 INFO:tasks.workunit.client.0.smithi077.stdout:0/654: getdents d0/d19/d1e/d82 0 2024-04-26T04:53:14.665 INFO:tasks.workunit.client.0.smithi077.stdout:6/760: read d8/f33 [2849330,103422] 0 2024-04-26T04:53:14.668 INFO:tasks.workunit.client.0.smithi077.stdout:9/642: link d9/dd/d3d/d7c/c87 d9/d35/d99/ce3 0 2024-04-26T04:53:14.672 INFO:tasks.workunit.client.0.smithi077.stdout:0/655: dwrite d0/d47/d64/d97/dba/fca [0,4194304] 0 2024-04-26T04:53:14.672 INFO:tasks.workunit.client.0.smithi077.stdout:6/761: rmdir d8/d2a/df2 39 2024-04-26T04:53:14.678 INFO:tasks.workunit.client.0.smithi077.stdout:9/643: mknod d9/d35/d99/d9f/ce4 0 2024-04-26T04:53:14.679 INFO:tasks.workunit.client.0.smithi077.stdout:9/644: readlink d9/d35/d99/lbc 0 2024-04-26T04:53:14.684 INFO:tasks.workunit.client.0.smithi077.stdout:6/762: truncate d8/d52/ff9 1216906 0 2024-04-26T04:53:14.686 INFO:tasks.workunit.client.0.smithi077.stdout:0/656: rename d0/d47/d64/f6d to d0/d47/d68/fe5 0 2024-04-26T04:53:14.688 INFO:tasks.workunit.client.0.smithi077.stdout:6/763: mknod d8/d1c/da4/d10f/c110 0 2024-04-26T04:53:14.690 INFO:tasks.workunit.client.0.smithi077.stdout:0/657: creat d0/d47/d64/d97/dba/dd8/dd7/fe6 x:0 0 0 2024-04-26T04:53:14.691 INFO:tasks.workunit.client.0.smithi077.stdout:6/764: rmdir d8/d1c/da4/d10f 39 2024-04-26T04:53:14.691 INFO:tasks.workunit.client.0.smithi077.stdout:6/765: fdatasync d8/f60 0 2024-04-26T04:53:14.695 INFO:tasks.workunit.client.0.smithi077.stdout:6/766: creat d8/d52/f111 x:0 0 0 2024-04-26T04:53:14.697 INFO:tasks.workunit.client.0.smithi077.stdout:0/658: rmdir d0/d19/d1e/d2f/ddc 0 2024-04-26T04:53:14.698 INFO:tasks.workunit.client.0.smithi077.stdout:0/659: fdatasync d0/d47/fc8 0 2024-04-26T04:53:14.698 INFO:tasks.workunit.client.0.smithi077.stdout:0/660: fdatasync d0/d47/f30 0 2024-04-26T04:53:14.702 INFO:tasks.workunit.client.0.smithi077.stdout:0/661: mknod d0/d19/ce7 0 2024-04-26T04:53:14.704 INFO:tasks.workunit.client.0.smithi077.stdout:0/662: readlink d0/d47/l3f 0 2024-04-26T04:53:14.707 INFO:tasks.workunit.client.0.smithi077.stdout:0/663: dread d0/d19/d69/f84 [0,4194304] 0 2024-04-26T04:53:14.708 INFO:tasks.workunit.client.0.smithi077.stdout:0/664: chown d0/d19/d1e/d2f/lb7 597306 1 2024-04-26T04:53:14.709 INFO:tasks.workunit.client.0.smithi077.stdout:0/665: readlink d0/l6 0 2024-04-26T04:53:14.794 INFO:tasks.workunit.client.0.smithi077.stdout:0/666: write d0/f1 [1749632,77449] 0 2024-04-26T04:53:14.795 INFO:tasks.workunit.client.0.smithi077.stdout:0/667: stat d0/d28/d40 0 2024-04-26T04:53:14.797 INFO:tasks.workunit.client.0.smithi077.stdout:0/668: creat d0/d19/d69/fe8 x:0 0 0 2024-04-26T04:53:14.803 INFO:tasks.workunit.client.0.smithi077.stdout:0/669: dread d0/d47/d93/f9a [0,4194304] 0 2024-04-26T04:53:14.805 INFO:tasks.workunit.client.0.smithi077.stdout:0/670: rmdir d0/d2c 39 2024-04-26T04:53:14.810 INFO:tasks.workunit.client.0.smithi077.stdout:0/671: dwrite d0/d19/d1e/d59/fb9 [0,4194304] 0 2024-04-26T04:53:14.811 INFO:tasks.workunit.client.0.smithi077.stdout:0/672: dread - d0/fdb zero size 2024-04-26T04:53:14.812 INFO:tasks.workunit.client.0.smithi077.stdout:0/673: fdatasync d0/d19/d1e/d2f/d52/f62 0 2024-04-26T04:53:14.820 INFO:tasks.workunit.client.0.smithi077.stdout:0/674: dread d0/d47/d93/d94/faa [0,4194304] 0 2024-04-26T04:53:14.823 INFO:tasks.workunit.client.0.smithi077.stdout:0/675: symlink d0/d19/d1e/d59/le9 0 2024-04-26T04:53:14.825 INFO:tasks.workunit.client.0.smithi077.stdout:0/676: truncate d0/d2c/f2e 4499185 0 2024-04-26T04:53:14.826 INFO:tasks.workunit.client.0.smithi077.stdout:0/677: write d0/d19/d69/fe0 [650402,127042] 0 2024-04-26T04:53:14.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/678: rename d0/d19/d1e/d2f/lb7 to d0/d2c/lea 0 2024-04-26T04:53:14.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/679: fdatasync d0/d47/d64/f80 0 2024-04-26T04:53:14.835 INFO:tasks.workunit.client.0.smithi077.stdout:0/680: write d0/d19/d1e/d59/d96/f99 [891850,91714] 0 2024-04-26T04:53:14.840 INFO:tasks.workunit.client.0.smithi077.stdout:0/681: mknod d0/d28/ceb 0 2024-04-26T04:53:14.841 INFO:tasks.workunit.client.0.smithi077.stdout:0/682: write d0/d19/d1e/d59/fde [3945884,114651] 0 2024-04-26T04:53:14.844 INFO:tasks.workunit.client.0.smithi077.stdout:0/683: creat d0/d47/dce/fec x:0 0 0 2024-04-26T04:53:14.847 INFO:tasks.workunit.client.0.smithi077.stdout:0/684: rename d0/d47/d93/cc0 to d0/d47/d68/d9f/ced 0 2024-04-26T04:53:14.850 INFO:tasks.workunit.client.0.smithi077.stdout:0/685: mkdir d0/d19/d8d/dee 0 2024-04-26T04:53:14.862 INFO:tasks.workunit.client.0.smithi077.stdout:0/686: dwrite d0/d47/d93/f9a [0,4194304] 0 2024-04-26T04:53:14.865 INFO:tasks.workunit.client.0.smithi077.stdout:0/687: readlink d0/d19/d1e/d2f/d52/lbd 0 2024-04-26T04:53:14.947 INFO:tasks.workunit.client.0.smithi077.stdout:4/715: sync 2024-04-26T04:53:14.953 INFO:tasks.workunit.client.0.smithi077.stdout:4/716: dwrite d4/d4e/dad/fd7 [4194304,4194304] 0 2024-04-26T04:53:14.958 INFO:tasks.workunit.client.0.smithi077.stdout:4/717: creat d4/d17/d18/d43/d63/d80/db6/fee x:0 0 0 2024-04-26T04:53:14.960 INFO:tasks.workunit.client.0.smithi077.stdout:4/718: fsync d4/d17/d18/d43/d51/db4/d15/d1f/fd3 0 2024-04-26T04:53:14.961 INFO:tasks.workunit.client.0.smithi077.stdout:4/719: chown d4/d17/d18/d43/d51/db4/d15/d1f/d25/c55 130 1 2024-04-26T04:53:14.964 INFO:tasks.workunit.client.0.smithi077.stdout:4/720: stat d4/d17/d18/d43/d51/db4/d15/d1f/d35/fc1 0 2024-04-26T04:53:14.965 INFO:tasks.workunit.client.0.smithi077.stdout:4/721: fsync d4/d17/d18/d43/d6e/f95 0 2024-04-26T04:53:14.971 INFO:tasks.workunit.client.0.smithi077.stdout:4/722: dwrite d4/d17/d18/d43/d63/d80/fcc [0,4194304] 0 2024-04-26T04:53:14.978 INFO:tasks.workunit.client.0.smithi077.stdout:4/723: rename d4/d17/d18/d43/d53/d56/c61 to d4/d4e/dad/cef 0 2024-04-26T04:53:14.979 INFO:tasks.workunit.client.0.smithi077.stdout:4/724: chown d4/d17/d18/d43/d6e/f84 16966916 1 2024-04-26T04:53:14.980 INFO:tasks.workunit.client.0.smithi077.stdout:4/725: chown d4/d17/d18/d43/d51/db4/d15/d1f/dd8/le6 14 1 2024-04-26T04:53:14.985 INFO:tasks.workunit.client.0.smithi077.stdout:4/726: symlink d4/d17/d18/d43/d6e/d6c/dda/lf0 0 2024-04-26T04:53:14.988 INFO:tasks.workunit.client.0.smithi077.stdout:4/727: mknod d4/d17/d18/d43/d6e/d6c/dae/dc5/cf1 0 2024-04-26T04:53:14.990 INFO:tasks.workunit.client.0.smithi077.stdout:4/728: mknod d4/d17/d18/d43/d6e/d6c/dae/dc5/cf2 0 2024-04-26T04:53:14.992 INFO:tasks.workunit.client.0.smithi077.stdout:4/729: write d4/d17/d18/d43/d51/db4/d15/d1f/fe7 [5240111,71793] 0 2024-04-26T04:53:15.101 INFO:tasks.workunit.client.0.smithi077.stdout:3/586: sync 2024-04-26T04:53:15.103 INFO:tasks.workunit.client.0.smithi077.stdout:3/587: mkdir d0/d9/d32/d29/db8 0 2024-04-26T04:53:15.106 INFO:tasks.workunit.client.0.smithi077.stdout:3/588: mknod d0/d9/d32/d29/db8/cb9 0 2024-04-26T04:53:15.107 INFO:tasks.workunit.client.0.smithi077.stdout:3/589: write d0/d7/d9a/db3/f3f [3994992,12691] 0 2024-04-26T04:53:15.110 INFO:tasks.workunit.client.0.smithi077.stdout:3/590: symlink d0/d7/d1f/d81/lba 0 2024-04-26T04:53:15.120 INFO:tasks.workunit.client.0.smithi077.stdout:3/591: write d0/d9/d32/f21 [718775,129962] 0 2024-04-26T04:53:15.126 INFO:tasks.workunit.client.0.smithi077.stdout:3/592: rename d0/d7/d1f/d4d/f82 to d0/d7/d1f/d81/fbb 0 2024-04-26T04:53:15.132 INFO:tasks.workunit.client.0.smithi077.stdout:3/593: rename d0/d9/d32/d29/lb5 to d0/d9/lbc 0 2024-04-26T04:53:15.134 INFO:tasks.workunit.client.0.smithi077.stdout:3/594: mknod d0/dae/db6/cbd 0 2024-04-26T04:53:15.136 INFO:tasks.workunit.client.0.smithi077.stdout:3/595: mkdir d0/d7/d1f/d4d/dbe 0 2024-04-26T04:53:15.139 INFO:tasks.workunit.client.0.smithi077.stdout:3/596: fdatasync d0/d7/d1f/d4d/f9b 0 2024-04-26T04:53:15.144 INFO:tasks.workunit.client.0.smithi077.stdout:3/597: rmdir d0/d9/d8e 0 2024-04-26T04:53:15.147 INFO:tasks.workunit.client.0.smithi077.stdout:3/598: unlink d0/d9/d32/f8a 0 2024-04-26T04:53:15.147 INFO:tasks.workunit.client.0.smithi077.stdout:3/599: dread - d0/d7/d1f/d76/f9f zero size 2024-04-26T04:53:15.148 INFO:tasks.workunit.client.0.smithi077.stdout:3/600: readlink d0/d7/d9a/db3/la1 0 2024-04-26T04:53:15.149 INFO:tasks.workunit.client.0.smithi077.stdout:3/601: chown d0/c13 9 1 2024-04-26T04:53:15.151 INFO:tasks.workunit.client.0.smithi077.stdout:3/602: read d0/d7/d1f/d4d/f92 [179496,41108] 0 2024-04-26T04:53:15.152 INFO:tasks.workunit.client.0.smithi077.stdout:3/603: fsync d0/d9/d32/d29/f6a 0 2024-04-26T04:53:15.154 INFO:tasks.workunit.client.0.smithi077.stdout:3/604: mkdir d0/d9/d32/d29/db8/dbf 0 2024-04-26T04:53:15.155 INFO:tasks.workunit.client.0.smithi077.stdout:3/605: chown d0/d9/d32/f35 5185 1 2024-04-26T04:53:15.169 INFO:tasks.workunit.client.0.smithi077.stdout:3/606: rmdir d0/d9/d32 39 2024-04-26T04:53:15.171 INFO:tasks.workunit.client.0.smithi077.stdout:3/607: mknod d0/d7/cc0 0 2024-04-26T04:53:15.187 INFO:tasks.workunit.client.0.smithi077.stdout:3/608: write d0/d7/f3e [2408911,86664] 0 2024-04-26T04:53:15.189 INFO:tasks.workunit.client.0.smithi077.stdout:1/676: sync 2024-04-26T04:53:15.189 INFO:tasks.workunit.client.0.smithi077.stdout:8/750: sync 2024-04-26T04:53:15.189 INFO:tasks.workunit.client.0.smithi077.stdout:7/710: sync 2024-04-26T04:53:15.190 INFO:tasks.workunit.client.0.smithi077.stdout:1/677: readlink d9/db/dd/lc6 0 2024-04-26T04:53:15.192 INFO:tasks.workunit.client.0.smithi077.stdout:8/751: write d4/df/d22/d80/ff3 [720055,22381] 0 2024-04-26T04:53:15.192 INFO:tasks.workunit.client.0.smithi077.stdout:1/678: chown d9/d53/d8b/d93/f6d 8776 1 2024-04-26T04:53:15.193 INFO:tasks.workunit.client.0.smithi077.stdout:3/609: link d0/d7/d1f/d4d/c75 d0/cc1 0 2024-04-26T04:53:15.193 INFO:tasks.workunit.client.0.smithi077.stdout:8/752: dread - d4/df/dce/d5b/d8d/fee zero size 2024-04-26T04:53:15.195 INFO:tasks.workunit.client.0.smithi077.stdout:1/679: symlink d9/db/dd/ld2 0 2024-04-26T04:53:15.198 INFO:tasks.workunit.client.0.smithi077.stdout:8/753: fdatasync d4/df/f8c 0 2024-04-26T04:53:15.201 INFO:tasks.workunit.client.0.smithi077.stdout:1/680: mkdir d9/d53/d6e/dd3 0 2024-04-26T04:53:15.206 INFO:tasks.workunit.client.0.smithi077.stdout:3/610: creat d0/d9/d32/d29/fc2 x:0 0 0 2024-04-26T04:53:15.207 INFO:tasks.workunit.client.0.smithi077.stdout:7/711: write da/d13/d22/d5d/f9a [127778,99197] 0 2024-04-26T04:53:15.212 INFO:tasks.workunit.client.0.smithi077.stdout:8/754: chown d4/df/d22/d2d/c53 3315 1 2024-04-26T04:53:15.216 INFO:tasks.workunit.client.0.smithi077.stdout:7/712: symlink da/d13/d22/d5d/d9f/le9 0 2024-04-26T04:53:15.219 INFO:tasks.workunit.client.0.smithi077.stdout:3/611: rename d0/d7/d1f/d81/lba to d0/d7/d9a/lc3 0 2024-04-26T04:53:15.222 INFO:tasks.workunit.client.0.smithi077.stdout:7/713: rename da/d15/d1a/dd7 to da/d13/d22/d5d/dea 0 2024-04-26T04:53:15.223 INFO:tasks.workunit.client.0.smithi077.stdout:3/612: write d0/d9/f84 [5176862,92788] 0 2024-04-26T04:53:15.223 INFO:tasks.workunit.client.0.smithi077.stdout:1/681: link d9/d53/d6e/d7c/lb0 d9/d53/dc1/ld4 0 2024-04-26T04:53:15.226 INFO:tasks.workunit.client.0.smithi077.stdout:7/714: truncate da/d15/f32 4266415 0 2024-04-26T04:53:15.238 INFO:tasks.workunit.client.0.smithi077.stdout:1/682: write d9/d53/d6e/f87 [6060120,21222] 0 2024-04-26T04:53:15.238 INFO:tasks.workunit.client.0.smithi077.stdout:7/715: creat da/d13/feb x:0 0 0 2024-04-26T04:53:15.246 INFO:tasks.workunit.client.0.smithi077.stdout:3/613: link d0/d7/l83 d0/lc4 0 2024-04-26T04:53:15.246 INFO:tasks.workunit.client.0.smithi077.stdout:1/683: rename d9/db/dd/f23 to d9/d90/d99/fd5 0 2024-04-26T04:53:15.251 INFO:tasks.workunit.client.0.smithi077.stdout:7/716: dread da/d6b/f93 [0,4194304] 0 2024-04-26T04:53:15.252 INFO:tasks.workunit.client.0.smithi077.stdout:7/717: chown da/d13/d22/d34/fa0 7 1 2024-04-26T04:53:15.253 INFO:tasks.workunit.client.0.smithi077.stdout:1/684: mkdir d9/d90/d99/d9a/dd6 0 2024-04-26T04:53:15.257 INFO:tasks.workunit.client.0.smithi077.stdout:7/718: mknod da/d15/dc6/cec 0 2024-04-26T04:53:15.257 INFO:tasks.workunit.client.0.smithi077.stdout:7/719: chown da/d15/d1a/dcc 6 1 2024-04-26T04:53:15.261 INFO:tasks.workunit.client.0.smithi077.stdout:7/720: symlink da/d15/d1a/d39/d96/led 0 2024-04-26T04:53:15.262 INFO:tasks.workunit.client.0.smithi077.stdout:7/721: fdatasync da/d13/d22/f46 0 2024-04-26T04:53:15.269 INFO:tasks.workunit.client.0.smithi077.stdout:7/722: link da/d13/feb da/d6b/d63/d64/db0/fee 0 2024-04-26T04:53:15.269 INFO:tasks.workunit.client.0.smithi077.stdout:3/614: write d0/d7/f54 [3819443,48295] 0 2024-04-26T04:53:15.270 INFO:tasks.workunit.client.0.smithi077.stdout:3/615: write d0/f38 [1225635,73439] 0 2024-04-26T04:53:15.272 INFO:tasks.workunit.client.0.smithi077.stdout:7/723: unlink da/d13/c38 0 2024-04-26T04:53:15.274 INFO:tasks.workunit.client.0.smithi077.stdout:3/616: mkdir d0/d7/d1f/d81/dc5 0 2024-04-26T04:53:15.280 INFO:tasks.workunit.client.0.smithi077.stdout:7/724: dwrite da/d15/d42/f77 [4194304,4194304] 0 2024-04-26T04:53:15.282 INFO:tasks.workunit.client.0.smithi077.stdout:3/617: dwrite d0/d7/d1f/d4d/f92 [0,4194304] 0 2024-04-26T04:53:15.285 INFO:tasks.workunit.client.0.smithi077.stdout:3/618: mkdir d0/d7/d1f/d4d/dc6 0 2024-04-26T04:53:15.287 INFO:tasks.workunit.client.0.smithi077.stdout:3/619: chown d0/d7/c80 271688 1 2024-04-26T04:53:15.287 INFO:tasks.workunit.client.0.smithi077.stdout:7/725: rmdir da/d13/d22/d5d/dea 0 2024-04-26T04:53:15.289 INFO:tasks.workunit.client.0.smithi077.stdout:3/620: mkdir d0/d7/d1f/d81/dc5/dc7 0 2024-04-26T04:53:15.290 INFO:tasks.workunit.client.0.smithi077.stdout:7/726: creat da/d6b/d63/fef x:0 0 0 2024-04-26T04:53:15.299 INFO:tasks.workunit.client.0.smithi077.stdout:1/685: truncate f7 3233102 0 2024-04-26T04:53:15.320 INFO:tasks.workunit.client.0.smithi077.stdout:3/621: dread d0/f1 [4194304,4194304] 0 2024-04-26T04:53:15.322 INFO:tasks.workunit.client.0.smithi077.stdout:7/727: dwrite da/d13/f30 [0,4194304] 0 2024-04-26T04:53:15.326 INFO:tasks.workunit.client.0.smithi077.stdout:3/622: mkdir d0/d9/d32/d29/dc8 0 2024-04-26T04:53:15.331 INFO:tasks.workunit.client.0.smithi077.stdout:3/623: creat d0/d7/d9a/fc9 x:0 0 0 2024-04-26T04:53:15.332 INFO:tasks.workunit.client.0.smithi077.stdout:7/728: creat da/d15/d1a/dcc/ff0 x:0 0 0 2024-04-26T04:53:15.338 INFO:tasks.workunit.client.0.smithi077.stdout:7/729: link da/d13/d22/d5d/d8d/lb8 da/d15/lf1 0 2024-04-26T04:53:15.342 INFO:tasks.workunit.client.0.smithi077.stdout:7/730: mkdir da/d13/d22/d5d/dbd/df2 0 2024-04-26T04:53:15.343 INFO:tasks.workunit.client.0.smithi077.stdout:5/709: sync 2024-04-26T04:53:15.343 INFO:tasks.workunit.client.0.smithi077.stdout:7/731: chown da/f89 30 1 2024-04-26T04:53:15.344 INFO:tasks.workunit.client.0.smithi077.stdout:3/624: write d0/fb [8947665,15976] 0 2024-04-26T04:53:15.344 INFO:tasks.workunit.client.0.smithi077.stdout:7/732: rmdir da/d15/d1a 39 2024-04-26T04:53:15.355 INFO:tasks.workunit.client.0.smithi077.stdout:7/733: truncate da/d6b/f93 2825228 0 2024-04-26T04:53:15.356 INFO:tasks.workunit.client.0.smithi077.stdout:5/710: creat d1/d30/d35/ff7 x:0 0 0 2024-04-26T04:53:15.358 INFO:tasks.workunit.client.0.smithi077.stdout:3/625: creat d0/d9/fca x:0 0 0 2024-04-26T04:53:15.360 INFO:tasks.workunit.client.0.smithi077.stdout:5/711: readlink d1/d12/d28/ld2 0 2024-04-26T04:53:15.361 INFO:tasks.workunit.client.0.smithi077.stdout:7/734: mkdir da/d15/d1a/d39/df3 0 2024-04-26T04:53:15.371 INFO:tasks.workunit.client.0.smithi077.stdout:7/735: creat da/d13/ff4 x:0 0 0 2024-04-26T04:53:15.374 INFO:tasks.workunit.client.0.smithi077.stdout:3/626: dread d0/d9/d32/d29/f31 [4194304,4194304] 0 2024-04-26T04:53:15.375 INFO:tasks.workunit.client.0.smithi077.stdout:7/736: unlink da/d13/d22/d34/c5e 0 2024-04-26T04:53:15.375 INFO:tasks.workunit.client.0.smithi077.stdout:3/627: read d0/d7/f33 [624195,50959] 0 2024-04-26T04:53:15.376 INFO:tasks.workunit.client.0.smithi077.stdout:3/628: write d0/d9/d32/d2a/faa [6588577,31564] 0 2024-04-26T04:53:15.380 INFO:tasks.workunit.client.0.smithi077.stdout:3/629: fsync d0/d9/f30 0 2024-04-26T04:53:15.383 INFO:tasks.workunit.client.0.smithi077.stdout:3/630: symlink d0/d9/d32/d29/lcb 0 2024-04-26T04:53:15.389 INFO:tasks.workunit.client.0.smithi077.stdout:5/712: write d1/d12/d28/f9f [1935872,60195] 0 2024-04-26T04:53:15.390 INFO:tasks.workunit.client.0.smithi077.stdout:7/737: dread da/d13/d22/f99 [0,4194304] 0 2024-04-26T04:53:15.397 INFO:tasks.workunit.client.0.smithi077.stdout:5/713: rmdir d1 39 2024-04-26T04:53:15.400 INFO:tasks.workunit.client.0.smithi077.stdout:5/714: write d1/fc1 [898815,85752] 0 2024-04-26T04:53:15.402 INFO:tasks.workunit.client.0.smithi077.stdout:3/631: dread d0/d7/d1f/f72 [0,4194304] 0 2024-04-26T04:53:15.404 INFO:tasks.workunit.client.0.smithi077.stdout:7/738: creat da/d6b/d63/ff5 x:0 0 0 2024-04-26T04:53:15.409 INFO:tasks.workunit.client.0.smithi077.stdout:3/632: dread d0/d7/d1f/d4d/f56 [0,4194304] 0 2024-04-26T04:53:15.410 INFO:tasks.workunit.client.0.smithi077.stdout:7/739: dread da/fc [4194304,4194304] 0 2024-04-26T04:53:15.411 INFO:tasks.workunit.client.0.smithi077.stdout:3/633: mkdir d0/d7/d9a/dcc 0 2024-04-26T04:53:15.411 INFO:tasks.workunit.client.0.smithi077.stdout:7/740: dread - da/d13/d22/d5d/dbd/fdc zero size 2024-04-26T04:53:15.413 INFO:tasks.workunit.client.0.smithi077.stdout:7/741: write da/d13/d22/d5d/f9a [1676787,69530] 0 2024-04-26T04:53:15.415 INFO:tasks.workunit.client.0.smithi077.stdout:3/634: symlink d0/d7/d1f/d76/lcd 0 2024-04-26T04:53:15.416 INFO:tasks.workunit.client.0.smithi077.stdout:3/635: read d0/f38 [6869514,96355] 0 2024-04-26T04:53:15.416 INFO:tasks.workunit.client.0.smithi077.stdout:3/636: chown d0/f87 14859 1 2024-04-26T04:53:15.417 INFO:tasks.workunit.client.0.smithi077.stdout:5/715: dread d1/d42/da4/d25/d31/d3c/f68 [0,4194304] 0 2024-04-26T04:53:15.417 INFO:tasks.workunit.client.0.smithi077.stdout:7/742: mkdir da/df6 0 2024-04-26T04:53:15.417 INFO:tasks.workunit.client.0.smithi077.stdout:5/716: readlink d1/d42/l4c 0 2024-04-26T04:53:15.424 INFO:tasks.workunit.client.0.smithi077.stdout:5/717: chown d1/d42/d62/dc8 3936 1 2024-04-26T04:53:15.425 INFO:tasks.workunit.client.0.smithi077.stdout:3/637: dwrite d0/d9/f78 [0,4194304] 0 2024-04-26T04:53:15.425 INFO:tasks.workunit.client.0.smithi077.stdout:5/718: dread - d1/d42/dac/fcf zero size 2024-04-26T04:53:15.433 INFO:tasks.workunit.client.0.smithi077.stdout:7/743: dread da/d15/f2f [0,4194304] 0 2024-04-26T04:53:15.436 INFO:tasks.workunit.client.0.smithi077.stdout:3/638: mkdir d0/d7/dce 0 2024-04-26T04:53:15.437 INFO:tasks.workunit.client.0.smithi077.stdout:3/639: chown d0/d7/f33 253861273 1 2024-04-26T04:53:15.438 INFO:tasks.workunit.client.0.smithi077.stdout:3/640: chown d0/d7/d9a/db3/f65 4 1 2024-04-26T04:53:15.438 INFO:tasks.workunit.client.0.smithi077.stdout:7/744: mknod da/d15/d1a/d39/d96/cf7 0 2024-04-26T04:53:15.445 INFO:tasks.workunit.client.0.smithi077.stdout:3/641: dread d0/d9/f78 [0,4194304] 0 2024-04-26T04:53:15.446 INFO:tasks.workunit.client.0.smithi077.stdout:3/642: write d0/d9/fca [831561,77784] 0 2024-04-26T04:53:15.446 INFO:tasks.workunit.client.0.smithi077.stdout:3/643: write d0/d7/d1f/d4d/f9b [2026253,8888] 0 2024-04-26T04:53:15.448 INFO:tasks.workunit.client.0.smithi077.stdout:7/745: link da/d15/d1a/d39/cc5 da/d15/d1a/d39/d96/cf8 0 2024-04-26T04:53:15.451 INFO:tasks.workunit.client.0.smithi077.stdout:3/644: readlink d0/d7/l83 0 2024-04-26T04:53:15.453 INFO:tasks.workunit.client.0.smithi077.stdout:3/645: write d0/d9/d32/d2a/faa [7835254,73506] 0 2024-04-26T04:53:15.453 INFO:tasks.workunit.client.0.smithi077.stdout:7/746: rmdir da/d13/d22/d7d 39 2024-04-26T04:53:15.463 INFO:tasks.workunit.client.0.smithi077.stdout:3/646: dwrite d0/f38 [0,4194304] 0 2024-04-26T04:53:15.464 INFO:tasks.workunit.client.0.smithi077.stdout:3/647: write d0/f87 [4328927,93066] 0 2024-04-26T04:53:15.466 INFO:tasks.workunit.client.0.smithi077.stdout:3/648: write d0/d7/d1f/d76/f9f [160467,34975] 0 2024-04-26T04:53:15.467 INFO:tasks.workunit.client.0.smithi077.stdout:3/649: chown d0/d7/f3e 40 1 2024-04-26T04:53:15.469 INFO:tasks.workunit.client.0.smithi077.stdout:3/650: stat d0/d7/d1f/d76/f8d 0 2024-04-26T04:53:15.471 INFO:tasks.workunit.client.0.smithi077.stdout:3/651: fsync d0/d7/d9a/db3/f44 0 2024-04-26T04:53:15.474 INFO:tasks.workunit.client.0.smithi077.stdout:3/652: unlink d0/f7a 0 2024-04-26T04:53:15.484 INFO:tasks.workunit.client.0.smithi077.stdout:7/747: dwrite da/d6b/d63/fe3 [0,4194304] 0 2024-04-26T04:53:15.488 INFO:tasks.workunit.client.0.smithi077.stdout:3/653: write d0/d9/d32/f3b [321553,67873] 0 2024-04-26T04:53:15.493 INFO:tasks.workunit.client.0.smithi077.stdout:7/748: getdents da/d13/d22/d5d/dbd 0 2024-04-26T04:53:15.494 INFO:tasks.workunit.client.0.smithi077.stdout:3/654: creat d0/d7/d1f/fcf x:0 0 0 2024-04-26T04:53:15.495 INFO:tasks.workunit.client.0.smithi077.stdout:3/655: chown d0/d7/d9a/db3/c2e 2860 1 2024-04-26T04:53:15.496 INFO:tasks.workunit.client.0.smithi077.stdout:3/656: write d0/f94 [451114,85980] 0 2024-04-26T04:53:15.499 INFO:tasks.workunit.client.0.smithi077.stdout:3/657: creat d0/dae/db6/fd0 x:0 0 0 2024-04-26T04:53:15.502 INFO:tasks.workunit.client.0.smithi077.stdout:3/658: mkdir d0/d7/d1f/d81/dc5/dd1 0 2024-04-26T04:53:15.509 INFO:tasks.workunit.client.0.smithi077.stdout:3/659: dwrite d0/d7/d1f/d76/f7e [4194304,4194304] 0 2024-04-26T04:53:15.515 INFO:tasks.workunit.client.0.smithi077.stdout:7/749: dread da/d15/d42/f84 [0,4194304] 0 2024-04-26T04:53:15.519 INFO:tasks.workunit.client.0.smithi077.stdout:7/750: creat da/d13/d22/d5d/dbd/df2/ff9 x:0 0 0 2024-04-26T04:53:15.523 INFO:tasks.workunit.client.0.smithi077.stdout:7/751: unlink da/d13/d22/c54 0 2024-04-26T04:53:15.529 INFO:tasks.workunit.client.0.smithi077.stdout:7/752: mknod da/d13/cfa 0 2024-04-26T04:53:15.530 INFO:tasks.workunit.client.0.smithi077.stdout:7/753: truncate da/d13/f6d 8404187 0 2024-04-26T04:53:15.531 INFO:tasks.workunit.client.0.smithi077.stdout:7/754: dread - da/d13/d22/d5d/d9f/fd5 zero size 2024-04-26T04:53:15.534 INFO:tasks.workunit.client.0.smithi077.stdout:7/755: creat da/d13/d22/d5d/d9f/da7/de0/ffb x:0 0 0 2024-04-26T04:53:15.535 INFO:tasks.workunit.client.0.smithi077.stdout:3/660: dread d0/d7/f34 [0,4194304] 0 2024-04-26T04:53:15.537 INFO:tasks.workunit.client.0.smithi077.stdout:7/756: mkdir da/d13/d22/d5d/dfc 0 2024-04-26T04:53:15.543 INFO:tasks.workunit.client.0.smithi077.stdout:3/661: link d0/d9/d32/f3b d0/d9/d32/d29/fd2 0 2024-04-26T04:53:15.545 INFO:tasks.workunit.client.0.smithi077.stdout:7/757: mkdir da/d13/d22/d7d/dfd 0 2024-04-26T04:53:15.545 INFO:tasks.workunit.client.0.smithi077.stdout:7/758: fsync da/d15/d1a/f20 0 2024-04-26T04:53:15.548 INFO:tasks.workunit.client.0.smithi077.stdout:3/662: mkdir d0/d9/d32/d2a/d85/dd3 0 2024-04-26T04:53:15.549 INFO:tasks.workunit.client.0.smithi077.stdout:7/759: dread da/d13/d22/f99 [0,4194304] 0 2024-04-26T04:53:15.550 INFO:tasks.workunit.client.0.smithi077.stdout:7/760: fsync da/d13/d22/f46 0 2024-04-26T04:53:15.550 INFO:tasks.workunit.client.0.smithi077.stdout:7/761: readlink da/d13/d22/lb7 0 2024-04-26T04:53:15.551 INFO:tasks.workunit.client.0.smithi077.stdout:3/663: rmdir d0/d7 39 2024-04-26T04:53:15.554 INFO:tasks.workunit.client.0.smithi077.stdout:7/762: dread da/d13/d22/f46 [0,4194304] 0 2024-04-26T04:53:15.556 INFO:tasks.workunit.client.0.smithi077.stdout:3/664: symlink d0/d9/d32/d29/dc8/ld4 0 2024-04-26T04:53:15.557 INFO:tasks.workunit.client.0.smithi077.stdout:3/665: dread - d0/d9/d32/d29/fc2 zero size 2024-04-26T04:53:15.557 INFO:tasks.workunit.client.0.smithi077.stdout:7/763: mknod da/d15/dc6/cfe 0 2024-04-26T04:53:15.560 INFO:tasks.workunit.client.0.smithi077.stdout:7/764: creat da/d15/dc6/fff x:0 0 0 2024-04-26T04:53:15.574 INFO:tasks.workunit.client.0.smithi077.stdout:3/666: dread d0/d7/f3e [0,4194304] 0 2024-04-26T04:53:15.575 INFO:tasks.workunit.client.0.smithi077.stdout:3/667: stat d0/d7/d1f/d4d/dbe 0 2024-04-26T04:53:15.575 INFO:tasks.workunit.client.0.smithi077.stdout:7/765: dread da/d13/fd3 [0,4194304] 0 2024-04-26T04:53:15.576 INFO:tasks.workunit.client.0.smithi077.stdout:7/766: truncate da/d15/f44 5163521 0 2024-04-26T04:53:15.581 INFO:tasks.workunit.client.0.smithi077.stdout:7/767: mknod da/d13/d22/d5d/dfc/c100 0 2024-04-26T04:53:15.586 INFO:tasks.workunit.client.0.smithi077.stdout:7/768: dread da/d13/d22/fd2 [0,4194304] 0 2024-04-26T04:53:15.588 INFO:tasks.workunit.client.0.smithi077.stdout:3/668: write d0/d7/d1f/d4d/f5f [1018317,2173] 0 2024-04-26T04:53:15.589 INFO:tasks.workunit.client.0.smithi077.stdout:3/669: write d0/d7/d9a/db3/f20 [5939982,118622] 0 2024-04-26T04:53:15.590 INFO:tasks.workunit.client.0.smithi077.stdout:7/769: unlink da/d6b/d63/d64/db0/ld4 0 2024-04-26T04:53:15.593 INFO:tasks.workunit.client.0.smithi077.stdout:7/770: rename da/d15/f48 to da/d13/d22/d34/f101 0 2024-04-26T04:53:15.595 INFO:tasks.workunit.client.0.smithi077.stdout:3/670: creat d0/d7/fd5 x:0 0 0 2024-04-26T04:53:15.596 INFO:tasks.workunit.client.0.smithi077.stdout:2/746: sync 2024-04-26T04:53:15.596 INFO:tasks.workunit.client.0.smithi077.stdout:9/645: sync 2024-04-26T04:53:15.598 INFO:tasks.workunit.client.0.smithi077.stdout:2/747: readlink d9/dc/d1d/d1e/l9e 0 2024-04-26T04:53:15.600 INFO:tasks.workunit.client.0.smithi077.stdout:7/771: rename da/d6b/l97 to da/d13/d22/l102 0 2024-04-26T04:53:15.603 INFO:tasks.workunit.client.0.smithi077.stdout:2/748: unlink d9/dc/d1d/d34/d39/d63/c75 0 2024-04-26T04:53:15.605 INFO:tasks.workunit.client.0.smithi077.stdout:7/772: creat da/d13/d22/d34/f103 x:0 0 0 2024-04-26T04:53:15.605 INFO:tasks.workunit.client.0.smithi077.stdout:7/773: fsync da/d15/d42/f77 0 2024-04-26T04:53:15.609 INFO:tasks.workunit.client.0.smithi077.stdout:7/774: mkdir da/d13/d22/d5d/dfc/d104 0 2024-04-26T04:53:15.611 INFO:tasks.workunit.client.0.smithi077.stdout:2/749: creat d9/dc/d1d/d34/d39/d63/f104 x:0 0 0 2024-04-26T04:53:15.615 INFO:tasks.workunit.client.0.smithi077.stdout:2/750: rmdir d9/d15/d44/d4e 39 2024-04-26T04:53:15.617 INFO:tasks.workunit.client.0.smithi077.stdout:7/775: link da/d15/d42/f84 da/d13/d22/d5d/d9f/da7/de0/f105 0 2024-04-26T04:53:15.618 INFO:tasks.workunit.client.0.smithi077.stdout:3/671: dwrite d0/d7/f70 [0,4194304] 0 2024-04-26T04:53:15.622 INFO:tasks.workunit.client.0.smithi077.stdout:2/751: symlink d9/d15/d91/l105 0 2024-04-26T04:53:15.626 INFO:tasks.workunit.client.0.smithi077.stdout:7/776: creat da/d6b/f106 x:0 0 0 2024-04-26T04:53:15.627 INFO:tasks.workunit.client.0.smithi077.stdout:3/672: creat d0/d7/d9a/fd6 x:0 0 0 2024-04-26T04:53:15.629 INFO:tasks.workunit.client.0.smithi077.stdout:2/752: dwrite d9/dc/d1d/d1e/d77/fc7 [0,4194304] 0 2024-04-26T04:53:15.630 INFO:tasks.workunit.client.0.smithi077.stdout:7/777: mknod da/d15/d42/c107 0 2024-04-26T04:53:15.632 INFO:tasks.workunit.client.0.smithi077.stdout:3/673: symlink d0/d7/d9a/db3/ld7 0 2024-04-26T04:53:15.633 INFO:tasks.workunit.client.0.smithi077.stdout:3/674: chown d0/d9/d32/d29/faf 0 1 2024-04-26T04:53:15.633 INFO:tasks.workunit.client.0.smithi077.stdout:2/753: mkdir d9/d15/d91/d106 0 2024-04-26T04:53:15.634 INFO:tasks.workunit.client.0.smithi077.stdout:3/675: dread - d0/f9d zero size 2024-04-26T04:53:15.635 INFO:tasks.workunit.client.0.smithi077.stdout:7/778: rename da/d15/d1a/d39/f71 to da/d13/d22/d5d/dbd/df2/f108 0 2024-04-26T04:53:15.636 INFO:tasks.workunit.client.0.smithi077.stdout:7/779: write da/ff [3138135,66282] 0 2024-04-26T04:53:15.637 INFO:tasks.workunit.client.0.smithi077.stdout:6/767: sync 2024-04-26T04:53:15.637 INFO:tasks.workunit.client.0.smithi077.stdout:4/730: sync 2024-04-26T04:53:15.637 INFO:tasks.workunit.client.0.smithi077.stdout:8/755: sync 2024-04-26T04:53:15.638 INFO:tasks.workunit.client.0.smithi077.stdout:0/688: sync 2024-04-26T04:53:15.638 INFO:tasks.workunit.client.0.smithi077.stdout:1/686: sync 2024-04-26T04:53:15.638 INFO:tasks.workunit.client.0.smithi077.stdout:3/676: mknod d0/d9/d32/d2a/cd8 0 2024-04-26T04:53:15.640 INFO:tasks.workunit.client.0.smithi077.stdout:2/754: mknod d9/d15/d91/d106/c107 0 2024-04-26T04:53:15.640 INFO:tasks.workunit.client.0.smithi077.stdout:6/768: write d8/d52/fb3 [1846197,47656] 0 2024-04-26T04:53:15.641 INFO:tasks.workunit.client.0.smithi077.stdout:2/755: fsync f5 0 2024-04-26T04:53:15.642 INFO:tasks.workunit.client.0.smithi077.stdout:6/769: fsync d8/d52/d53/d5e/f8c 0 2024-04-26T04:53:15.643 INFO:tasks.workunit.client.0.smithi077.stdout:1/687: mkdir d9/d53/d8b/d93/da8/dd7 0 2024-04-26T04:53:15.643 INFO:tasks.workunit.client.0.smithi077.stdout:4/731: readlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/le6 0 2024-04-26T04:53:15.644 INFO:tasks.workunit.client.0.smithi077.stdout:6/770: fdatasync d8/d2a/d86/f106 0 2024-04-26T04:53:15.649 INFO:tasks.workunit.client.0.smithi077.stdout:8/756: creat d4/df/d22/d2d/d2b/d33/dd8/ffb x:0 0 0 2024-04-26T04:53:15.650 INFO:tasks.workunit.client.0.smithi077.stdout:0/689: mknod d0/d47/d64/d97/cef 0 2024-04-26T04:53:15.656 INFO:tasks.workunit.client.0.smithi077.stdout:1/688: mknod d9/d53/dba/cd8 0 2024-04-26T04:53:15.657 INFO:tasks.workunit.client.0.smithi077.stdout:1/689: write d9/d53/d8b/d93/da8/db8/fc9 [1376257,64813] 0 2024-04-26T04:53:15.658 INFO:tasks.workunit.client.0.smithi077.stdout:6/771: mknod d8/d52/d53/d5e/da6/c112 0 2024-04-26T04:53:15.660 INFO:tasks.workunit.client.0.smithi077.stdout:4/732: creat d4/d17/d18/d43/d51/db4/d15/ff3 x:0 0 0 2024-04-26T04:53:15.661 INFO:tasks.workunit.client.0.smithi077.stdout:4/733: write d4/d17/d7f/f9e [2037639,30769] 0 2024-04-26T04:53:15.663 INFO:tasks.workunit.client.0.smithi077.stdout:2/756: getdents d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0 0 2024-04-26T04:53:15.664 INFO:tasks.workunit.client.0.smithi077.stdout:3/677: rmdir d0/d7/d9a/dcc 0 2024-04-26T04:53:15.666 INFO:tasks.workunit.client.0.smithi077.stdout:3/678: chown d0/d9/d32/d2a/c49 241663 1 2024-04-26T04:53:15.670 INFO:tasks.workunit.client.0.smithi077.stdout:6/772: creat d8/d52/d53/da7/f113 x:0 0 0 2024-04-26T04:53:15.675 INFO:tasks.workunit.client.0.smithi077.stdout:4/734: creat d4/d4e/ff4 x:0 0 0 2024-04-26T04:53:15.678 INFO:tasks.workunit.client.0.smithi077.stdout:8/757: dread d4/df/f16 [0,4194304] 0 2024-04-26T04:53:15.678 INFO:tasks.workunit.client.0.smithi077.stdout:6/773: dwrite d8/d2a/d86/f106 [0,4194304] 0 2024-04-26T04:53:15.681 INFO:tasks.workunit.client.0.smithi077.stdout:2/757: rename d9/d15/d44/f58 to d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f108 0 2024-04-26T04:53:15.682 INFO:tasks.workunit.client.0.smithi077.stdout:3/679: mknod d0/d7/d1f/d81/dc5/cd9 0 2024-04-26T04:53:15.687 INFO:tasks.workunit.client.0.smithi077.stdout:0/690: creat d0/d47/ff0 x:0 0 0 2024-04-26T04:53:15.693 INFO:tasks.workunit.client.0.smithi077.stdout:1/690: dwrite d9/db/d2b/d3f/f4b [0,4194304] 0 2024-04-26T04:53:15.695 INFO:tasks.workunit.client.0.smithi077.stdout:1/691: fsync d9/db/dd/d34/d63/fb1 0 2024-04-26T04:53:15.696 INFO:tasks.workunit.client.0.smithi077.stdout:1/692: write d9/db/f60 [855751,2258] 0 2024-04-26T04:53:15.696 INFO:tasks.workunit.client.0.smithi077.stdout:3/680: truncate d0/d7/d1f/d4d/f61 1076411 0 2024-04-26T04:53:15.699 INFO:tasks.workunit.client.0.smithi077.stdout:0/691: symlink d0/d19/d1e/d2f/d52/lf1 0 2024-04-26T04:53:15.701 INFO:tasks.workunit.client.0.smithi077.stdout:0/692: write d0/d19/d1e/d2f/f44 [1810803,85573] 0 2024-04-26T04:53:15.701 INFO:tasks.workunit.client.0.smithi077.stdout:4/735: symlink d4/d17/d18/d43/d51/db4/d15/d1f/d76/lf5 0 2024-04-26T04:53:15.703 INFO:tasks.workunit.client.0.smithi077.stdout:1/693: dwrite d9/db/dd/d68/f7b [0,4194304] 0 2024-04-26T04:53:15.705 INFO:tasks.workunit.client.0.smithi077.stdout:1/694: write d9/db/faf [632738,46857] 0 2024-04-26T04:53:15.721 INFO:tasks.workunit.client.0.smithi077.stdout:1/695: creat d9/d53/d8b/d93/da8/da7/fd9 x:0 0 0 2024-04-26T04:53:15.722 INFO:tasks.workunit.client.0.smithi077.stdout:6/774: dwrite d8/f48 [0,4194304] 0 2024-04-26T04:53:15.723 INFO:tasks.workunit.client.0.smithi077.stdout:6/775: readlink d8/d52/d53/da7/l100 0 2024-04-26T04:53:15.729 INFO:tasks.workunit.client.0.smithi077.stdout:2/758: getdents d9/dc/d5f/d78 0 2024-04-26T04:53:15.731 INFO:tasks.workunit.client.0.smithi077.stdout:1/696: creat d9/db/dd/d9c/dab/fda x:0 0 0 2024-04-26T04:53:15.734 INFO:tasks.workunit.client.0.smithi077.stdout:2/759: dread d9/dc/d1d/d1e/f9b [0,4194304] 0 2024-04-26T04:53:15.735 INFO:tasks.workunit.client.0.smithi077.stdout:2/760: chown d9/dc/d1d/d34/df8/c3e 267190225 1 2024-04-26T04:53:15.736 INFO:tasks.workunit.client.0.smithi077.stdout:2/761: chown d9/dc/d5f/d8e/db9/fe5 1923881 1 2024-04-26T04:53:15.737 INFO:tasks.workunit.client.0.smithi077.stdout:2/762: chown d9/dc/d5f/d8e/da6/dbd 61611564 1 2024-04-26T04:53:15.739 INFO:tasks.workunit.client.0.smithi077.stdout:1/697: dread d9/db/dd/d34/d4a/f59 [0,4194304] 0 2024-04-26T04:53:15.740 INFO:tasks.workunit.client.0.smithi077.stdout:6/776: mknod d8/d1c/da4/de7/c114 0 2024-04-26T04:53:15.746 INFO:tasks.workunit.client.0.smithi077.stdout:0/693: dwrite d0/d19/f87 [0,4194304] 0 2024-04-26T04:53:15.746 INFO:tasks.workunit.client.0.smithi077.stdout:2/763: rmdir d9/dc/d1d/d34/d39/d63/d79/d5b 39 2024-04-26T04:53:15.747 INFO:tasks.workunit.client.0.smithi077.stdout:0/694: fdatasync d0/d47/ff 0 2024-04-26T04:53:15.748 INFO:tasks.workunit.client.0.smithi077.stdout:4/736: dwrite d4/f27 [0,4194304] 0 2024-04-26T04:53:15.750 INFO:tasks.workunit.client.0.smithi077.stdout:2/764: read - d9/dc/d1d/d34/df8/dd1/fdf zero size 2024-04-26T04:53:15.751 INFO:tasks.workunit.client.0.smithi077.stdout:4/737: stat d4/d17/d18/d43/d51/db4/d15/d1f/dd8 0 2024-04-26T04:53:15.759 INFO:tasks.workunit.client.0.smithi077.stdout:6/777: rename d8/f3f to d8/d1c/da4/d10f/f115 0 2024-04-26T04:53:15.761 INFO:tasks.workunit.client.0.smithi077.stdout:0/695: mknod d0/d47/d68/d9f/cf2 0 2024-04-26T04:53:15.764 INFO:tasks.workunit.client.0.smithi077.stdout:2/765: rmdir d9/dc/d6e 39 2024-04-26T04:53:15.766 INFO:tasks.workunit.client.0.smithi077.stdout:6/778: dwrite d8/fd4 [0,4194304] 0 2024-04-26T04:53:15.767 INFO:tasks.workunit.client.0.smithi077.stdout:6/779: dread - d8/d2a/fec zero size 2024-04-26T04:53:15.770 INFO:tasks.workunit.client.0.smithi077.stdout:4/738: dread d4/f8 [4194304,4194304] 0 2024-04-26T04:53:15.775 INFO:tasks.workunit.client.0.smithi077.stdout:0/696: rename d0/d19/d69/fa6 to d0/d47/dce/ff3 0 2024-04-26T04:53:15.781 INFO:tasks.workunit.client.0.smithi077.stdout:1/698: link d9/d53/d6e/c48 d9/d90/d99/d9a/cdb 0 2024-04-26T04:53:15.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/739: dwrite d4/f8 [8388608,4194304] 0 2024-04-26T04:53:15.790 INFO:tasks.workunit.client.0.smithi077.stdout:2/766: mknod d9/dc/d1d/d34/c109 0 2024-04-26T04:53:15.791 INFO:tasks.workunit.client.0.smithi077.stdout:6/780: creat d8/d52/d53/d5e/da6/de9/f116 x:0 0 0 2024-04-26T04:53:15.797 INFO:tasks.workunit.client.0.smithi077.stdout:1/699: dread d9/f44 [0,4194304] 0 2024-04-26T04:53:15.799 INFO:tasks.workunit.client.0.smithi077.stdout:1/700: fsync d9/d53/d8b/d93/da8/db8/fc9 0 2024-04-26T04:53:15.801 INFO:tasks.workunit.client.0.smithi077.stdout:6/781: creat d8/d1c/da4/d10f/f117 x:0 0 0 2024-04-26T04:53:15.802 INFO:tasks.workunit.client.0.smithi077.stdout:0/697: link d0/d47/dce/fec d0/ff4 0 2024-04-26T04:53:15.804 INFO:tasks.workunit.client.0.smithi077.stdout:4/740: symlink d4/lf6 0 2024-04-26T04:53:15.808 INFO:tasks.workunit.client.0.smithi077.stdout:6/782: creat d8/d1c/f118 x:0 0 0 2024-04-26T04:53:15.810 INFO:tasks.workunit.client.0.smithi077.stdout:4/741: mkdir d4/d17/d18/d43/df7 0 2024-04-26T04:53:15.810 INFO:tasks.workunit.client.0.smithi077.stdout:2/767: write d9/dc/d1d/d34/d39/f68 [1785188,3730] 0 2024-04-26T04:53:15.811 INFO:tasks.workunit.client.0.smithi077.stdout:1/701: mkdir d9/d53/d8b/d93/da8/dd7/ddc 0 2024-04-26T04:53:15.812 INFO:tasks.workunit.client.0.smithi077.stdout:6/783: chown d8/d52/d53/d5e/d6c/feb 245244 1 2024-04-26T04:53:15.813 INFO:tasks.workunit.client.0.smithi077.stdout:4/742: mkdir d4/d4e/dad/df8 0 2024-04-26T04:53:15.814 INFO:tasks.workunit.client.0.smithi077.stdout:6/784: truncate d8/d52/da0/dad/ff3 547711 0 2024-04-26T04:53:15.816 INFO:tasks.workunit.client.0.smithi077.stdout:4/743: mknod d4/d4e/dad/d8b/ddc/cf9 0 2024-04-26T04:53:15.817 INFO:tasks.workunit.client.0.smithi077.stdout:1/702: mknod d9/d53/d6e/d7c/dc0/cdd 0 2024-04-26T04:53:15.818 INFO:tasks.workunit.client.0.smithi077.stdout:2/768: dread d9/dc/d1d/d1e/f5e [0,4194304] 0 2024-04-26T04:53:15.818 INFO:tasks.workunit.client.0.smithi077.stdout:6/785: write d8/d1c/f118 [3385,39990] 0 2024-04-26T04:53:15.820 INFO:tasks.workunit.client.0.smithi077.stdout:1/703: read d9/db/d2b/d3f/f55 [738804,130507] 0 2024-04-26T04:53:15.824 INFO:tasks.workunit.client.0.smithi077.stdout:2/769: truncate d9/dc/d1d/d34/d39/d63/d79/d5b/ffb 637048 0 2024-04-26T04:53:15.825 INFO:tasks.workunit.client.0.smithi077.stdout:4/744: dwrite d4/d17/d18/d43/d63/fb3 [0,4194304] 0 2024-04-26T04:53:15.827 INFO:tasks.workunit.client.0.smithi077.stdout:6/786: link d8/d55/l8d d8/d52/l119 0 2024-04-26T04:53:15.829 INFO:tasks.workunit.client.0.smithi077.stdout:6/787: write d8/d52/d53/d5e/da6/fd0 [861033,81298] 0 2024-04-26T04:53:15.830 INFO:tasks.workunit.client.0.smithi077.stdout:2/770: rename d9/dc/d1d/d1e/fc0 to d9/dc/d5f/d8e/db9/f10a 0 2024-04-26T04:53:15.832 INFO:tasks.workunit.client.0.smithi077.stdout:4/745: creat d4/d17/d18/d43/d51/db4/d15/ffa x:0 0 0 2024-04-26T04:53:15.833 INFO:tasks.workunit.client.0.smithi077.stdout:4/746: fsync d4/d4e/dad/fd7 0 2024-04-26T04:53:15.837 INFO:tasks.workunit.client.0.smithi077.stdout:2/771: write d9/d15/d44/fcc [1883214,103731] 0 2024-04-26T04:53:15.841 INFO:tasks.workunit.client.0.smithi077.stdout:6/788: truncate d8/d52/d53/f107 141476 0 2024-04-26T04:53:15.843 INFO:tasks.workunit.client.0.smithi077.stdout:2/772: dwrite d9/dc/d1d/d34/f84 [0,4194304] 0 2024-04-26T04:53:15.846 INFO:tasks.workunit.client.0.smithi077.stdout:2/773: dread d9/fe [4194304,4194304] 0 2024-04-26T04:53:15.846 INFO:tasks.workunit.client.0.smithi077.stdout:4/747: link d4/d17/d18/d43/d6e/d6c/dae/dc5/cf1 d4/d17/d18/d43/d51/cfb 0 2024-04-26T04:53:15.878 INFO:tasks.workunit.client.0.smithi077.stdout:6/789: creat d8/d2a/df2/f11a x:0 0 0 2024-04-26T04:53:15.878 INFO:tasks.workunit.client.0.smithi077.stdout:1/704: write d9/db/d2b/f36 [1207239,13289] 0 2024-04-26T04:53:15.879 INFO:tasks.workunit.client.0.smithi077.stdout:4/748: stat d4/d17/d18/d43/d51/db4/d15/d1f/d25/ca2 0 2024-04-26T04:53:15.880 INFO:tasks.workunit.client.0.smithi077.stdout:1/705: write d9/d90/fbf [494826,33746] 0 2024-04-26T04:53:15.882 INFO:tasks.workunit.client.0.smithi077.stdout:4/749: stat d4/d17/d18/d43/d51/db4/d15/f50 0 2024-04-26T04:53:15.883 INFO:tasks.workunit.client.0.smithi077.stdout:4/750: dread - d4/d17/d18/d43/d51/db4/d15/ff3 zero size 2024-04-26T04:53:15.888 INFO:tasks.workunit.client.0.smithi077.stdout:4/751: symlink d4/d17/d18/d43/d53/lfc 0 2024-04-26T04:53:15.892 INFO:tasks.workunit.client.0.smithi077.stdout:1/706: getdents d9/d53/d8b/d93 0 2024-04-26T04:53:15.895 INFO:tasks.workunit.client.0.smithi077.stdout:2/774: dwrite d9/f64 [0,4194304] 0 2024-04-26T04:53:15.896 INFO:tasks.workunit.client.0.smithi077.stdout:4/752: dread d4/d17/d18/d43/d51/db4/faf [0,4194304] 0 2024-04-26T04:53:15.900 INFO:tasks.workunit.client.0.smithi077.stdout:6/790: dread d8/d2a/f2e [0,4194304] 0 2024-04-26T04:53:15.900 INFO:tasks.workunit.client.0.smithi077.stdout:1/707: mknod d9/d53/d6e/cde 0 2024-04-26T04:53:15.904 INFO:tasks.workunit.client.0.smithi077.stdout:2/775: rmdir d9/dc/d1d/d34/d39/d63 39 2024-04-26T04:53:15.905 INFO:tasks.workunit.client.0.smithi077.stdout:4/753: creat d4/d4e/dad/ffd x:0 0 0 2024-04-26T04:53:15.907 INFO:tasks.workunit.client.0.smithi077.stdout:6/791: mknod d8/d52/d53/da7/c11b 0 2024-04-26T04:53:15.908 INFO:tasks.workunit.client.0.smithi077.stdout:1/708: mknod d9/d53/dba/cdf 0 2024-04-26T04:53:15.911 INFO:tasks.workunit.client.0.smithi077.stdout:2/776: mkdir d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/d10b 0 2024-04-26T04:53:15.912 INFO:tasks.workunit.client.0.smithi077.stdout:4/754: symlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/lfe 0 2024-04-26T04:53:15.913 INFO:tasks.workunit.client.0.smithi077.stdout:5/719: sync 2024-04-26T04:53:15.915 INFO:tasks.workunit.client.0.smithi077.stdout:4/755: chown d4/d4e/dad/d8b/ddc/dec/led 441460 1 2024-04-26T04:53:15.915 INFO:tasks.workunit.client.0.smithi077.stdout:1/709: dwrite d9/db/dd/d34/d63/fb1 [0,4194304] 0 2024-04-26T04:53:15.916 INFO:tasks.workunit.client.0.smithi077.stdout:2/777: read d9/dc/d1d/d34/fa9 [607655,62061] 0 2024-04-26T04:53:15.916 INFO:tasks.workunit.client.0.smithi077.stdout:5/720: dread - d1/d42/da4/d25/fd7 zero size 2024-04-26T04:53:15.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/756: write d4/d4e/dad/ffd [672900,52583] 0 2024-04-26T04:53:15.921 INFO:tasks.workunit.client.0.smithi077.stdout:1/710: chown d9/db/dd/f49 338 1 2024-04-26T04:53:15.927 INFO:tasks.workunit.client.0.smithi077.stdout:2/778: symlink d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/l10c 0 2024-04-26T04:53:15.928 INFO:tasks.workunit.client.0.smithi077.stdout:4/757: mknod d4/d17/d18/d43/d6e/d6c/cff 0 2024-04-26T04:53:15.937 INFO:tasks.workunit.client.0.smithi077.stdout:6/792: dwrite d8/d2a/f63 [0,4194304] 0 2024-04-26T04:53:15.938 INFO:tasks.workunit.client.0.smithi077.stdout:1/711: write d9/db/dd/f5b [1808379,82080] 0 2024-04-26T04:53:15.938 INFO:tasks.workunit.client.0.smithi077.stdout:2/779: chown d9/d15/d44/d4e/lcb 17 1 2024-04-26T04:53:15.940 INFO:tasks.workunit.client.0.smithi077.stdout:5/721: dwrite d1/d8/f49 [0,4194304] 0 2024-04-26T04:53:15.943 INFO:tasks.workunit.client.0.smithi077.stdout:6/793: chown d8/d2a/d86/c96 1896216 1 2024-04-26T04:53:15.947 INFO:tasks.workunit.client.0.smithi077.stdout:6/794: mkdir d8/d52/d11c 0 2024-04-26T04:53:15.950 INFO:tasks.workunit.client.0.smithi077.stdout:5/722: mkdir d1/d42/d62/dc8/dd5/df8 0 2024-04-26T04:53:15.952 INFO:tasks.workunit.client.0.smithi077.stdout:1/712: mkdir d9/d90/de0 0 2024-04-26T04:53:15.954 INFO:tasks.workunit.client.0.smithi077.stdout:4/758: write d4/d4e/fd4 [4016694,67271] 0 2024-04-26T04:53:15.956 INFO:tasks.workunit.client.0.smithi077.stdout:1/713: readlink d9/d53/d6e/d7c/lb0 0 2024-04-26T04:53:15.963 INFO:tasks.workunit.client.0.smithi077.stdout:4/759: dwrite d4/d4e/dad/d8b/ddc/fe9 [0,4194304] 0 2024-04-26T04:53:15.964 INFO:tasks.workunit.client.0.smithi077.stdout:1/714: rename d9/db/dd/f96 to d9/d53/d6e/d7c/fe1 0 2024-04-26T04:53:15.969 INFO:tasks.workunit.client.0.smithi077.stdout:1/715: mknod d9/d53/d8b/d93/da8/dd7/ce2 0 2024-04-26T04:53:15.970 INFO:tasks.workunit.client.0.smithi077.stdout:2/780: dread d9/dc/f7a [0,4194304] 0 2024-04-26T04:53:15.971 INFO:tasks.workunit.client.0.smithi077.stdout:2/781: dread - d9/dc/d1d/d34/df8/dd1/fdf zero size 2024-04-26T04:53:15.973 INFO:tasks.workunit.client.0.smithi077.stdout:1/716: rename d9/db/d2b/d3f/f41 to d9/db/dd/d68/fe3 0 2024-04-26T04:53:15.975 INFO:tasks.workunit.client.0.smithi077.stdout:2/782: unlink d9/d15/d91/l105 0 2024-04-26T04:53:15.977 INFO:tasks.workunit.client.0.smithi077.stdout:1/717: unlink d9/d53/dba/fca 0 2024-04-26T04:53:15.977 INFO:tasks.workunit.client.0.smithi077.stdout:1/718: chown d9/d53/dba/cd8 759927 1 2024-04-26T04:53:15.979 INFO:tasks.workunit.client.0.smithi077.stdout:1/719: write d9/db/dd/f69 [4026745,21214] 0 2024-04-26T04:53:15.985 INFO:tasks.workunit.client.0.smithi077.stdout:5/723: getdents d1/d42/d62/dc8/dd5 0 2024-04-26T04:53:15.991 INFO:tasks.workunit.client.0.smithi077.stdout:5/724: dwrite d1/d8/f49 [4194304,4194304] 0 2024-04-26T04:53:15.996 INFO:tasks.workunit.client.0.smithi077.stdout:5/725: mkdir d1/d8c/df9 0 2024-04-26T04:53:15.997 INFO:tasks.workunit.client.0.smithi077.stdout:5/726: dread - d1/d12/fed zero size 2024-04-26T04:53:15.997 INFO:tasks.workunit.client.0.smithi077.stdout:4/760: dwrite d4/d17/d18/d43/d6e/f84 [0,4194304] 0 2024-04-26T04:53:15.999 INFO:tasks.workunit.client.0.smithi077.stdout:5/727: chown d1/d42/da4/d25/d31/l82 18 1 2024-04-26T04:53:16.002 INFO:tasks.workunit.client.0.smithi077.stdout:5/728: fdatasync d1/d12/d45/f63 0 2024-04-26T04:53:16.005 INFO:tasks.workunit.client.0.smithi077.stdout:5/729: chown d1/d42/da4/d25/d31/f92 0 1 2024-04-26T04:53:16.008 INFO:tasks.workunit.client.0.smithi077.stdout:4/761: dwrite d4/d17/d18/d43/d63/d80/db6/fee [0,4194304] 0 2024-04-26T04:53:16.009 INFO:tasks.workunit.client.0.smithi077.stdout:1/720: write d9/f43 [5241396,10526] 0 2024-04-26T04:53:16.012 INFO:tasks.workunit.client.0.smithi077.stdout:4/762: fsync d4/d17/d18/d43/d6e/f86 0 2024-04-26T04:53:16.018 INFO:tasks.workunit.client.0.smithi077.stdout:5/730: creat d1/d8d/da7/ffa x:0 0 0 2024-04-26T04:53:16.019 INFO:tasks.workunit.client.0.smithi077.stdout:2/783: dwrite d9/dc/d1d/d34/f36 [0,4194304] 0 2024-04-26T04:53:16.020 INFO:tasks.workunit.client.0.smithi077.stdout:5/731: chown d1/d42/da4/l73 4 1 2024-04-26T04:53:16.057 INFO:tasks.workunit.client.0.smithi077.stdout:1/721: rename d9/db/dd/d34/db2 to d9/d90/de0/de4 0 2024-04-26T04:53:16.061 INFO:tasks.workunit.client.0.smithi077.stdout:4/763: symlink d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/l100 0 2024-04-26T04:53:16.065 INFO:tasks.workunit.client.0.smithi077.stdout:4/764: creat d4/d17/d18/d43/f101 x:0 0 0 2024-04-26T04:53:16.067 INFO:tasks.workunit.client.0.smithi077.stdout:4/765: creat d4/d4e/dad/d8b/f102 x:0 0 0 2024-04-26T04:53:16.068 INFO:tasks.workunit.client.0.smithi077.stdout:4/766: fsync d4/d17/d7f/feb 0 2024-04-26T04:53:16.068 INFO:tasks.workunit.client.0.smithi077.stdout:1/722: creat d9/fe5 x:0 0 0 2024-04-26T04:53:16.095 INFO:tasks.workunit.client.0.smithi077.stdout:1/723: fsync d9/db/dd/f11 0 2024-04-26T04:53:16.098 INFO:tasks.workunit.client.0.smithi077.stdout:1/724: stat d9/d90/fad 0 2024-04-26T04:53:16.100 INFO:tasks.workunit.client.0.smithi077.stdout:2/784: dwrite d9/d15/d44/d4e/fb7 [0,4194304] 0 2024-04-26T04:53:16.100 INFO:tasks.workunit.client.0.smithi077.stdout:5/732: dwrite d1/d30/d33/d66/fae [0,4194304] 0 2024-04-26T04:53:16.186 INFO:tasks.workunit.client.0.smithi077.stdout:4/767: write d4/d17/d18/d43/d51/db4/d15/d1f/d35/fc1 [602814,48727] 0 2024-04-26T04:53:16.188 INFO:tasks.workunit.client.0.smithi077.stdout:5/733: rename d1/d30/d35/f57 to d1/d42/da4/dd3/ffb 0 2024-04-26T04:53:16.190 INFO:tasks.workunit.client.0.smithi077.stdout:5/734: write d1/d42/da4/d25/d31/f71 [3125673,118093] 0 2024-04-26T04:53:16.191 INFO:tasks.workunit.client.0.smithi077.stdout:9/646: sync 2024-04-26T04:53:16.192 INFO:tasks.workunit.client.0.smithi077.stdout:4/768: dwrite d4/d17/d18/d43/d53/f79 [0,4194304] 0 2024-04-26T04:53:16.203 INFO:tasks.workunit.client.0.smithi077.stdout:2/785: rename d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/ced to d9/dc/d1d/d34/d39/d63/de9/c10d 0 2024-04-26T04:53:16.205 INFO:tasks.workunit.client.0.smithi077.stdout:4/769: mknod d4/d17/c103 0 2024-04-26T04:53:16.206 INFO:tasks.workunit.client.0.smithi077.stdout:1/725: creat d9/d90/fe6 x:0 0 0 2024-04-26T04:53:16.207 INFO:tasks.workunit.client.0.smithi077.stdout:9/647: dread d9/dd/d1b/d2b/f83 [0,4194304] 0 2024-04-26T04:53:16.209 INFO:tasks.workunit.client.0.smithi077.stdout:5/735: creat d1/d12/d28/ffc x:0 0 0 2024-04-26T04:53:16.211 INFO:tasks.workunit.client.0.smithi077.stdout:5/736: fsync d1/d8/d7e/fc6 0 2024-04-26T04:53:16.215 INFO:tasks.workunit.client.0.smithi077.stdout:1/726: creat d9/db/dd/d34/fe7 x:0 0 0 2024-04-26T04:53:16.216 INFO:tasks.workunit.client.0.smithi077.stdout:9/648: dwrite d9/d35/d99/d9f/fe2 [0,4194304] 0 2024-04-26T04:53:16.218 INFO:tasks.workunit.client.0.smithi077.stdout:9/649: dread - d9/dd/d1b/d2b/d32/fce zero size 2024-04-26T04:53:16.222 INFO:tasks.workunit.client.0.smithi077.stdout:5/737: dwrite d1/d42/d62/d6b/f85 [0,4194304] 0 2024-04-26T04:53:16.223 INFO:tasks.workunit.client.0.smithi077.stdout:2/786: symlink d9/dc/d1d/d34/dd4/l10e 0 2024-04-26T04:53:16.224 INFO:tasks.workunit.client.0.smithi077.stdout:5/738: fsync d1/d30/d33/d66/fae 0 2024-04-26T04:53:16.227 INFO:tasks.workunit.client.0.smithi077.stdout:4/770: dread d4/f9c [0,4194304] 0 2024-04-26T04:53:16.231 INFO:tasks.workunit.client.0.smithi077.stdout:2/787: dwrite d9/d15/d44/fcc [0,4194304] 0 2024-04-26T04:53:16.234 INFO:tasks.workunit.client.0.smithi077.stdout:5/739: fsync d1/d42/d62/dc8/fdc 0 2024-04-26T04:53:16.235 INFO:tasks.workunit.client.0.smithi077.stdout:4/771: symlink d4/d17/l104 0 2024-04-26T04:53:16.241 INFO:tasks.workunit.client.0.smithi077.stdout:2/788: creat d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f10f x:0 0 0 2024-04-26T04:53:16.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:15 smithi077 ceph-mon[26311]: pgmap v27: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 91 MiB/s rd, 229 MiB/s wr, 454 op/s 2024-04-26T04:53:16.250 INFO:tasks.workunit.client.0.smithi077.stdout:2/789: dread - d9/dc/d1d/d34/d39/fa3 zero size 2024-04-26T04:53:16.253 INFO:tasks.workunit.client.0.smithi077.stdout:9/650: dwrite d9/f26 [0,4194304] 0 2024-04-26T04:53:16.253 INFO:tasks.workunit.client.0.smithi077.stdout:2/790: creat d9/dc/d1d/d1e/d77/df9/f110 x:0 0 0 2024-04-26T04:53:16.256 INFO:tasks.workunit.client.0.smithi077.stdout:1/727: write d9/db/f4c [377645,11752] 0 2024-04-26T04:53:16.260 INFO:tasks.workunit.client.0.smithi077.stdout:5/740: write d1/d8/f10 [6012200,28861] 0 2024-04-26T04:53:16.262 INFO:tasks.workunit.client.0.smithi077.stdout:1/728: mkdir d9/db/dd/d9c/dab/de8 0 2024-04-26T04:53:16.264 INFO:tasks.workunit.client.0.smithi077.stdout:2/791: creat d9/dc/f111 x:0 0 0 2024-04-26T04:53:16.264 INFO:tasks.workunit.client.0.smithi077.stdout:4/772: dread d4/d17/d18/d43/d51/db4/d15/d1f/f32 [0,4194304] 0 2024-04-26T04:53:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:15 smithi078 ceph-mon[33746]: pgmap v27: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 91 MiB/s rd, 229 MiB/s wr, 454 op/s 2024-04-26T04:53:16.268 INFO:tasks.workunit.client.0.smithi077.stdout:9/651: rename d9/d3a/d79/d95/dad/fd5 to d9/fe5 0 2024-04-26T04:53:16.270 INFO:tasks.workunit.client.0.smithi077.stdout:5/741: mkdir d1/d42/d62/dc8/dd5/df8/dfd 0 2024-04-26T04:53:16.271 INFO:tasks.workunit.client.0.smithi077.stdout:1/729: creat d9/d90/d99/d9a/dd6/fe9 x:0 0 0 2024-04-26T04:53:16.272 INFO:tasks.workunit.client.0.smithi077.stdout:5/742: write d1/d12/d28/d38/f8b [2700680,74307] 0 2024-04-26T04:53:16.273 INFO:tasks.workunit.client.0.smithi077.stdout:4/773: truncate d4/d17/d18/d43/d51/db4/d15/f40 4837337 0 2024-04-26T04:53:16.273 INFO:tasks.workunit.client.0.smithi077.stdout:1/730: write d9/db/fd1 [86413,119581] 0 2024-04-26T04:53:16.274 INFO:tasks.workunit.client.0.smithi077.stdout:9/652: truncate d9/dd/d1b/d2b/d32/d36/fa7 2253267 0 2024-04-26T04:53:16.277 INFO:tasks.workunit.client.0.smithi077.stdout:5/743: chown d1/d42/da4/dd3/ffb 580405 1 2024-04-26T04:53:16.277 INFO:tasks.workunit.client.0.smithi077.stdout:9/653: dread - d9/dd/d3d/f59 zero size 2024-04-26T04:53:16.277 INFO:tasks.workunit.client.0.smithi077.stdout:5/744: readlink d1/d42/da4/l73 0 2024-04-26T04:53:16.278 INFO:tasks.workunit.client.0.smithi077.stdout:9/654: chown d9/d35/f4c 587724610 1 2024-04-26T04:53:16.279 INFO:tasks.workunit.client.0.smithi077.stdout:4/774: mknod d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/c105 0 2024-04-26T04:53:16.280 INFO:tasks.workunit.client.0.smithi077.stdout:1/731: truncate d9/db/dd/d34/d4a/f59 623890 0 2024-04-26T04:53:16.283 INFO:tasks.workunit.client.0.smithi077.stdout:5/745: symlink d1/d8/d72/lfe 0 2024-04-26T04:53:16.292 INFO:tasks.workunit.client.0.smithi077.stdout:2/792: truncate d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/fdd 351453 0 2024-04-26T04:53:16.299 INFO:tasks.workunit.client.0.smithi077.stdout:7/780: sync 2024-04-26T04:53:16.304 INFO:tasks.workunit.client.0.smithi077.stdout:5/746: creat d1/d12/d28/fff x:0 0 0 2024-04-26T04:53:16.306 INFO:tasks.workunit.client.0.smithi077.stdout:2/793: dread d9/dc/d1d/d34/d39/d63/d79/f3f [0,4194304] 0 2024-04-26T04:53:16.308 INFO:tasks.workunit.client.0.smithi077.stdout:9/655: getdents d9/d35/d99 0 2024-04-26T04:53:16.316 INFO:tasks.workunit.client.0.smithi077.stdout:4/775: dwrite d4/d17/d18/d43/f74 [0,4194304] 0 2024-04-26T04:53:16.316 INFO:tasks.workunit.client.0.smithi077.stdout:7/781: truncate da/d13/d22/d7d/d80/f8b 820064 0 2024-04-26T04:53:16.318 INFO:tasks.workunit.client.0.smithi077.stdout:1/732: dwrite d9/db/dd/f74 [0,4194304] 0 2024-04-26T04:53:16.319 INFO:tasks.workunit.client.0.smithi077.stdout:7/782: dread - da/d6b/d63/ff5 zero size 2024-04-26T04:53:16.320 INFO:tasks.workunit.client.0.smithi077.stdout:9/656: mknod d9/dd/d1b/d2b/d32/d36/d7f/dae/ce6 0 2024-04-26T04:53:16.321 INFO:tasks.workunit.client.0.smithi077.stdout:1/733: dread - d9/d90/d99/d9a/dd6/fe9 zero size 2024-04-26T04:53:16.326 INFO:tasks.workunit.client.0.smithi077.stdout:7/783: creat da/d15/d42/d8c/f109 x:0 0 0 2024-04-26T04:53:16.330 INFO:tasks.workunit.client.0.smithi077.stdout:1/734: mknod d9/d90/d99/d9a/dd6/cea 0 2024-04-26T04:53:16.331 INFO:tasks.workunit.client.0.smithi077.stdout:1/735: dread - d9/db/d2b/d75/f97 zero size 2024-04-26T04:53:16.336 INFO:tasks.workunit.client.0.smithi077.stdout:4/776: truncate d4/d4e/f8d 1433404 0 2024-04-26T04:53:16.336 INFO:tasks.workunit.client.0.smithi077.stdout:5/747: dread d1/f5 [0,4194304] 0 2024-04-26T04:53:16.338 INFO:tasks.workunit.client.0.smithi077.stdout:2/794: dwrite d9/dc/d5f/f62 [0,4194304] 0 2024-04-26T04:53:16.339 INFO:tasks.workunit.client.0.smithi077.stdout:1/736: symlink d9/d53/d8b/d93/da8/da7/leb 0 2024-04-26T04:53:16.340 INFO:tasks.workunit.client.0.smithi077.stdout:7/784: unlink da/d6b/d63/c86 0 2024-04-26T04:53:16.350 INFO:tasks.workunit.client.0.smithi077.stdout:1/737: dwrite d9/db/fd1 [0,4194304] 0 2024-04-26T04:53:16.352 INFO:tasks.workunit.client.0.smithi077.stdout:5/748: symlink d1/d42/da4/d25/d31/d3c/dec/l100 0 2024-04-26T04:53:16.352 INFO:tasks.workunit.client.0.smithi077.stdout:2/795: dwrite d9/dc/d1d/d34/d39/d63/d79/d5b/ffb [0,4194304] 0 2024-04-26T04:53:16.354 INFO:tasks.workunit.client.0.smithi077.stdout:1/738: write d9/fe5 [355054,49583] 0 2024-04-26T04:53:16.356 INFO:tasks.workunit.client.0.smithi077.stdout:0/698: sync 2024-04-26T04:53:16.356 INFO:tasks.workunit.client.0.smithi077.stdout:8/758: sync 2024-04-26T04:53:16.356 INFO:tasks.workunit.client.0.smithi077.stdout:3/681: sync 2024-04-26T04:53:16.361 INFO:tasks.workunit.client.0.smithi077.stdout:9/657: dwrite d9/f8d [0,4194304] 0 2024-04-26T04:53:16.367 INFO:tasks.workunit.client.0.smithi077.stdout:4/777: dread d4/d17/d18/d43/d51/db4/d15/d1f/d76/f82 [0,4194304] 0 2024-04-26T04:53:16.374 INFO:tasks.workunit.client.0.smithi077.stdout:0/699: chown d0/c6c 4249 1 2024-04-26T04:53:16.374 INFO:tasks.workunit.client.0.smithi077.stdout:4/778: dwrite d4/d17/d18/d43/d51/db4/d15/f62 [4194304,4194304] 0 2024-04-26T04:53:16.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/700: fdatasync d0/d47/f8c 0 2024-04-26T04:53:16.377 INFO:tasks.workunit.client.0.smithi077.stdout:3/682: creat d0/d7/d1f/d81/fda x:0 0 0 2024-04-26T04:53:16.379 INFO:tasks.workunit.client.0.smithi077.stdout:8/759: symlink d4/d17/d90/lfc 0 2024-04-26T04:53:16.382 INFO:tasks.workunit.client.0.smithi077.stdout:7/785: link da/d6b/d63/fef da/d13/d22/d5d/dfc/f10a 0 2024-04-26T04:53:16.383 INFO:tasks.workunit.client.0.smithi077.stdout:4/779: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/fe7 [4194304,4194304] 0 2024-04-26T04:53:16.386 INFO:tasks.workunit.client.0.smithi077.stdout:2/796: mkdir d9/dc/d5f/d8e/da6/dbd/d112 0 2024-04-26T04:53:16.391 INFO:tasks.workunit.client.0.smithi077.stdout:0/701: fdatasync d0/d19/d1e/d59/f6e 0 2024-04-26T04:53:16.392 INFO:tasks.workunit.client.0.smithi077.stdout:0/702: chown d0/d2c/d4e/f95 8 1 2024-04-26T04:53:16.393 INFO:tasks.workunit.client.0.smithi077.stdout:0/703: write d0/d47/f31 [2950942,54827] 0 2024-04-26T04:53:16.397 INFO:tasks.workunit.client.0.smithi077.stdout:7/786: stat da/d15/f32 0 2024-04-26T04:53:16.397 INFO:tasks.workunit.client.0.smithi077.stdout:6/795: sync 2024-04-26T04:53:16.399 INFO:tasks.workunit.client.0.smithi077.stdout:1/739: creat d9/fec x:0 0 0 2024-04-26T04:53:16.401 INFO:tasks.workunit.client.0.smithi077.stdout:1/740: chown d9/db/dd/d34/d63 227267536 1 2024-04-26T04:53:16.402 INFO:tasks.workunit.client.0.smithi077.stdout:2/797: creat d9/dc/d1d/d34/d39/d63/d79/d5b/db8/f113 x:0 0 0 2024-04-26T04:53:16.407 INFO:tasks.workunit.client.0.smithi077.stdout:0/704: mknod d0/d19/d1e/d2f/cf5 0 2024-04-26T04:53:16.408 INFO:tasks.workunit.client.0.smithi077.stdout:0/705: readlink d0/d47/le4 0 2024-04-26T04:53:16.410 INFO:tasks.workunit.client.0.smithi077.stdout:6/796: fdatasync d8/fa1 0 2024-04-26T04:53:16.411 INFO:tasks.workunit.client.0.smithi077.stdout:7/787: mknod da/d13/d22/d5d/dfc/c10b 0 2024-04-26T04:53:16.413 INFO:tasks.workunit.client.0.smithi077.stdout:5/749: getdents d1/d12/d87 0 2024-04-26T04:53:16.414 INFO:tasks.workunit.client.0.smithi077.stdout:5/750: stat d1/d42/da4/c23 0 2024-04-26T04:53:16.415 INFO:tasks.workunit.client.0.smithi077.stdout:1/741: rename d9/f20 to d9/d53/d8b/d93/da8/db8/fed 0 2024-04-26T04:53:16.417 INFO:tasks.workunit.client.0.smithi077.stdout:9/658: getdents d9/d45/db1 0 2024-04-26T04:53:16.417 INFO:tasks.workunit.client.0.smithi077.stdout:6/797: dwrite d8/ff [8388608,4194304] 0 2024-04-26T04:53:16.423 INFO:tasks.workunit.client.0.smithi077.stdout:0/706: read d0/fe [2797414,4955] 0 2024-04-26T04:53:16.427 INFO:tasks.workunit.client.0.smithi077.stdout:5/751: unlink d1/d42/da4/d25/d31/d3c/f8e 0 2024-04-26T04:53:16.433 INFO:tasks.workunit.client.0.smithi077.stdout:2/798: rename d9/d15/fb4 to d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f114 0 2024-04-26T04:53:16.435 INFO:tasks.workunit.client.0.smithi077.stdout:1/742: creat d9/d90/d99/d9a/dd6/fee x:0 0 0 2024-04-26T04:53:16.445 INFO:tasks.workunit.client.0.smithi077.stdout:4/780: getdents d4/d17/d18/d43/d51/db4/d15/d1f/dd8 0 2024-04-26T04:53:16.450 INFO:tasks.workunit.client.0.smithi077.stdout:4/781: fsync d4/d17/d18/d43/d51/db4/d15/ff3 0 2024-04-26T04:53:16.456 INFO:tasks.workunit.client.0.smithi077.stdout:6/798: dwrite d8/d2a/f32 [4194304,4194304] 0 2024-04-26T04:53:16.456 INFO:tasks.workunit.client.0.smithi077.stdout:4/782: write d4/d17/f57 [2631648,58194] 0 2024-04-26T04:53:16.457 INFO:tasks.workunit.client.0.smithi077.stdout:2/799: mknod d9/d15/c115 0 2024-04-26T04:53:16.459 INFO:tasks.workunit.client.0.smithi077.stdout:6/799: chown d8/d52/d53/c71 404534 1 2024-04-26T04:53:16.462 INFO:tasks.workunit.client.0.smithi077.stdout:0/707: dread d0/d2c/d4e/f4c [0,4194304] 0 2024-04-26T04:53:16.462 INFO:tasks.workunit.client.0.smithi077.stdout:1/743: mkdir d9/db/dd/d9c/dab/de8/def 0 2024-04-26T04:53:16.462 INFO:tasks.workunit.client.0.smithi077.stdout:8/760: dwrite d4/df/d22/fc8 [0,4194304] 0 2024-04-26T04:53:16.463 INFO:tasks.workunit.client.0.smithi077.stdout:3/683: write d0/d9/d32/d29/f4b [1032906,59192] 0 2024-04-26T04:53:16.466 INFO:tasks.workunit.client.0.smithi077.stdout:5/752: link d1/d8c/l93 d1/d42/dac/l101 0 2024-04-26T04:53:16.468 INFO:tasks.workunit.client.0.smithi077.stdout:6/800: fdatasync d8/d52/da0/f9a 0 2024-04-26T04:53:16.469 INFO:tasks.workunit.client.0.smithi077.stdout:5/753: stat d1/d12/d45/cd6 0 2024-04-26T04:53:16.471 INFO:tasks.workunit.client.0.smithi077.stdout:8/761: fdatasync d4/d17/f3c 0 2024-04-26T04:53:16.473 INFO:tasks.workunit.client.0.smithi077.stdout:0/708: creat d0/d47/dce/ff6 x:0 0 0 2024-04-26T04:53:16.474 INFO:tasks.workunit.client.0.smithi077.stdout:1/744: dwrite d9/db/d2b/f36 [0,4194304] 0 2024-04-26T04:53:16.477 INFO:tasks.workunit.client.0.smithi077.stdout:5/754: dread d1/d30/d33/fe4 [0,4194304] 0 2024-04-26T04:53:16.478 INFO:tasks.workunit.client.0.smithi077.stdout:9/659: dread d9/f1c [4194304,4194304] 0 2024-04-26T04:53:16.479 INFO:tasks.workunit.client.0.smithi077.stdout:6/801: rmdir d8/d2a/ddd 39 2024-04-26T04:53:16.482 INFO:tasks.workunit.client.0.smithi077.stdout:6/802: write d8/d52/f111 [327908,89152] 0 2024-04-26T04:53:16.488 INFO:tasks.workunit.client.0.smithi077.stdout:0/709: dwrite d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:16.492 INFO:tasks.workunit.client.0.smithi077.stdout:5/755: creat d1/d42/d62/dc8/dd5/f102 x:0 0 0 2024-04-26T04:53:16.496 INFO:tasks.workunit.client.0.smithi077.stdout:7/788: dwrite da/d6b/fb3 [0,4194304] 0 2024-04-26T04:53:16.501 INFO:tasks.workunit.client.0.smithi077.stdout:0/710: mkdir d0/d19/d8d/df7 0 2024-04-26T04:53:16.501 INFO:tasks.workunit.client.0.smithi077.stdout:3/684: getdents d0/d7/d9a/db3 0 2024-04-26T04:53:16.502 INFO:tasks.workunit.client.0.smithi077.stdout:3/685: write d0/d7/fad [683509,32736] 0 2024-04-26T04:53:16.504 INFO:tasks.workunit.client.0.smithi077.stdout:3/686: fdatasync d0/d7/d1f/d4d/f9b 0 2024-04-26T04:53:16.506 INFO:tasks.workunit.client.0.smithi077.stdout:5/756: mknod d1/d42/d62/dc8/dd5/df8/c103 0 2024-04-26T04:53:16.515 INFO:tasks.workunit.client.0.smithi077.stdout:2/800: truncate d9/dc/d5f/ffe 347591 0 2024-04-26T04:53:16.516 INFO:tasks.workunit.client.0.smithi077.stdout:9/660: dread d9/dd/d1b/f23 [0,4194304] 0 2024-04-26T04:53:16.518 INFO:tasks.workunit.client.0.smithi077.stdout:1/745: getdents d9/d53/d8b/d93/da8 0 2024-04-26T04:53:16.521 INFO:tasks.workunit.client.0.smithi077.stdout:5/757: creat d1/d42/da4/d25/f104 x:0 0 0 2024-04-26T04:53:16.522 INFO:tasks.workunit.client.0.smithi077.stdout:6/803: write d8/d52/f70 [569095,120914] 0 2024-04-26T04:53:16.525 INFO:tasks.workunit.client.0.smithi077.stdout:8/762: truncate d4/df/d22/d2d/d2b/d33/f92 3468590 0 2024-04-26T04:53:16.526 INFO:tasks.workunit.client.0.smithi077.stdout:2/801: symlink d9/dc/d1d/d1e/d77/df9/l116 0 2024-04-26T04:53:16.529 INFO:tasks.workunit.client.0.smithi077.stdout:9/661: truncate d9/dd/d1b/fab 1059925 0 2024-04-26T04:53:16.531 INFO:tasks.workunit.client.0.smithi077.stdout:3/687: mkdir d0/d7/d9a/db7/ddb 0 2024-04-26T04:53:16.533 INFO:tasks.workunit.client.0.smithi077.stdout:1/746: symlink d9/db/dd/d9c/dab/lf0 0 2024-04-26T04:53:16.534 INFO:tasks.workunit.client.0.smithi077.stdout:0/711: write d0/ff4 [970131,65060] 0 2024-04-26T04:53:16.535 INFO:tasks.workunit.client.0.smithi077.stdout:7/789: dwrite da/d13/d22/d5d/f66 [0,4194304] 0 2024-04-26T04:53:16.537 INFO:tasks.workunit.client.0.smithi077.stdout:6/804: chown d8/d2a/ddd/cfe 479381471 1 2024-04-26T04:53:16.538 INFO:tasks.workunit.client.0.smithi077.stdout:0/712: read d0/d47/fc8 [320107,89612] 0 2024-04-26T04:53:16.538 INFO:tasks.workunit.client.0.smithi077.stdout:8/763: unlink d4/df/d22/d2d/d2b/l6d 0 2024-04-26T04:53:16.540 INFO:tasks.workunit.client.0.smithi077.stdout:7/790: dread - da/d13/d22/d5d/d9f/fdf zero size 2024-04-26T04:53:16.542 INFO:tasks.workunit.client.0.smithi077.stdout:5/758: dwrite d1/d42/dac/ff0 [0,4194304] 0 2024-04-26T04:53:16.544 INFO:tasks.workunit.client.0.smithi077.stdout:5/759: write d1/d30/d35/ff7 [463124,99452] 0 2024-04-26T04:53:16.546 INFO:tasks.workunit.client.0.smithi077.stdout:5/760: chown d1/d30/d33/d66 9 1 2024-04-26T04:53:16.546 INFO:tasks.workunit.client.0.smithi077.stdout:5/761: write d1/d8c/ff4 [567454,28712] 0 2024-04-26T04:53:16.549 INFO:tasks.workunit.client.0.smithi077.stdout:1/747: stat d9/d53/dc1/cce 0 2024-04-26T04:53:16.549 INFO:tasks.workunit.client.0.smithi077.stdout:2/802: rmdir d9/dc/d1d/d1e/d77 39 2024-04-26T04:53:16.551 INFO:tasks.workunit.client.0.smithi077.stdout:0/713: creat d0/d47/d68/d9f/ff8 x:0 0 0 2024-04-26T04:53:16.555 INFO:tasks.workunit.client.0.smithi077.stdout:8/764: unlink d4/df/d22/d75/c93 0 2024-04-26T04:53:16.555 INFO:tasks.workunit.client.0.smithi077.stdout:8/765: chown d4/df/dc6 626 1 2024-04-26T04:53:16.558 INFO:tasks.workunit.client.0.smithi077.stdout:6/805: mkdir d8/d1c/da4/d10f/d11d 0 2024-04-26T04:53:16.560 INFO:tasks.workunit.client.0.smithi077.stdout:9/662: unlink d9/d7d/d94/l96 0 2024-04-26T04:53:16.563 INFO:tasks.workunit.client.0.smithi077.stdout:3/688: creat d0/d7/d1f/d4d/dbe/fdc x:0 0 0 2024-04-26T04:53:16.572 INFO:tasks.workunit.client.0.smithi077.stdout:9/663: rmdir d9/d35/d99 39 2024-04-26T04:53:16.573 INFO:tasks.workunit.client.0.smithi077.stdout:3/689: rmdir d0/d7/d1f 39 2024-04-26T04:53:16.574 INFO:tasks.workunit.client.0.smithi077.stdout:7/791: dwrite da/d15/f94 [0,4194304] 0 2024-04-26T04:53:16.574 INFO:tasks.workunit.client.0.smithi077.stdout:5/762: rename d1/c2c to d1/d8f/c105 0 2024-04-26T04:53:16.577 INFO:tasks.workunit.client.0.smithi077.stdout:1/748: mkdir d9/df1 0 2024-04-26T04:53:16.578 INFO:tasks.workunit.client.0.smithi077.stdout:8/766: getdents d4/df/d6f/da3/dc7/df2 0 2024-04-26T04:53:16.580 INFO:tasks.workunit.client.0.smithi077.stdout:8/767: read d4/df/d22/d80/ff3 [341263,104943] 0 2024-04-26T04:53:16.580 INFO:tasks.workunit.client.0.smithi077.stdout:6/806: creat d8/d2a/d10b/f11e x:0 0 0 2024-04-26T04:53:16.581 INFO:tasks.workunit.client.0.smithi077.stdout:8/768: stat d4/df/d6f/da3/fb1 0 2024-04-26T04:53:16.583 INFO:tasks.workunit.client.0.smithi077.stdout:9/664: mkdir d9/d3a/d79/d95/dad/de7 0 2024-04-26T04:53:16.584 INFO:tasks.workunit.client.0.smithi077.stdout:9/665: stat d9/dd/d1b/cc6 0 2024-04-26T04:53:16.585 INFO:tasks.workunit.client.0.smithi077.stdout:7/792: fdatasync da/d15/f2f 0 2024-04-26T04:53:16.588 INFO:tasks.workunit.client.0.smithi077.stdout:5/763: fsync d1/d42/f69 0 2024-04-26T04:53:16.589 INFO:tasks.workunit.client.0.smithi077.stdout:2/803: creat d9/dc/d1d/d34/d39/d63/d79/f117 x:0 0 0 2024-04-26T04:53:16.590 INFO:tasks.workunit.client.0.smithi077.stdout:8/769: dwrite d4/df/d18/d56/f72 [0,4194304] 0 2024-04-26T04:53:16.591 INFO:tasks.workunit.client.0.smithi077.stdout:6/807: symlink d8/d52/d53/da7/df4/l11f 0 2024-04-26T04:53:16.591 INFO:tasks.workunit.client.0.smithi077.stdout:5/764: fdatasync d1/d42/da4/d25/d31/d3c/f68 0 2024-04-26T04:53:16.596 INFO:tasks.workunit.client.0.smithi077.stdout:3/690: symlink d0/d7/d9a/db7/ddb/ldd 0 2024-04-26T04:53:16.599 INFO:tasks.workunit.client.0.smithi077.stdout:7/793: dwrite da/d13/ff4 [0,4194304] 0 2024-04-26T04:53:16.601 INFO:tasks.workunit.client.0.smithi077.stdout:0/714: dread d0/d47/d68/f86 [0,4194304] 0 2024-04-26T04:53:16.603 INFO:tasks.workunit.client.0.smithi077.stdout:9/666: truncate d9/d7d/d94/fc3 968291 0 2024-04-26T04:53:16.611 INFO:tasks.workunit.client.0.smithi077.stdout:2/804: write d9/dc/f41 [3719194,72213] 0 2024-04-26T04:53:16.623 INFO:tasks.workunit.client.0.smithi077.stdout:9/667: creat d9/d3a/d79/d95/dad/db2/fe8 x:0 0 0 2024-04-26T04:53:16.625 INFO:tasks.workunit.client.0.smithi077.stdout:9/668: write d9/dd/d1b/d2b/ddb/f9a [3882134,30890] 0 2024-04-26T04:53:16.626 INFO:tasks.workunit.client.0.smithi077.stdout:2/805: write d9/f12 [5760689,100373] 0 2024-04-26T04:53:16.627 INFO:tasks.workunit.client.0.smithi077.stdout:2/806: chown d9/dc/d1d/d34/f36 1906869918 1 2024-04-26T04:53:16.628 INFO:tasks.workunit.client.0.smithi077.stdout:5/765: mknod d1/d42/d62/dc8/dd5/df8/dfd/c106 0 2024-04-26T04:53:16.629 INFO:tasks.workunit.client.0.smithi077.stdout:7/794: unlink da/d15/d1a/d39/d96/cf8 0 2024-04-26T04:53:16.630 INFO:tasks.workunit.client.0.smithi077.stdout:7/795: dread - da/d15/dc6/fff zero size 2024-04-26T04:53:16.635 INFO:tasks.workunit.client.0.smithi077.stdout:8/770: creat d4/df/d22/d80/ffd x:0 0 0 2024-04-26T04:53:16.636 INFO:tasks.workunit.client.0.smithi077.stdout:9/669: dread d9/f8c [0,4194304] 0 2024-04-26T04:53:16.637 INFO:tasks.workunit.client.0.smithi077.stdout:2/807: creat d9/dc/d5f/d8e/db9/f118 x:0 0 0 2024-04-26T04:53:16.637 INFO:tasks.workunit.client.0.smithi077.stdout:5/766: symlink d1/d42/d62/dc8/dd5/l107 0 2024-04-26T04:53:16.638 INFO:tasks.workunit.client.0.smithi077.stdout:5/767: truncate d1/d8c/ff4 1003714 0 2024-04-26T04:53:16.642 INFO:tasks.workunit.client.0.smithi077.stdout:6/808: link d8/d52/d53/da7/dfa/c10c d8/d52/d53/da7/c120 0 2024-04-26T04:53:16.643 INFO:tasks.workunit.client.0.smithi077.stdout:8/771: fdatasync d4/df/d18/d56/f7c 0 2024-04-26T04:53:16.647 INFO:tasks.workunit.client.0.smithi077.stdout:6/809: dwrite d8/d1c/f24 [4194304,4194304] 0 2024-04-26T04:53:16.649 INFO:tasks.workunit.client.0.smithi077.stdout:0/715: getdents d0/d47/d64/d97/dba/dd8/dd7 0 2024-04-26T04:53:16.650 INFO:tasks.workunit.client.0.smithi077.stdout:0/716: write d0/d19/f87 [4261280,98622] 0 2024-04-26T04:53:16.654 INFO:tasks.workunit.client.0.smithi077.stdout:0/717: write d0/d19/d1e/d2f/f44 [1372711,124578] 0 2024-04-26T04:53:16.660 INFO:tasks.workunit.client.0.smithi077.stdout:9/670: mknod d9/d3a/d79/ce9 0 2024-04-26T04:53:16.661 INFO:tasks.workunit.client.0.smithi077.stdout:3/691: dwrite d0/d9/d32/d2a/f4e [0,4194304] 0 2024-04-26T04:53:16.662 INFO:tasks.workunit.client.0.smithi077.stdout:6/810: stat d8/d8f/f10a 0 2024-04-26T04:53:16.670 INFO:tasks.workunit.client.0.smithi077.stdout:0/718: mkdir d0/d47/d64/d97/df9 0 2024-04-26T04:53:16.677 INFO:tasks.workunit.client.0.smithi077.stdout:9/671: creat d9/d3a/fea x:0 0 0 2024-04-26T04:53:16.677 INFO:tasks.workunit.client.0.smithi077.stdout:2/808: write d9/dc/d1d/f47 [1330765,93893] 0 2024-04-26T04:53:16.679 INFO:tasks.workunit.client.0.smithi077.stdout:8/772: write d4/df/d18/f26 [222044,3666] 0 2024-04-26T04:53:16.680 INFO:tasks.workunit.client.0.smithi077.stdout:6/811: symlink d8/d1c/l121 0 2024-04-26T04:53:16.681 INFO:tasks.workunit.client.0.smithi077.stdout:7/796: dwrite da/f87 [0,4194304] 0 2024-04-26T04:53:16.682 INFO:tasks.workunit.client.0.smithi077.stdout:0/719: mkdir d0/d47/d93/dfa 0 2024-04-26T04:53:16.683 INFO:tasks.workunit.client.0.smithi077.stdout:0/720: readlink d0/d19/d1e/l3d 0 2024-04-26T04:53:16.688 INFO:tasks.workunit.client.0.smithi077.stdout:9/672: creat d9/d3a/d79/d95/feb x:0 0 0 2024-04-26T04:53:16.689 INFO:tasks.workunit.client.0.smithi077.stdout:9/673: chown d9/dd/d1b/d2b/ddb/fbd 522153903 1 2024-04-26T04:53:16.690 INFO:tasks.workunit.client.0.smithi077.stdout:0/721: dwrite d0/d28/d40/f85 [0,4194304] 0 2024-04-26T04:53:16.690 INFO:tasks.workunit.client.0.smithi077.stdout:3/692: creat d0/d7/d1f/d81/fde x:0 0 0 2024-04-26T04:53:16.691 INFO:tasks.workunit.client.0.smithi077.stdout:3/693: truncate d0/d7/f33 1768905 0 2024-04-26T04:53:16.768 INFO:tasks.workunit.client.0.smithi077.stdout:8/773: mkdir d4/d17/d90/dfe 0 2024-04-26T04:53:16.769 INFO:tasks.workunit.client.0.smithi077.stdout:8/774: chown d4/df/d18/f4c 55447668 1 2024-04-26T04:53:16.772 INFO:tasks.workunit.client.0.smithi077.stdout:7/797: mkdir da/d13/d22/d5d/d8d/d10c 0 2024-04-26T04:53:16.780 INFO:tasks.workunit.client.0.smithi077.stdout:0/722: creat d0/d47/d64/d97/ffb x:0 0 0 2024-04-26T04:53:16.780 INFO:tasks.workunit.client.0.smithi077.stdout:0/723: chown d0/d19/d1e/c2a 427 1 2024-04-26T04:53:16.783 INFO:tasks.workunit.client.0.smithi077.stdout:3/694: unlink d0/d7/f3e 0 2024-04-26T04:53:16.784 INFO:tasks.workunit.client.0.smithi077.stdout:0/724: dread d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:16.790 INFO:tasks.workunit.client.0.smithi077.stdout:0/725: dwrite d0/d2c/f88 [0,4194304] 0 2024-04-26T04:53:16.800 INFO:tasks.workunit.client.0.smithi077.stdout:3/695: creat d0/dae/fdf x:0 0 0 2024-04-26T04:53:16.802 INFO:tasks.workunit.client.0.smithi077.stdout:9/674: dwrite d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:16.806 INFO:tasks.workunit.client.0.smithi077.stdout:6/812: link d8/d2a/lc6 d8/d1c/da4/d10f/d11d/l122 0 2024-04-26T04:53:16.811 INFO:tasks.workunit.client.0.smithi077.stdout:2/809: link d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/ff5 d9/dc/d1d/d1e/d77/f119 0 2024-04-26T04:53:16.814 INFO:tasks.workunit.client.0.smithi077.stdout:2/810: write d9/dc/d1d/d34/f84 [3174033,45657] 0 2024-04-26T04:53:16.815 INFO:tasks.workunit.client.0.smithi077.stdout:0/726: fsync d0/d2c/d4e/f95 0 2024-04-26T04:53:16.817 INFO:tasks.workunit.client.0.smithi077.stdout:0/727: chown d0/c6c 74343 1 2024-04-26T04:53:16.818 INFO:tasks.workunit.client.0.smithi077.stdout:7/798: dwrite da/d13/d22/d7d/f85 [0,4194304] 0 2024-04-26T04:53:16.820 INFO:tasks.workunit.client.0.smithi077.stdout:9/675: truncate d9/d3a/d79/f6b 804220 0 2024-04-26T04:53:16.820 INFO:tasks.workunit.client.0.smithi077.stdout:7/799: write da/d15/d42/f77 [4346485,2545] 0 2024-04-26T04:53:16.822 INFO:tasks.workunit.client.0.smithi077.stdout:8/775: rename d4/df/d22/d2d/l38 to d4/df/d22/d80/d86/dd0/lff 0 2024-04-26T04:53:16.824 INFO:tasks.workunit.client.0.smithi077.stdout:2/811: chown d9/dc/d6e/l89 29 1 2024-04-26T04:53:16.826 INFO:tasks.workunit.client.0.smithi077.stdout:9/676: dread d9/f8d [0,4194304] 0 2024-04-26T04:53:16.834 INFO:tasks.workunit.client.0.smithi077.stdout:8/776: write d4/d17/f6a [902034,28907] 0 2024-04-26T04:53:16.835 INFO:tasks.workunit.client.0.smithi077.stdout:2/812: symlink d9/d15/d91/l11a 0 2024-04-26T04:53:16.837 INFO:tasks.workunit.client.0.smithi077.stdout:2/813: fsync d9/dc/d1d/f47 0 2024-04-26T04:53:16.838 INFO:tasks.workunit.client.0.smithi077.stdout:0/728: dread d0/d19/d1e/d59/fde [0,4194304] 0 2024-04-26T04:53:16.839 INFO:tasks.workunit.client.0.smithi077.stdout:9/677: mkdir d9/dd/dec 0 2024-04-26T04:53:16.840 INFO:tasks.workunit.client.0.smithi077.stdout:7/800: dwrite da/d13/f6d [8388608,4194304] 0 2024-04-26T04:53:16.842 INFO:tasks.workunit.client.0.smithi077.stdout:9/678: write d9/f2d [3227895,106483] 0 2024-04-26T04:53:16.844 INFO:tasks.workunit.client.0.smithi077.stdout:6/813: dread d8/f60 [0,4194304] 0 2024-04-26T04:53:16.845 INFO:tasks.workunit.client.0.smithi077.stdout:6/814: readlink d8/d52/d53/d5e/d6c/lf6 0 2024-04-26T04:53:16.846 INFO:tasks.workunit.client.0.smithi077.stdout:8/777: fdatasync d4/df/d22/d2d/f1e 0 2024-04-26T04:53:16.856 INFO:tasks.workunit.client.0.smithi077.stdout:0/729: mkdir d0/d19/d8d/df7/dfc 0 2024-04-26T04:53:16.858 INFO:tasks.workunit.client.0.smithi077.stdout:3/696: truncate d0/d7/d9a/db3/f3f 342106 0 2024-04-26T04:53:16.860 INFO:tasks.workunit.client.0.smithi077.stdout:3/697: write d0/f1b [1981689,120274] 0 2024-04-26T04:53:16.860 INFO:tasks.workunit.client.0.smithi077.stdout:6/815: mknod d8/d52/d53/d5e/d6c/c123 0 2024-04-26T04:53:16.864 INFO:tasks.workunit.client.0.smithi077.stdout:8/778: dread d4/df/d18/d56/f7c [4194304,4194304] 0 2024-04-26T04:53:16.865 INFO:tasks.workunit.client.0.smithi077.stdout:8/779: read - d4/df/dce/d5b/d8d/ff9 zero size 2024-04-26T04:53:16.868 INFO:tasks.workunit.client.0.smithi077.stdout:6/816: mknod d8/d52/d11c/c124 0 2024-04-26T04:53:16.869 INFO:tasks.workunit.client.0.smithi077.stdout:3/698: dread d0/d9/d32/d29/f6a [0,4194304] 0 2024-04-26T04:53:16.871 INFO:tasks.workunit.client.0.smithi077.stdout:8/780: creat d4/df/d22/d2d/d2b/d33/dbc/f100 x:0 0 0 2024-04-26T04:53:16.872 INFO:tasks.workunit.client.0.smithi077.stdout:6/817: write d8/f50 [1922286,123789] 0 2024-04-26T04:53:16.878 INFO:tasks.workunit.client.0.smithi077.stdout:0/730: dread d0/d19/f2d [0,4194304] 0 2024-04-26T04:53:16.878 INFO:tasks.workunit.client.0.smithi077.stdout:9/679: dwrite d9/f62 [0,4194304] 0 2024-04-26T04:53:16.878 INFO:tasks.workunit.client.0.smithi077.stdout:3/699: rmdir d0/d9/d32/d29/db8 39 2024-04-26T04:53:16.878 INFO:tasks.workunit.client.0.smithi077.stdout:0/731: readlink d0/d47/dce/lc7 0 2024-04-26T04:53:16.880 INFO:tasks.workunit.client.0.smithi077.stdout:7/801: dread da/d13/f30 [0,4194304] 0 2024-04-26T04:53:16.883 INFO:tasks.workunit.client.0.smithi077.stdout:2/814: dread d9/dc/d1d/d34/d39/d63/d79/f54 [0,4194304] 0 2024-04-26T04:53:16.884 INFO:tasks.workunit.client.0.smithi077.stdout:6/818: rename d8/d2a/f63 to d8/d2a/ddd/f125 0 2024-04-26T04:53:16.884 INFO:tasks.workunit.client.0.smithi077.stdout:8/781: symlink d4/d69/l101 0 2024-04-26T04:53:16.886 INFO:tasks.workunit.client.0.smithi077.stdout:6/819: write d8/d52/d53/f72 [3998822,66405] 0 2024-04-26T04:53:16.887 INFO:tasks.workunit.client.0.smithi077.stdout:6/820: readlink d8/d52/d53/da7/l100 0 2024-04-26T04:53:16.888 INFO:tasks.workunit.client.0.smithi077.stdout:6/821: stat d8/d52/f87 0 2024-04-26T04:53:16.889 INFO:tasks.workunit.client.0.smithi077.stdout:0/732: creat d0/d47/d68/ffd x:0 0 0 2024-04-26T04:53:16.893 INFO:tasks.workunit.client.0.smithi077.stdout:2/815: mkdir d9/dc/d1d/d1e/d77/d11b 0 2024-04-26T04:53:16.897 INFO:tasks.workunit.client.0.smithi077.stdout:3/700: mknod d0/ce0 0 2024-04-26T04:53:16.898 INFO:tasks.workunit.client.0.smithi077.stdout:3/701: write d0/d9/fca [914939,33947] 0 2024-04-26T04:53:16.900 INFO:tasks.workunit.client.0.smithi077.stdout:6/822: creat d8/d52/d53/da7/dfa/f126 x:0 0 0 2024-04-26T04:53:16.903 INFO:tasks.workunit.client.0.smithi077.stdout:0/733: mkdir d0/d47/d64/dfe 0 2024-04-26T04:53:16.906 INFO:tasks.workunit.client.0.smithi077.stdout:8/782: fdatasync d4/df/d22/d2d/f94 0 2024-04-26T04:53:16.907 INFO:tasks.workunit.client.0.smithi077.stdout:7/802: dwrite da/d15/d1a/d39/f7a [0,4194304] 0 2024-04-26T04:53:16.911 INFO:tasks.workunit.client.0.smithi077.stdout:3/702: unlink d0/c58 0 2024-04-26T04:53:16.916 INFO:tasks.workunit.client.0.smithi077.stdout:2/816: mknod d9/dc/d1d/c11c 0 2024-04-26T04:53:16.922 INFO:tasks.workunit.client.0.smithi077.stdout:6/823: mknod d8/d2a/c127 0 2024-04-26T04:53:16.922 INFO:tasks.workunit.client.0.smithi077.stdout:9/680: link d9/dd/d1b/d2b/f83 d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/dd4/fed 0 2024-04-26T04:53:16.925 INFO:tasks.workunit.client.0.smithi077.stdout:2/817: creat d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/f11d x:0 0 0 2024-04-26T04:53:16.926 INFO:tasks.workunit.client.0.smithi077.stdout:2/818: dread - d9/dc/d1d/d34/d39/d63/d79/dc2/ff1 zero size 2024-04-26T04:53:16.927 INFO:tasks.workunit.client.0.smithi077.stdout:2/819: write d9/f12 [5382841,74033] 0 2024-04-26T04:53:16.929 INFO:tasks.workunit.client.0.smithi077.stdout:8/783: symlink d4/df/d6f/db8/l102 0 2024-04-26T04:53:16.933 INFO:tasks.workunit.client.0.smithi077.stdout:9/681: readlink d9/d43/ld9 0 2024-04-26T04:53:16.934 INFO:tasks.workunit.client.0.smithi077.stdout:9/682: fdatasync d9/dd/d1b/d2b/d32/fce 0 2024-04-26T04:53:16.936 INFO:tasks.workunit.client.0.smithi077.stdout:0/734: creat d0/d19/d1e/d81/fff x:0 0 0 2024-04-26T04:53:16.940 INFO:tasks.workunit.client.0.smithi077.stdout:7/803: dwrite da/d13/d22/d5d/dfc/f10a [0,4194304] 0 2024-04-26T04:53:16.941 INFO:tasks.workunit.client.0.smithi077.stdout:3/703: dread d0/d9/f8b [0,4194304] 0 2024-04-26T04:53:16.941 INFO:tasks.workunit.client.0.smithi077.stdout:2/820: rename d9/dc/d5f/d8e/dcd to d9/dc/d1d/d34/d39/d63/de9/d11e 0 2024-04-26T04:53:16.942 INFO:tasks.workunit.client.0.smithi077.stdout:8/784: mkdir d4/df/d22/d80/d86/d103 0 2024-04-26T04:53:16.943 INFO:tasks.workunit.client.0.smithi077.stdout:3/704: chown d0/d9/d32/d2a/f4c 378301636 1 2024-04-26T04:53:16.948 INFO:tasks.workunit.client.0.smithi077.stdout:0/735: mknod d0/d47/d64/d97/dba/dd8/dd7/c100 0 2024-04-26T04:53:16.949 INFO:tasks.workunit.client.0.smithi077.stdout:7/804: creat da/d15/d1a/dcc/f10d x:0 0 0 2024-04-26T04:53:16.949 INFO:tasks.workunit.client.0.smithi077.stdout:0/736: chown d0/d19/d1e/d59 567 1 2024-04-26T04:53:16.950 INFO:tasks.workunit.client.0.smithi077.stdout:0/737: chown d0/d28/c8f 3 1 2024-04-26T04:53:16.951 INFO:tasks.workunit.client.0.smithi077.stdout:2/821: mknod d9/dc/d6e/c11f 0 2024-04-26T04:53:16.952 INFO:tasks.workunit.client.0.smithi077.stdout:8/785: chown d4/df/d18/le0 0 1 2024-04-26T04:53:16.953 INFO:tasks.workunit.client.0.smithi077.stdout:2/822: fsync d9/dc/d5f/d8e/db9/ff7 0 2024-04-26T04:53:16.953 INFO:tasks.workunit.client.0.smithi077.stdout:3/705: symlink d0/d7/d9a/le1 0 2024-04-26T04:53:16.954 INFO:tasks.workunit.client.0.smithi077.stdout:9/683: truncate d9/dd/d3d/f3e 5177446 0 2024-04-26T04:53:16.956 INFO:tasks.workunit.client.0.smithi077.stdout:3/706: write d0/d9/d32/d29/f4b [54199,79113] 0 2024-04-26T04:53:16.956 INFO:tasks.workunit.client.0.smithi077.stdout:0/738: mkdir d0/d19/d1e/d2f/da8/d101 0 2024-04-26T04:53:16.958 INFO:tasks.workunit.client.0.smithi077.stdout:7/805: dwrite da/d15/f44 [4194304,4194304] 0 2024-04-26T04:53:16.958 INFO:tasks.workunit.client.0.smithi077.stdout:0/739: chown d0/d19/d8d 3467597 1 2024-04-26T04:53:16.960 INFO:tasks.workunit.client.0.smithi077.stdout:2/823: creat d9/dc/d1d/d34/dd4/f120 x:0 0 0 2024-04-26T04:53:16.961 INFO:tasks.workunit.client.0.smithi077.stdout:2/824: chown l6 3 1 2024-04-26T04:53:16.961 INFO:tasks.workunit.client.0.smithi077.stdout:8/786: truncate d4/d17/d90/faf 716027 0 2024-04-26T04:53:16.963 INFO:tasks.workunit.client.0.smithi077.stdout:9/684: dread f2 [0,4194304] 0 2024-04-26T04:53:16.963 INFO:tasks.workunit.client.0.smithi077.stdout:8/787: fsync d4/df/d18/f26 0 2024-04-26T04:53:16.964 INFO:tasks.workunit.client.0.smithi077.stdout:3/707: creat d0/dae/fe2 x:0 0 0 2024-04-26T04:53:16.969 INFO:tasks.workunit.client.0.smithi077.stdout:2/825: dwrite d9/f2c [4194304,4194304] 0 2024-04-26T04:53:16.970 INFO:tasks.workunit.client.0.smithi077.stdout:2/826: dread - d9/dc/d1d/d34/d39/d63/f104 zero size 2024-04-26T04:53:16.975 INFO:tasks.workunit.client.0.smithi077.stdout:3/708: mkdir d0/d7/d1f/d4d/dc6/de3 0 2024-04-26T04:53:16.975 INFO:tasks.workunit.client.0.smithi077.stdout:6/824: write d8/d1c/f5c [1167406,65922] 0 2024-04-26T04:53:16.976 INFO:tasks.workunit.client.0.smithi077.stdout:8/788: mkdir d4/df/d22/d2d/d104 0 2024-04-26T04:53:16.980 INFO:tasks.workunit.client.0.smithi077.stdout:8/789: truncate d4/df/d22/d2d/d2b/d33/dbc/f100 24325 0 2024-04-26T04:53:16.982 INFO:tasks.workunit.client.0.smithi077.stdout:6/825: creat d8/d52/d53/da7/df4/f128 x:0 0 0 2024-04-26T04:53:16.986 INFO:tasks.workunit.client.0.smithi077.stdout:3/709: rename d0/d7/d1f/d76/ca0 to d0/d7/d1f/d81/dc5/ce4 0 2024-04-26T04:53:16.990 INFO:tasks.workunit.client.0.smithi077.stdout:8/790: symlink d4/df/d22/d80/d86/l105 0 2024-04-26T04:53:16.992 INFO:tasks.workunit.client.0.smithi077.stdout:7/806: dread da/d6b/d63/f81 [0,4194304] 0 2024-04-26T04:53:16.994 INFO:tasks.workunit.client.0.smithi077.stdout:9/685: link d9/dd/d1b/d2b/d32/lc5 d9/d35/d99/lee 0 2024-04-26T04:53:17.000 INFO:tasks.workunit.client.0.smithi077.stdout:0/740: dwrite d0/d47/d64/f80 [0,4194304] 0 2024-04-26T04:53:17.000 INFO:tasks.workunit.client.0.smithi077.stdout:4/783: sync 2024-04-26T04:53:17.000 INFO:tasks.workunit.client.0.smithi077.stdout:9/686: rmdir d9/d45/db1 39 2024-04-26T04:53:17.003 INFO:tasks.workunit.client.0.smithi077.stdout:3/710: mknod d0/ce5 0 2024-04-26T04:53:17.004 INFO:tasks.workunit.client.0.smithi077.stdout:0/741: dread - d0/d47/dce/fd6 zero size 2024-04-26T04:53:17.006 INFO:tasks.workunit.client.0.smithi077.stdout:2/827: dwrite d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:53:17.008 INFO:tasks.workunit.client.0.smithi077.stdout:3/711: chown d0/d9/d32/d29/dc8/ld4 273295790 1 2024-04-26T04:53:17.008 INFO:tasks.workunit.client.0.smithi077.stdout:7/807: dread da/d13/d22/f99 [0,4194304] 0 2024-04-26T04:53:17.011 INFO:tasks.workunit.client.0.smithi077.stdout:9/687: stat d9/d35/d99/ce3 0 2024-04-26T04:53:17.014 INFO:tasks.workunit.client.0.smithi077.stdout:4/784: creat d4/d17/d18/d43/d6e/d6c/dae/dc5/f106 x:0 0 0 2024-04-26T04:53:17.015 INFO:tasks.workunit.client.0.smithi077.stdout:0/742: truncate d0/d47/d93/d94/faa 1319921 0 2024-04-26T04:53:17.015 INFO:tasks.workunit.client.0.smithi077.stdout:8/791: write d4/d17/fc5 [74198,23638] 0 2024-04-26T04:53:17.018 INFO:tasks.workunit.client.0.smithi077.stdout:3/712: mknod d0/d7/d1f/d76/ce6 0 2024-04-26T04:53:17.026 INFO:tasks.workunit.client.0.smithi077.stdout:6/826: dwrite d8/d2a/d86/fe0 [0,4194304] 0 2024-04-26T04:53:17.028 INFO:tasks.workunit.client.0.smithi077.stdout:3/713: write d0/d7/d1f/d76/f7e [7392288,94488] 0 2024-04-26T04:53:17.033 INFO:tasks.workunit.client.0.smithi077.stdout:0/743: dwrite d0/d47/f30 [0,4194304] 0 2024-04-26T04:53:17.034 INFO:tasks.workunit.client.0.smithi077.stdout:0/744: chown d0/d2c/d4e/c67 4503 1 2024-04-26T04:53:17.034 INFO:tasks.workunit.client.0.smithi077.stdout:1/749: sync 2024-04-26T04:53:17.034 INFO:tasks.workunit.client.0.smithi077.stdout:5/768: sync 2024-04-26T04:53:17.043 INFO:tasks.workunit.client.0.smithi077.stdout:0/745: dwrite d0/d47/d64/d97/dba/dd8/dd7/fe6 [0,4194304] 0 2024-04-26T04:53:17.045 INFO:tasks.workunit.client.0.smithi077.stdout:0/746: write d0/d19/d69/fe8 [53022,90055] 0 2024-04-26T04:53:17.060 INFO:tasks.workunit.client.0.smithi077.stdout:7/808: getdents da/d15/d1a/d39/df3 0 2024-04-26T04:53:17.062 INFO:tasks.workunit.client.0.smithi077.stdout:8/792: symlink d4/df/d22/d80/d86/dd0/l106 0 2024-04-26T04:53:17.065 INFO:tasks.workunit.client.0.smithi077.stdout:9/688: mkdir d9/dd/def 0 2024-04-26T04:53:17.067 INFO:tasks.workunit.client.0.smithi077.stdout:4/785: fsync d4/d17/d18/d43/d51/db4/d15/d1f/fcf 0 2024-04-26T04:53:17.073 INFO:tasks.workunit.client.0.smithi077.stdout:1/750: creat d9/db/d2b/ff2 x:0 0 0 2024-04-26T04:53:17.075 INFO:tasks.workunit.client.0.smithi077.stdout:5/769: mknod d1/d42/d62/dc8/dd5/c108 0 2024-04-26T04:53:17.077 INFO:tasks.workunit.client.0.smithi077.stdout:5/770: write d1/d8/d7e/fc6 [267313,105920] 0 2024-04-26T04:53:17.077 INFO:tasks.workunit.client.0.smithi077.stdout:5/771: stat d1/d30/d33/d66/cab 0 2024-04-26T04:53:17.081 INFO:tasks.workunit.client.0.smithi077.stdout:8/793: fsync d4/df/dce/d5b/fa0 0 2024-04-26T04:53:17.086 INFO:tasks.workunit.client.0.smithi077.stdout:6/827: dread d8/d1c/f25 [0,4194304] 0 2024-04-26T04:53:17.088 INFO:tasks.workunit.client.0.smithi077.stdout:2/828: rename d9/dc/d1d/d34/df8/dd1/dc3/cc5 to d9/dc/d1d/d34/c121 0 2024-04-26T04:53:17.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/809: getdents da/d13/d22/d5d/d8d/d10c 0 2024-04-26T04:53:17.095 INFO:tasks.workunit.client.0.smithi077.stdout:7/810: fsync da/d6b/f106 0 2024-04-26T04:53:17.097 INFO:tasks.workunit.client.0.smithi077.stdout:7/811: fdatasync da/d13/d22/d5d/d9f/da7/de0/ffb 0 2024-04-26T04:53:17.098 INFO:tasks.workunit.client.0.smithi077.stdout:8/794: dread - d4/df/d6f/da3/fb1 zero size 2024-04-26T04:53:17.099 INFO:tasks.workunit.client.0.smithi077.stdout:9/689: fdatasync d9/d45/db1/fc0 0 2024-04-26T04:53:17.100 INFO:tasks.workunit.client.0.smithi077.stdout:6/828: symlink d8/d52/d53/d5e/da6/de6/l129 0 2024-04-26T04:53:17.101 INFO:tasks.workunit.client.0.smithi077.stdout:0/747: dwrite d0/d2c/d4e/f3e [0,4194304] 0 2024-04-26T04:53:17.101 INFO:tasks.workunit.client.0.smithi077.stdout:0/748: read d0/d19/f2d [2004584,13294] 0 2024-04-26T04:53:17.103 INFO:tasks.workunit.client.0.smithi077.stdout:4/786: write d4/d17/d18/d43/d63/d80/fbd [764553,646] 0 2024-04-26T04:53:17.104 INFO:tasks.workunit.client.0.smithi077.stdout:2/829: write d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f114 [964863,125454] 0 2024-04-26T04:53:17.105 INFO:tasks.workunit.client.0.smithi077.stdout:3/714: truncate d0/fb 8861041 0 2024-04-26T04:53:17.106 INFO:tasks.workunit.client.0.smithi077.stdout:2/830: chown d9/dc/d1d/d1e/d77/df9/l116 29798188 1 2024-04-26T04:53:17.110 INFO:tasks.workunit.client.0.smithi077.stdout:1/751: unlink d9/db/dd/d68/fe3 0 2024-04-26T04:53:17.111 INFO:tasks.workunit.client.0.smithi077.stdout:5/772: dwrite d1/d42/d62/fc4 [0,4194304] 0 2024-04-26T04:53:17.112 INFO:tasks.workunit.client.0.smithi077.stdout:5/773: write d1/d8d/da7/ffa [274114,67293] 0 2024-04-26T04:53:17.113 INFO:tasks.workunit.client.0.smithi077.stdout:7/812: mknod da/d6b/c10e 0 2024-04-26T04:53:17.113 INFO:tasks.workunit.client.0.smithi077.stdout:8/795: fdatasync d4/df/d22/d75/fcf 0 2024-04-26T04:53:17.121 INFO:tasks.workunit.client.0.smithi077.stdout:0/749: creat d0/d28/d40/f102 x:0 0 0 2024-04-26T04:53:17.130 INFO:tasks.workunit.client.0.smithi077.stdout:4/787: unlink d4/d17/d18/d43/d51/db4/cc6 0 2024-04-26T04:53:17.134 INFO:tasks.workunit.client.0.smithi077.stdout:3/715: rename d0/d7/d1f/d76/la2 to d0/d7/dce/le7 0 2024-04-26T04:53:17.138 INFO:tasks.workunit.client.0.smithi077.stdout:7/813: fdatasync da/d13/d22/d34/fac 0 2024-04-26T04:53:17.139 INFO:tasks.workunit.client.0.smithi077.stdout:3/716: dread d0/d7/fad [0,4194304] 0 2024-04-26T04:53:17.140 INFO:tasks.workunit.client.0.smithi077.stdout:8/796: symlink d4/df/d6f/db8/l107 0 2024-04-26T04:53:17.141 INFO:tasks.workunit.client.0.smithi077.stdout:9/690: fsync d9/dd/d3d/f3e 0 2024-04-26T04:53:17.146 INFO:tasks.workunit.client.0.smithi077.stdout:1/752: dread d9/db/dd/d34/d63/f76 [0,4194304] 0 2024-04-26T04:53:17.149 INFO:tasks.workunit.client.0.smithi077.stdout:0/750: rmdir d0/d19 39 2024-04-26T04:53:17.150 INFO:tasks.workunit.client.0.smithi077.stdout:4/788: readlink d4/l6 0 2024-04-26T04:53:17.151 INFO:tasks.workunit.client.0.smithi077.stdout:4/789: write d4/d17/d18/d43/d51/db4/d15/d1f/fcf [397118,82529] 0 2024-04-26T04:53:17.152 INFO:tasks.workunit.client.0.smithi077.stdout:2/831: mkdir d9/dc/d1d/d122 0 2024-04-26T04:53:17.160 INFO:tasks.workunit.client.0.smithi077.stdout:3/717: rename d0/ce0 to d0/d9/d32/d2a/d85/dd3/ce8 0 2024-04-26T04:53:17.164 INFO:tasks.workunit.client.0.smithi077.stdout:9/691: unlink d9/dd/d3d/faf 0 2024-04-26T04:53:17.164 INFO:tasks.workunit.client.0.smithi077.stdout:5/774: dwrite d1/d30/f53 [0,4194304] 0 2024-04-26T04:53:17.168 INFO:tasks.workunit.client.0.smithi077.stdout:1/753: mkdir d9/db/dd/d9c/dab/de8/df3 0 2024-04-26T04:53:17.168 INFO:tasks.workunit.client.0.smithi077.stdout:3/718: dwrite d0/d9/fca [0,4194304] 0 2024-04-26T04:53:17.171 INFO:tasks.workunit.client.0.smithi077.stdout:0/751: chown d0/d19/d69/fe8 7270 1 2024-04-26T04:53:17.173 INFO:tasks.workunit.client.0.smithi077.stdout:0/752: chown d0/d19/d1e/d59/d96 4190 1 2024-04-26T04:53:17.174 INFO:tasks.workunit.client.0.smithi077.stdout:8/797: dwrite d4/df/d22/d2d/f1e [0,4194304] 0 2024-04-26T04:53:17.178 INFO:tasks.workunit.client.0.smithi077.stdout:4/790: rename d4/d17/d18/l33 to d4/d17/d18/d43/df7/l107 0 2024-04-26T04:53:17.182 INFO:tasks.workunit.client.0.smithi077.stdout:6/829: getdents d8/d1c/da4/d10f/d11d 0 2024-04-26T04:53:17.187 INFO:tasks.workunit.client.0.smithi077.stdout:4/791: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/d76/f82 [0,4194304] 0 2024-04-26T04:53:17.190 INFO:tasks.workunit.client.0.smithi077.stdout:9/692: dwrite d9/dd/fb3 [0,4194304] 0 2024-04-26T04:53:17.193 INFO:tasks.workunit.client.0.smithi077.stdout:9/693: write d9/dd/d3d/f5a [42455,25917] 0 2024-04-26T04:53:17.197 INFO:tasks.workunit.client.0.smithi077.stdout:0/753: creat d0/d47/d64/d9e/f103 x:0 0 0 2024-04-26T04:53:17.198 INFO:tasks.workunit.client.0.smithi077.stdout:7/814: creat da/d15/d1a/f10f x:0 0 0 2024-04-26T04:53:17.200 INFO:tasks.workunit.client.0.smithi077.stdout:4/792: dwrite d4/d17/d18/d43/d63/d80/db6/fee [4194304,4194304] 0 2024-04-26T04:53:17.200 INFO:tasks.workunit.client.0.smithi077.stdout:0/754: chown d0/d47/le4 2095544526 1 2024-04-26T04:53:17.209 INFO:tasks.workunit.client.0.smithi077.stdout:1/754: dwrite d9/d53/d8b/d93/f6d [0,4194304] 0 2024-04-26T04:53:17.213 INFO:tasks.workunit.client.0.smithi077.stdout:1/755: write d9/d53/d8b/d93/da8/da7/fd9 [776499,5506] 0 2024-04-26T04:53:17.214 INFO:tasks.workunit.client.0.smithi077.stdout:2/832: creat d9/dc/f123 x:0 0 0 2024-04-26T04:53:17.214 INFO:tasks.workunit.client.0.smithi077.stdout:9/694: mknod d9/dd/d1b/d2b/d32/d36/d7f/cf0 0 2024-04-26T04:53:17.216 INFO:tasks.workunit.client.0.smithi077.stdout:4/793: mknod d4/d17/d18/d43/d63/d80/c108 0 2024-04-26T04:53:17.216 INFO:tasks.workunit.client.0.smithi077.stdout:2/833: fdatasync d9/dc/d1d/d1e/d77/df9/f110 0 2024-04-26T04:53:17.220 INFO:tasks.workunit.client.0.smithi077.stdout:2/834: read d9/d15/f31 [4175562,30969] 0 2024-04-26T04:53:17.221 INFO:tasks.workunit.client.0.smithi077.stdout:8/798: write d4/df/d18/d56/fa6 [2651950,18612] 0 2024-04-26T04:53:17.222 INFO:tasks.workunit.client.0.smithi077.stdout:0/755: creat d0/d47/d64/dfe/f104 x:0 0 0 2024-04-26T04:53:17.226 INFO:tasks.workunit.client.0.smithi077.stdout:0/756: write d0/d47/d68/d9f/ff8 [445190,125346] 0 2024-04-26T04:53:17.230 INFO:tasks.workunit.client.0.smithi077.stdout:4/794: mknod d4/d17/d18/d43/df7/c109 0 2024-04-26T04:53:17.230 INFO:tasks.workunit.client.0.smithi077.stdout:3/719: write d0/fb [1170964,97442] 0 2024-04-26T04:53:17.231 INFO:tasks.workunit.client.0.smithi077.stdout:9/695: truncate d9/d3a/d79/f68 561045 0 2024-04-26T04:53:17.231 INFO:tasks.workunit.client.0.smithi077.stdout:7/815: dread da/d6b/f93 [0,4194304] 0 2024-04-26T04:53:17.232 INFO:tasks.workunit.client.0.smithi077.stdout:6/830: dread d8/d1c/f79 [0,4194304] 0 2024-04-26T04:53:17.233 INFO:tasks.workunit.client.0.smithi077.stdout:5/775: dwrite d1/d12/f1f [0,4194304] 0 2024-04-26T04:53:17.240 INFO:tasks.workunit.client.0.smithi077.stdout:1/756: write d9/d53/d6e/f30 [216511,85784] 0 2024-04-26T04:53:17.242 INFO:tasks.workunit.client.0.smithi077.stdout:5/776: dwrite d1/d42/da4/d25/fda [0,4194304] 0 2024-04-26T04:53:17.249 INFO:tasks.workunit.client.0.smithi077.stdout:9/696: unlink d9/d3a/d79/d95/dad/fd0 0 2024-04-26T04:53:17.252 INFO:tasks.workunit.client.0.smithi077.stdout:8/799: mknod d4/df/d22/c108 0 2024-04-26T04:53:17.253 INFO:tasks.workunit.client.0.smithi077.stdout:2/835: dwrite d9/dc/d1d/d34/fa9 [0,4194304] 0 2024-04-26T04:53:17.255 INFO:tasks.workunit.client.0.smithi077.stdout:6/831: dread - d8/d52/d53/da7/fc3 zero size 2024-04-26T04:53:17.255 INFO:tasks.workunit.client.0.smithi077.stdout:8/800: chown d4/df/d22/d2d/d2b/d33/fa7 774430 1 2024-04-26T04:53:17.256 INFO:tasks.workunit.client.0.smithi077.stdout:2/836: write d9/d15/d91/f96 [3324550,81781] 0 2024-04-26T04:53:17.260 INFO:tasks.workunit.client.0.smithi077.stdout:2/837: dread - d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/f11d zero size 2024-04-26T04:53:17.260 INFO:tasks.workunit.client.0.smithi077.stdout:3/720: write d0/d7/fad [2481305,41872] 0 2024-04-26T04:53:17.262 INFO:tasks.workunit.client.0.smithi077.stdout:0/757: mkdir d0/d19/d1e/d105 0 2024-04-26T04:53:17.267 INFO:tasks.workunit.client.0.smithi077.stdout:5/777: mkdir d1/d30/d33/d109 0 2024-04-26T04:53:17.268 INFO:tasks.workunit.client.0.smithi077.stdout:5/778: readlink d1/d42/l4c 0 2024-04-26T04:53:17.268 INFO:tasks.workunit.client.0.smithi077.stdout:1/757: dread d9/db/dd/d34/d63/f77 [0,4194304] 0 2024-04-26T04:53:17.269 INFO:tasks.workunit.client.0.smithi077.stdout:5/779: chown d1/d12/c4d 18722799 1 2024-04-26T04:53:17.269 INFO:tasks.workunit.client.0.smithi077.stdout:4/795: dread d4/d4e/dad/fd7 [0,4194304] 0 2024-04-26T04:53:17.270 INFO:tasks.workunit.client.0.smithi077.stdout:9/697: read d9/f82 [107317,1493] 0 2024-04-26T04:53:17.271 INFO:tasks.workunit.client.0.smithi077.stdout:9/698: read d9/d45/f7e [82341,10213] 0 2024-04-26T04:53:17.274 INFO:tasks.workunit.client.0.smithi077.stdout:3/721: mknod d0/d7/d1f/d4d/dbe/ce9 0 2024-04-26T04:53:17.276 INFO:tasks.workunit.client.0.smithi077.stdout:0/758: rename d0/d47/d64/d9e to d0/d19/d1e/d59/d96/d106 0 2024-04-26T04:53:17.278 INFO:tasks.workunit.client.0.smithi077.stdout:1/758: fdatasync d9/db/d2b/f61 0 2024-04-26T04:53:17.280 INFO:tasks.workunit.client.0.smithi077.stdout:5/780: creat d1/d42/da4/d84/f10a x:0 0 0 2024-04-26T04:53:17.282 INFO:tasks.workunit.client.0.smithi077.stdout:4/796: mkdir d4/d4e/dad/d10a 0 2024-04-26T04:53:17.283 INFO:tasks.workunit.client.0.smithi077.stdout:4/797: chown d4/d17/d7f/cd2 4183360 1 2024-04-26T04:53:17.284 INFO:tasks.workunit.client.0.smithi077.stdout:8/801: mknod d4/df/c109 0 2024-04-26T04:53:17.284 INFO:tasks.workunit.client.0.smithi077.stdout:4/798: chown d4/d17/d18/d43/d53 2679 1 2024-04-26T04:53:17.285 INFO:tasks.workunit.client.0.smithi077.stdout:6/832: mkdir d8/d1c/d108/d12a 0 2024-04-26T04:53:17.286 INFO:tasks.workunit.client.0.smithi077.stdout:9/699: chown d9/d3a/d79/f6b 14 1 2024-04-26T04:53:17.289 INFO:tasks.workunit.client.0.smithi077.stdout:5/781: dwrite d1/d12/d28/d38/f8b [0,4194304] 0 2024-04-26T04:53:17.289 INFO:tasks.workunit.client.0.smithi077.stdout:9/700: fsync d9/dd/d1b/d2b/ddb/f9a 0 2024-04-26T04:53:17.290 INFO:tasks.workunit.client.0.smithi077.stdout:2/838: mknod d9/dc/d1d/d34/df8/dd1/c124 0 2024-04-26T04:53:17.290 INFO:tasks.workunit.client.0.smithi077.stdout:2/839: readlink l6 0 2024-04-26T04:53:17.291 INFO:tasks.workunit.client.0.smithi077.stdout:3/722: symlink d0/d7/d9a/db7/lea 0 2024-04-26T04:53:17.291 INFO:tasks.workunit.client.0.smithi077.stdout:0/759: creat d0/d47/d64/d97/dba/dd8/dd7/f107 x:0 0 0 2024-04-26T04:53:17.299 INFO:tasks.workunit.client.0.smithi077.stdout:5/782: creat d1/d42/d62/d6b/de8/f10b x:0 0 0 2024-04-26T04:53:17.300 INFO:tasks.workunit.client.0.smithi077.stdout:9/701: chown d9/d43/lcf 3710 1 2024-04-26T04:53:17.303 INFO:tasks.workunit.client.0.smithi077.stdout:1/759: fsync d9/db/f39 0 2024-04-26T04:53:17.304 INFO:tasks.workunit.client.0.smithi077.stdout:4/799: creat d4/d4e/dad/df8/f10b x:0 0 0 2024-04-26T04:53:17.306 INFO:tasks.workunit.client.0.smithi077.stdout:6/833: mkdir d8/d52/d53/d5e/da6/dd1/d12b 0 2024-04-26T04:53:17.307 INFO:tasks.workunit.client.0.smithi077.stdout:2/840: dwrite d9/dc/f7a [4194304,4194304] 0 2024-04-26T04:53:17.309 INFO:tasks.workunit.client.0.smithi077.stdout:2/841: write d9/dc/d1d/d34/d39/d63/f104 [104045,96502] 0 2024-04-26T04:53:17.311 INFO:tasks.workunit.client.0.smithi077.stdout:5/783: rename d1/d12/d28/d38/f90 to d1/d8d/da7/f10c 0 2024-04-26T04:53:17.315 INFO:tasks.workunit.client.0.smithi077.stdout:0/760: mknod d0/d19/d8d/dee/c108 0 2024-04-26T04:53:17.315 INFO:tasks.workunit.client.0.smithi077.stdout:9/702: dread d9/d35/d99/fa3 [0,4194304] 0 2024-04-26T04:53:17.316 INFO:tasks.workunit.client.0.smithi077.stdout:0/761: chown d0/d47/d64/d97/dba/dd8 509 1 2024-04-26T04:53:17.319 INFO:tasks.workunit.client.0.smithi077.stdout:4/800: chown d4/d17/d18/d43/d63/d80/db6/lb9 1302 1 2024-04-26T04:53:17.321 INFO:tasks.workunit.client.0.smithi077.stdout:6/834: write d8/d52/d53/d5e/fb0 [1669654,34842] 0 2024-04-26T04:53:17.322 INFO:tasks.workunit.client.0.smithi077.stdout:5/784: mknod d1/d12/d28/d38/c10d 0 2024-04-26T04:53:17.325 INFO:tasks.workunit.client.0.smithi077.stdout:2/842: mknod d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/c125 0 2024-04-26T04:53:17.327 INFO:tasks.workunit.client.0.smithi077.stdout:3/723: creat d0/feb x:0 0 0 2024-04-26T04:53:17.329 INFO:tasks.workunit.client.0.smithi077.stdout:5/785: dwrite d1/d30/f53 [0,4194304] 0 2024-04-26T04:53:17.331 INFO:tasks.workunit.client.0.smithi077.stdout:5/786: chown d1/d8/d7e 5496 1 2024-04-26T04:53:17.331 INFO:tasks.workunit.client.0.smithi077.stdout:0/762: fdatasync d0/d47/f30 0 2024-04-26T04:53:17.332 INFO:tasks.workunit.client.0.smithi077.stdout:5/787: chown d1/d42/d62/d6b 794 1 2024-04-26T04:53:17.333 INFO:tasks.workunit.client.0.smithi077.stdout:0/763: stat d0/d47/l8b 0 2024-04-26T04:53:17.334 INFO:tasks.workunit.client.0.smithi077.stdout:8/802: dwrite d4/df/d22/d2d/d2b/f4d [0,4194304] 0 2024-04-26T04:53:17.335 INFO:tasks.workunit.client.0.smithi077.stdout:1/760: symlink d9/db/dd/d9c/dab/de8/df3/lf4 0 2024-04-26T04:53:17.336 INFO:tasks.workunit.client.0.smithi077.stdout:1/761: write d9/db/dd/f5b [2802148,53089] 0 2024-04-26T04:53:17.338 INFO:tasks.workunit.client.0.smithi077.stdout:4/801: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d25/d10c 0 2024-04-26T04:53:17.351 INFO:tasks.workunit.client.0.smithi077.stdout:9/703: dwrite d9/f8d [0,4194304] 0 2024-04-26T04:53:17.352 INFO:tasks.workunit.client.0.smithi077.stdout:1/762: mknod d9/d53/d8b/d93/da8/db8/cf5 0 2024-04-26T04:53:17.353 INFO:tasks.workunit.client.0.smithi077.stdout:1/763: dread - d9/db/d2b/d3f/fc3 zero size 2024-04-26T04:53:17.359 INFO:tasks.workunit.client.0.smithi077.stdout:2/843: rename d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/l10c to d9/dc/d1d/d34/l126 0 2024-04-26T04:53:17.361 INFO:tasks.workunit.client.0.smithi077.stdout:2/844: write d9/dc/d1d/d34/df8/dd1/fdf [1031473,58534] 0 2024-04-26T04:53:17.370 INFO:tasks.workunit.client.0.smithi077.stdout:5/788: link d1/d12/l96 d1/d12/d87/l10e 0 2024-04-26T04:53:17.371 INFO:tasks.workunit.client.0.smithi077.stdout:5/789: truncate d1/d42/da4/d25/fda 4223646 0 2024-04-26T04:53:17.373 INFO:tasks.workunit.client.0.smithi077.stdout:9/704: truncate d9/d45/f4a 4535123 0 2024-04-26T04:53:17.377 INFO:tasks.workunit.client.0.smithi077.stdout:1/764: mkdir d9/d90/df6 0 2024-04-26T04:53:17.378 INFO:tasks.workunit.client.0.smithi077.stdout:8/803: getdents d4/df/d6f/db8 0 2024-04-26T04:53:17.379 INFO:tasks.workunit.client.0.smithi077.stdout:4/802: truncate d4/d17/d18/d43/d51/db4/d15/d1f/f32 7774115 0 2024-04-26T04:53:17.385 INFO:tasks.workunit.client.0.smithi077.stdout:8/804: dwrite d4/df/f32 [0,4194304] 0 2024-04-26T04:53:17.388 INFO:tasks.workunit.client.0.smithi077.stdout:9/705: symlink d9/dd/def/lf1 0 2024-04-26T04:53:17.390 INFO:tasks.workunit.client.0.smithi077.stdout:2/845: chown d9/dc/d1d/d34/l126 31165988 1 2024-04-26T04:53:17.392 INFO:tasks.workunit.client.0.smithi077.stdout:0/764: rename d0/d47/d64/d97/dba/dd8 to d0/d2c/d109 0 2024-04-26T04:53:17.393 INFO:tasks.workunit.client.0.smithi077.stdout:0/765: stat d0/d28/ceb 0 2024-04-26T04:53:17.396 INFO:tasks.workunit.client.0.smithi077.stdout:8/805: fsync d4/df/fd6 0 2024-04-26T04:53:17.404 INFO:tasks.workunit.client.0.smithi077.stdout:8/806: dwrite d4/d69/d73/fef [0,4194304] 0 2024-04-26T04:53:17.404 INFO:tasks.workunit.client.0.smithi077.stdout:9/706: dwrite d9/dd/fb3 [0,4194304] 0 2024-04-26T04:53:17.404 INFO:tasks.workunit.client.0.smithi077.stdout:2/846: rmdir d9/dc 39 2024-04-26T04:53:17.406 INFO:tasks.workunit.client.0.smithi077.stdout:9/707: chown d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3 1812183 1 2024-04-26T04:53:17.408 INFO:tasks.workunit.client.0.smithi077.stdout:9/708: dread d9/d43/f56 [0,4194304] 0 2024-04-26T04:53:17.412 INFO:tasks.workunit.client.0.smithi077.stdout:4/803: link d4/d17/d18/d43/lea d4/d4e/dad/d8b/dba/de5/l10d 0 2024-04-26T04:53:17.421 INFO:tasks.workunit.client.0.smithi077.stdout:9/709: fdatasync d9/dd/d1b/d2b/d32/d36/d7f/f85 0 2024-04-26T04:53:17.422 INFO:tasks.workunit.client.0.smithi077.stdout:2/847: dwrite d9/dc/d1d/d34/d39/d63/fca [0,4194304] 0 2024-04-26T04:53:17.428 INFO:tasks.workunit.client.0.smithi077.stdout:9/710: dwrite d9/d7d/dd1/fd7 [0,4194304] 0 2024-04-26T04:53:17.433 INFO:tasks.workunit.client.0.smithi077.stdout:1/765: write d9/d53/d6e/d7c/fcf [449844,64750] 0 2024-04-26T04:53:17.438 INFO:tasks.workunit.client.0.smithi077.stdout:2/848: dread d9/f12 [0,4194304] 0 2024-04-26T04:53:17.441 INFO:tasks.workunit.client.0.smithi077.stdout:2/849: write d9/dc/d1d/d34/fd7 [698503,38587] 0 2024-04-26T04:53:17.442 INFO:tasks.workunit.client.0.smithi077.stdout:1/766: dwrite d9/d90/fad [0,4194304] 0 2024-04-26T04:53:17.446 INFO:tasks.workunit.client.0.smithi077.stdout:9/711: getdents d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3 0 2024-04-26T04:53:17.448 INFO:tasks.workunit.client.0.smithi077.stdout:2/850: symlink d9/dc/d5f/d8e/da6/dbd/l127 0 2024-04-26T04:53:17.449 INFO:tasks.workunit.client.0.smithi077.stdout:1/767: mknod d9/db/d2b/d75/cf7 0 2024-04-26T04:53:17.450 INFO:tasks.workunit.client.0.smithi077.stdout:0/766: write d0/d19/d69/f84 [580801,59225] 0 2024-04-26T04:53:17.455 INFO:tasks.workunit.client.0.smithi077.stdout:1/768: symlink d9/db/dd/lf8 0 2024-04-26T04:53:17.457 INFO:tasks.workunit.client.0.smithi077.stdout:2/851: dwrite d9/dc/d5f/d8e/db9/ff7 [0,4194304] 0 2024-04-26T04:53:17.461 INFO:tasks.workunit.client.0.smithi077.stdout:1/769: creat d9/db/dd/d9c/dab/de8/ff9 x:0 0 0 2024-04-26T04:53:17.465 INFO:tasks.workunit.client.0.smithi077.stdout:1/770: fsync d9/d90/f83 0 2024-04-26T04:53:17.467 INFO:tasks.workunit.client.0.smithi077.stdout:8/807: write d4/df/d22/d2d/f4a [2330787,127753] 0 2024-04-26T04:53:17.470 INFO:tasks.workunit.client.0.smithi077.stdout:2/852: creat d9/dc/d1d/d34/d39/d63/d79/f128 x:0 0 0 2024-04-26T04:53:17.473 INFO:tasks.workunit.client.0.smithi077.stdout:8/808: creat d4/df/d22/d80/d86/d103/f10a x:0 0 0 2024-04-26T04:53:17.473 INFO:tasks.workunit.client.0.smithi077.stdout:1/771: dwrite d9/db/f4c [0,4194304] 0 2024-04-26T04:53:17.474 INFO:tasks.workunit.client.0.smithi077.stdout:2/853: dread - d9/dc/d1d/d34/d39/d63/d79/dc2/ff1 zero size 2024-04-26T04:53:17.477 INFO:tasks.workunit.client.0.smithi077.stdout:1/772: write d9/f1b [838022,66835] 0 2024-04-26T04:53:17.477 INFO:tasks.workunit.client.0.smithi077.stdout:2/854: chown d9/dc/d5f/d78/c95 64517 1 2024-04-26T04:53:17.481 INFO:tasks.workunit.client.0.smithi077.stdout:1/773: rmdir d9/db/d2b/d3f 39 2024-04-26T04:53:17.482 INFO:tasks.workunit.client.0.smithi077.stdout:2/855: dwrite d9/d15/f9f [0,4194304] 0 2024-04-26T04:53:17.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/774: fsync d9/db/dd/d34/d63/f9e 0 2024-04-26T04:53:17.486 INFO:tasks.workunit.client.0.smithi077.stdout:1/775: dread - d9/d53/d8b/d93/da8/db8/fc5 zero size 2024-04-26T04:53:17.486 INFO:tasks.workunit.client.0.smithi077.stdout:1/776: chown d9 0 1 2024-04-26T04:53:17.488 INFO:tasks.workunit.client.0.smithi077.stdout:8/809: rename d4/df/d22/d2d/d2b/d33/c84 to d4/df/d22/c10b 0 2024-04-26T04:53:17.489 INFO:tasks.workunit.client.0.smithi077.stdout:8/810: write d4/d17/fc5 [567724,118775] 0 2024-04-26T04:53:17.494 INFO:tasks.workunit.client.0.smithi077.stdout:8/811: rmdir d4/df/d22/d2d/d2b/d33/dd8 39 2024-04-26T04:53:17.496 INFO:tasks.workunit.client.0.smithi077.stdout:8/812: creat d4/d17/ddc/f10c x:0 0 0 2024-04-26T04:53:17.498 INFO:tasks.workunit.client.0.smithi077.stdout:8/813: creat d4/d17/ddc/f10d x:0 0 0 2024-04-26T04:53:17.500 INFO:tasks.workunit.client.0.smithi077.stdout:8/814: symlink d4/df/d6f/da3/l10e 0 2024-04-26T04:53:17.503 INFO:tasks.workunit.client.0.smithi077.stdout:8/815: unlink d4/df/d18/l3e 0 2024-04-26T04:53:17.534 INFO:tasks.workunit.client.0.smithi077.stdout:9/712: dwrite d9/d3a/d79/f6b [0,4194304] 0 2024-04-26T04:53:17.539 INFO:tasks.workunit.client.0.smithi077.stdout:8/816: mknod d4/df/d6f/da3/dc7/dde/c10f 0 2024-04-26T04:53:17.541 INFO:tasks.workunit.client.0.smithi077.stdout:9/713: mkdir d9/dd/df2 0 2024-04-26T04:53:17.542 INFO:tasks.workunit.client.0.smithi077.stdout:1/777: write d9/d53/f6f [771828,100858] 0 2024-04-26T04:53:17.543 INFO:tasks.workunit.client.0.smithi077.stdout:8/817: chown d4/l29 15 1 2024-04-26T04:53:17.646 INFO:tasks.workunit.client.0.smithi077.stdout:1/778: fdatasync d9/db/d2b/d75/f97 0 2024-04-26T04:53:17.646 INFO:tasks.workunit.client.0.smithi077.stdout:2/856: dread d9/dc/f7a [0,4194304] 0 2024-04-26T04:53:17.647 INFO:tasks.workunit.client.0.smithi077.stdout:9/714: truncate d9/d3a/d79/d95/fb8 1496666 0 2024-04-26T04:53:17.650 INFO:tasks.workunit.client.0.smithi077.stdout:1/779: read - d9/db/d2b/d3f/fc2 zero size 2024-04-26T04:53:17.651 INFO:tasks.workunit.client.0.smithi077.stdout:8/818: creat d4/df/d22/d2d/d2b/d33/dd8/f110 x:0 0 0 2024-04-26T04:53:17.652 INFO:tasks.workunit.client.0.smithi077.stdout:4/804: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/f32 [4194304,4194304] 0 2024-04-26T04:53:17.654 INFO:tasks.workunit.client.0.smithi077.stdout:9/715: mkdir d9/d45/dc4/df3 0 2024-04-26T04:53:17.656 INFO:tasks.workunit.client.0.smithi077.stdout:4/805: unlink f2 0 2024-04-26T04:53:17.658 INFO:tasks.workunit.client.0.smithi077.stdout:8/819: creat d4/df/d22/d2d/d2b/d33/dd8/f111 x:0 0 0 2024-04-26T04:53:17.659 INFO:tasks.workunit.client.0.smithi077.stdout:9/716: dread d9/d45/f7e [0,4194304] 0 2024-04-26T04:53:17.659 INFO:tasks.workunit.client.0.smithi077.stdout:4/806: stat d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/la8 0 2024-04-26T04:53:17.659 INFO:tasks.workunit.client.0.smithi077.stdout:8/820: readlink d4/d17/l23 0 2024-04-26T04:53:17.660 INFO:tasks.workunit.client.0.smithi077.stdout:8/821: stat d4/d17/d90/ced 0 2024-04-26T04:53:17.662 INFO:tasks.workunit.client.0.smithi077.stdout:1/780: rename d9/db/d2b/d75/cf7 to d9/db/dd/d9c/dab/de8/df3/cfa 0 2024-04-26T04:53:17.664 INFO:tasks.workunit.client.0.smithi077.stdout:2/857: dread d9/fe [0,4194304] 0 2024-04-26T04:53:17.665 INFO:tasks.workunit.client.0.smithi077.stdout:2/858: write d9/dc/d1d/f47 [3327966,51877] 0 2024-04-26T04:53:17.669 INFO:tasks.workunit.client.0.smithi077.stdout:4/807: unlink d4/d17/d18/d43/d51/db4/d15/d1f/d76/lf5 0 2024-04-26T04:53:17.671 INFO:tasks.workunit.client.0.smithi077.stdout:2/859: read d9/dc/d1d/d34/fd7 [3980395,128120] 0 2024-04-26T04:53:17.672 INFO:tasks.workunit.client.0.smithi077.stdout:9/717: dread d9/dd/d1b/d2b/f2f [0,4194304] 0 2024-04-26T04:53:17.672 INFO:tasks.workunit.client.0.smithi077.stdout:8/822: symlink d4/d87/l112 0 2024-04-26T04:53:17.673 INFO:tasks.workunit.client.0.smithi077.stdout:8/823: fdatasync d4/df/d22/d80/ffd 0 2024-04-26T04:53:17.673 INFO:tasks.workunit.client.0.smithi077.stdout:1/781: symlink d9/db/dd/lfb 0 2024-04-26T04:53:17.679 INFO:tasks.workunit.client.0.smithi077.stdout:1/782: dwrite d9/d53/d8b/d93/f6d [0,4194304] 0 2024-04-26T04:53:17.688 INFO:tasks.workunit.client.0.smithi077.stdout:9/718: symlink d9/dd/d1b/d2b/ddb/lf4 0 2024-04-26T04:53:17.690 INFO:tasks.workunit.client.0.smithi077.stdout:9/719: write d9/f26 [5800478,63752] 0 2024-04-26T04:53:17.691 INFO:tasks.workunit.client.0.smithi077.stdout:9/720: write d9/d35/d99/fe0 [700253,90813] 0 2024-04-26T04:53:17.692 INFO:tasks.workunit.client.0.smithi077.stdout:8/824: symlink d4/df/d22/d80/d86/d103/l113 0 2024-04-26T04:53:17.703 INFO:tasks.workunit.client.0.smithi077.stdout:1/783: write d9/f21 [2173146,9340] 0 2024-04-26T04:53:17.704 INFO:tasks.workunit.client.0.smithi077.stdout:4/808: mkdir d4/d10e 0 2024-04-26T04:53:17.709 INFO:tasks.workunit.client.0.smithi077.stdout:2/860: unlink d9/dc/d1d/d34/d39/d63/d79/c88 0 2024-04-26T04:53:17.711 INFO:tasks.workunit.client.0.smithi077.stdout:9/721: dread d9/dd/d3d/f5a [0,4194304] 0 2024-04-26T04:53:17.712 INFO:tasks.workunit.client.0.smithi077.stdout:8/825: truncate d4/df/fd6 1152702 0 2024-04-26T04:53:17.713 INFO:tasks.workunit.client.0.smithi077.stdout:2/861: mknod d9/d15/d44/d4e/c129 0 2024-04-26T04:53:17.715 INFO:tasks.workunit.client.0.smithi077.stdout:9/722: fdatasync d9/d45/f7e 0 2024-04-26T04:53:17.719 INFO:tasks.workunit.client.0.smithi077.stdout:2/862: creat d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f12a x:0 0 0 2024-04-26T04:53:17.720 INFO:tasks.workunit.client.0.smithi077.stdout:2/863: truncate d9/dc/d1d/d34/df8/dd1/fdf 1221243 0 2024-04-26T04:53:17.721 INFO:tasks.workunit.client.0.smithi077.stdout:9/723: rmdir d9/d45/db1 39 2024-04-26T04:53:17.731 INFO:tasks.workunit.client.0.smithi077.stdout:9/724: dread d9/dd/d1b/d2b/ddb/f9a [0,4194304] 0 2024-04-26T04:53:17.732 INFO:tasks.workunit.client.0.smithi077.stdout:9/725: write d9/f62 [3913774,74036] 0 2024-04-26T04:53:17.758 INFO:tasks.workunit.client.0.smithi077.stdout:4/809: mkdir d4/d17/d18/d43/d51/db4/d15/d10f 0 2024-04-26T04:53:17.759 INFO:tasks.workunit.client.0.smithi077.stdout:8/826: write d4/f5 [3190161,5016] 0 2024-04-26T04:53:17.819 INFO:tasks.workunit.client.0.smithi077.stdout:2/864: mkdir d9/dc/d1d/d34/d39/d63/df6/d12b 0 2024-04-26T04:53:17.820 INFO:tasks.workunit.client.0.smithi077.stdout:2/865: truncate d9/dc/d5f/d8e/db9/fbc 1680712 0 2024-04-26T04:53:17.823 INFO:tasks.workunit.client.0.smithi077.stdout:9/726: write d9/dd/d1b/f66 [109722,91706] 0 2024-04-26T04:53:17.826 INFO:tasks.workunit.client.0.smithi077.stdout:2/866: creat d9/dc/d5f/d8e/da6/dbd/f12c x:0 0 0 2024-04-26T04:53:17.830 INFO:tasks.workunit.client.0.smithi077.stdout:2/867: mkdir d9/d15/d56/d12d 0 2024-04-26T04:53:17.832 INFO:tasks.workunit.client.0.smithi077.stdout:4/810: fdatasync d4/d17/d18/d43/d51/db4/d15/f6d 0 2024-04-26T04:53:17.833 INFO:tasks.workunit.client.0.smithi077.stdout:8/827: getdents d4/d17/ddc 0 2024-04-26T04:53:17.834 INFO:tasks.workunit.client.0.smithi077.stdout:8/828: write d4/f5 [3880043,13925] 0 2024-04-26T04:53:17.837 INFO:tasks.workunit.client.0.smithi077.stdout:9/727: write d9/d7d/fa1 [986513,100189] 0 2024-04-26T04:53:17.840 INFO:tasks.workunit.client.0.smithi077.stdout:9/728: truncate d9/d3a/d79/d95/feb 438356 0 2024-04-26T04:53:17.844 INFO:tasks.workunit.client.0.smithi077.stdout:4/811: symlink d4/d17/d18/l110 0 2024-04-26T04:53:17.846 INFO:tasks.workunit.client.0.smithi077.stdout:2/868: dread d9/d15/d44/fcc [0,4194304] 0 2024-04-26T04:53:17.849 INFO:tasks.workunit.client.0.smithi077.stdout:2/869: fsync d9/d15/f93 0 2024-04-26T04:53:17.852 INFO:tasks.workunit.client.0.smithi077.stdout:8/829: dwrite d4/df/f15 [4194304,4194304] 0 2024-04-26T04:53:17.859 INFO:tasks.workunit.client.0.smithi077.stdout:4/812: dread d4/d4e/f8d [0,4194304] 0 2024-04-26T04:53:17.863 INFO:tasks.workunit.client.0.smithi077.stdout:4/813: write d4/d17/d18/d43/f74 [4946031,21043] 0 2024-04-26T04:53:17.863 INFO:tasks.workunit.client.0.smithi077.stdout:9/729: dread d9/f26 [0,4194304] 0 2024-04-26T04:53:17.864 INFO:tasks.workunit.client.0.smithi077.stdout:8/830: dwrite d4/df/d22/d75/fcf [0,4194304] 0 2024-04-26T04:53:17.877 INFO:tasks.workunit.client.0.smithi077.stdout:2/870: dread d9/dc/d1d/d1e/f13 [0,4194304] 0 2024-04-26T04:53:17.880 INFO:tasks.workunit.client.0.smithi077.stdout:2/871: mknod d9/dc/d1d/d34/d39/d63/df6/d12b/c12e 0 2024-04-26T04:53:17.881 INFO:tasks.workunit.client.0.smithi077.stdout:9/730: write d9/d35/f91 [781282,24302] 0 2024-04-26T04:53:17.888 INFO:tasks.workunit.client.0.smithi077.stdout:2/872: dwrite d9/dc/f111 [0,4194304] 0 2024-04-26T04:53:17.888 INFO:tasks.workunit.client.0.smithi077.stdout:9/731: dread d9/dd/d1b/d2b/f83 [0,4194304] 0 2024-04-26T04:53:17.889 INFO:tasks.workunit.client.0.smithi077.stdout:9/732: chown d9/f82 119816361 1 2024-04-26T04:53:17.890 INFO:tasks.workunit.client.0.smithi077.stdout:2/873: chown d9/dc/d1d/c27 3836079 1 2024-04-26T04:53:17.894 INFO:tasks.workunit.client.0.smithi077.stdout:4/814: dread d4/d17/d18/d43/d6e/f86 [0,4194304] 0 2024-04-26T04:53:17.899 INFO:tasks.workunit.client.0.smithi077.stdout:9/733: getdents d9/dd/d1b/d2b 0 2024-04-26T04:53:17.901 INFO:tasks.workunit.client.0.smithi077.stdout:9/734: dread d9/dd/d1b/d2b/f83 [0,4194304] 0 2024-04-26T04:53:17.904 INFO:tasks.workunit.client.0.smithi077.stdout:4/815: dread d4/d17/d18/d43/d51/db4/d15/d1f/d35/fc1 [0,4194304] 0 2024-04-26T04:53:17.904 INFO:tasks.workunit.client.0.smithi077.stdout:2/874: write d9/dc/d1d/f4a [1317691,71394] 0 2024-04-26T04:53:17.905 INFO:tasks.workunit.client.0.smithi077.stdout:4/816: fsync d4/d17/d18/d43/d63/d80/fbd 0 2024-04-26T04:53:17.906 INFO:tasks.workunit.client.0.smithi077.stdout:4/817: fdatasync d4/d17/f7a 0 2024-04-26T04:53:17.907 INFO:tasks.workunit.client.0.smithi077.stdout:4/818: dread - d4/d17/d18/d43/d6e/d6c/fd6 zero size 2024-04-26T04:53:17.908 INFO:tasks.workunit.client.0.smithi077.stdout:2/875: symlink d9/d15/d56/l12f 0 2024-04-26T04:53:17.909 INFO:tasks.workunit.client.0.smithi077.stdout:2/876: write d9/dc/d1d/d1e/f3b [959056,13917] 0 2024-04-26T04:53:17.912 INFO:tasks.workunit.client.0.smithi077.stdout:9/735: getdents d9/dd/d1b/d2b/d32/d36 0 2024-04-26T04:53:17.915 INFO:tasks.workunit.client.0.smithi077.stdout:2/877: link d9/dc/d1d/d34/d39/d63/d79/dc2/ff1 d9/d15/d91/d106/f130 0 2024-04-26T04:53:17.917 INFO:tasks.workunit.client.0.smithi077.stdout:9/736: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/ff5 x:0 0 0 2024-04-26T04:53:17.918 INFO:tasks.workunit.client.0.smithi077.stdout:2/878: read d9/dc/d1d/d34/d39/d63/d79/f3f [2653861,105377] 0 2024-04-26T04:53:17.919 INFO:tasks.workunit.client.0.smithi077.stdout:4/819: link d4/d17/d18/d43/d51/db4/d15/l22 d4/d4e/dad/d8b/ddc/l111 0 2024-04-26T04:53:17.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/820: read - d4/d17/d18/d43/f101 zero size 2024-04-26T04:53:17.921 INFO:tasks.workunit.client.0.smithi077.stdout:4/821: truncate d4/d4e/dad/df8/f10b 873260 0 2024-04-26T04:53:17.924 INFO:tasks.workunit.client.0.smithi077.stdout:2/879: rename d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f114 to d9/dc/d5f/d8e/f131 0 2024-04-26T04:53:17.926 INFO:tasks.workunit.client.0.smithi077.stdout:4/822: creat d4/d17/d18/d43/d6e/d6c/dda/f112 x:0 0 0 2024-04-26T04:53:17.929 INFO:tasks.workunit.client.0.smithi077.stdout:2/880: creat d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/d10b/f132 x:0 0 0 2024-04-26T04:53:17.933 INFO:tasks.workunit.client.0.smithi077.stdout:4/823: dwrite d4/d17/d7f/feb [0,4194304] 0 2024-04-26T04:53:17.933 INFO:tasks.workunit.client.0.smithi077.stdout:2/881: write d9/dc/d1d/d34/d39/d63/d79/f3f [3252307,106721] 0 2024-04-26T04:53:17.934 INFO:tasks.workunit.client.0.smithi077.stdout:2/882: chown d9/dc/d1d/c27 1078627965 1 2024-04-26T04:53:17.941 INFO:tasks.workunit.client.0.smithi077.stdout:2/883: getdents d9/dc/d1d/d1e/d77/df9 0 2024-04-26T04:53:17.942 INFO:tasks.workunit.client.0.smithi077.stdout:2/884: write d9/dc/d1d/d34/ffc [762614,4622] 0 2024-04-26T04:53:17.945 INFO:tasks.workunit.client.0.smithi077.stdout:2/885: creat d9/dc/d1d/d1e/d77/d9d/f133 x:0 0 0 2024-04-26T04:53:17.946 INFO:tasks.workunit.client.0.smithi077.stdout:2/886: rmdir d9/dc/d1d/d34/dd4 39 2024-04-26T04:53:17.949 INFO:tasks.workunit.client.0.smithi077.stdout:2/887: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/db8/c134 0 2024-04-26T04:53:17.960 INFO:tasks.workunit.client.0.smithi077.stdout:2/888: dwrite d9/d15/fab [0,4194304] 0 2024-04-26T04:53:17.966 INFO:tasks.workunit.client.0.smithi077.stdout:2/889: dwrite d9/dc/d1d/d1e/f3b [0,4194304] 0 2024-04-26T04:53:17.966 INFO:tasks.workunit.client.0.smithi077.stdout:2/890: stat d9/d15/f37 0 2024-04-26T04:53:17.967 INFO:tasks.workunit.client.0.smithi077.stdout:2/891: chown f5 1508404389 1 2024-04-26T04:53:17.968 INFO:tasks.workunit.client.0.smithi077.stdout:2/892: fdatasync d9/dc/d1d/d34/df8/dd1/fdf 0 2024-04-26T04:53:17.972 INFO:tasks.workunit.client.0.smithi077.stdout:2/893: creat d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f135 x:0 0 0 2024-04-26T04:53:17.976 INFO:tasks.workunit.client.0.smithi077.stdout:2/894: rename d9/dc/d1d/d1e/c28 to d9/d15/d56/de6/c136 0 2024-04-26T04:53:17.982 INFO:tasks.workunit.client.0.smithi077.stdout:2/895: dwrite d9/d15/d91/f96 [4194304,4194304] 0 2024-04-26T04:53:17.984 INFO:tasks.workunit.client.0.smithi077.stdout:2/896: mkdir d9/d15/d56/d137 0 2024-04-26T04:53:17.986 INFO:tasks.workunit.client.0.smithi077.stdout:2/897: fsync d9/f10 0 2024-04-26T04:53:17.986 INFO:tasks.workunit.client.0.smithi077.stdout:2/898: stat d9/dc/d1d/d1e/d77/df9 0 2024-04-26T04:53:17.988 INFO:tasks.workunit.client.0.smithi077.stdout:2/899: write d9/dc/d1d/d34/f36 [567304,102876] 0 2024-04-26T04:53:17.999 INFO:tasks.workunit.client.0.smithi077.stdout:2/900: truncate d9/dc/d1d/f4a 2926245 0 2024-04-26T04:53:18.001 INFO:tasks.workunit.client.0.smithi077.stdout:2/901: symlink d9/dc/d1d/d34/d39/d63/d79/d5b/db8/l138 0 2024-04-26T04:53:18.004 INFO:tasks.workunit.client.0.smithi077.stdout:2/902: mkdir d9/dc/d1d/d34/dd4/d139 0 2024-04-26T04:53:18.008 INFO:tasks.workunit.client.0.smithi077.stdout:2/903: dread d9/dc/d1d/d34/df8/f46 [4194304,4194304] 0 2024-04-26T04:53:18.009 INFO:tasks.workunit.client.0.smithi077.stdout:2/904: read - d9/dc/d1d/d34/d39/d63/d79/dc2/ff1 zero size 2024-04-26T04:53:18.014 INFO:tasks.workunit.client.0.smithi077.stdout:2/905: unlink d9/dc/fef 0 2024-04-26T04:53:18.016 INFO:tasks.workunit.client.0.smithi077.stdout:2/906: write d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f135 [368794,74358] 0 2024-04-26T04:53:18.022 INFO:tasks.workunit.client.0.smithi077.stdout:2/907: link d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/fdd d9/dc/d1d/d122/f13a 0 2024-04-26T04:53:18.028 INFO:tasks.workunit.client.0.smithi077.stdout:2/908: creat d9/f13b x:0 0 0 2024-04-26T04:53:18.055 INFO:tasks.workunit.client.0.smithi077.stdout:2/909: link d9/f13b d9/d15/d44/d4e/f13c 0 2024-04-26T04:53:18.083 INFO:tasks.workunit.client.0.smithi077.stdout:2/910: dwrite d9/dc/d1d/fac [4194304,4194304] 0 2024-04-26T04:53:18.088 INFO:tasks.workunit.client.0.smithi077.stdout:2/911: rmdir d9/dc 39 2024-04-26T04:53:18.093 INFO:tasks.workunit.client.0.smithi077.stdout:2/912: truncate d9/dc/d1d/d34/f50 526983 0 2024-04-26T04:53:18.097 INFO:tasks.workunit.client.0.smithi077.stdout:2/913: creat d9/dc/d1d/f13d x:0 0 0 2024-04-26T04:53:18.098 INFO:tasks.workunit.client.0.smithi077.stdout:2/914: chown d9/dc/d1d/d34/c121 7 1 2024-04-26T04:53:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:17 smithi077 ceph-mon[26311]: pgmap v28: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 75 MiB/s rd, 183 MiB/s wr, 398 op/s 2024-04-26T04:53:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:17 smithi078 ceph-mon[33746]: pgmap v28: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 75 MiB/s rd, 183 MiB/s wr, 398 op/s 2024-04-26T04:53:18.275 INFO:tasks.workunit.client.0.smithi077.stdout:7/816: sync 2024-04-26T04:53:18.281 INFO:tasks.workunit.client.0.smithi077.stdout:7/817: write da/d6b/d63/f81 [1419161,36886] 0 2024-04-26T04:53:18.281 INFO:tasks.workunit.client.0.smithi077.stdout:7/818: stat da/f68 0 2024-04-26T04:53:18.288 INFO:tasks.workunit.client.0.smithi077.stdout:7/819: dwrite da/d13/d22/d5d/d9f/fdf [0,4194304] 0 2024-04-26T04:53:18.291 INFO:tasks.workunit.client.0.smithi077.stdout:7/820: creat da/d15/d42/f110 x:0 0 0 2024-04-26T04:53:18.293 INFO:tasks.workunit.client.0.smithi077.stdout:7/821: mkdir da/d15/d1a/d111 0 2024-04-26T04:53:18.295 INFO:tasks.workunit.client.0.smithi077.stdout:7/822: mkdir da/d13/d22/d5d/dbd/d112 0 2024-04-26T04:53:18.297 INFO:tasks.workunit.client.0.smithi077.stdout:7/823: read - da/d13/fa1 zero size 2024-04-26T04:53:18.301 INFO:tasks.workunit.client.0.smithi077.stdout:7/824: rename da/d13/d22/d5d/dfc to da/d13/d22/d7d/d80/d113 0 2024-04-26T04:53:18.302 INFO:tasks.workunit.client.0.smithi077.stdout:7/825: read - da/d13/d22/d5d/dbd/fdc zero size 2024-04-26T04:53:18.306 INFO:tasks.workunit.client.0.smithi077.stdout:7/826: rename da/d13/d22/d5d/d9f/da7/de0/ffb to da/d13/d22/d7d/d80/d113/f114 0 2024-04-26T04:53:18.307 INFO:tasks.workunit.client.0.smithi077.stdout:7/827: creat da/d15/dc6/f115 x:0 0 0 2024-04-26T04:53:18.308 INFO:tasks.workunit.client.0.smithi077.stdout:7/828: chown da/d15/d1a/d39/d96/fbf 344836896 1 2024-04-26T04:53:18.322 INFO:tasks.workunit.client.0.smithi077.stdout:7/829: dread da/d13/f19 [0,4194304] 0 2024-04-26T04:53:18.334 INFO:tasks.workunit.client.0.smithi077.stdout:7/830: write da/d13/d22/d5d/d9f/fc0 [167909,115262] 0 2024-04-26T04:53:18.338 INFO:tasks.workunit.client.0.smithi077.stdout:7/831: rename da/f89 to da/d13/d22/d5d/d9f/ddd/f116 0 2024-04-26T04:53:18.339 INFO:tasks.workunit.client.0.smithi077.stdout:7/832: write da/d15/d42/f77 [217512,32181] 0 2024-04-26T04:53:18.342 INFO:tasks.workunit.client.0.smithi077.stdout:7/833: unlink da/d15/d1a/dcc/f10d 0 2024-04-26T04:53:18.345 INFO:tasks.workunit.client.0.smithi077.stdout:7/834: mkdir da/d6b/d63/d64/d117 0 2024-04-26T04:53:18.349 INFO:tasks.workunit.client.0.smithi077.stdout:7/835: rename da/d13/d22/d7d/d80/d113/d104 to da/d13/d118 0 2024-04-26T04:53:18.350 INFO:tasks.workunit.client.0.smithi077.stdout:7/836: mkdir da/d13/d22/d7d/d80/d113/d119 0 2024-04-26T04:53:18.352 INFO:tasks.workunit.client.0.smithi077.stdout:7/837: symlink da/d13/d22/d34/l11a 0 2024-04-26T04:53:18.372 INFO:tasks.workunit.client.0.smithi077.stdout:7/838: dwrite da/d15/d1a/fb5 [0,4194304] 0 2024-04-26T04:53:18.380 INFO:tasks.workunit.client.0.smithi077.stdout:7/839: getdents da/d13/d22/d7d/dfd 0 2024-04-26T04:53:18.462 INFO:tasks.workunit.client.0.smithi077.stdout:6/835: sync 2024-04-26T04:53:18.465 INFO:tasks.workunit.client.0.smithi077.stdout:6/836: unlink d8/d52/c51 0 2024-04-26T04:53:18.466 INFO:tasks.workunit.client.0.smithi077.stdout:6/837: truncate d8/d52/d53/d5e/da6/de9/f116 154464 0 2024-04-26T04:53:18.470 INFO:tasks.workunit.client.0.smithi077.stdout:6/838: rename d8/d52/da0/dad/fbe to d8/d52/d53/da7/dfa/f12c 0 2024-04-26T04:53:18.484 INFO:tasks.workunit.client.0.smithi077.stdout:6/839: dread d8/d52/f4d [4194304,4194304] 0 2024-04-26T04:53:18.484 INFO:tasks.workunit.client.0.smithi077.stdout:6/840: chown d8/f22 29 1 2024-04-26T04:53:18.491 INFO:tasks.workunit.client.0.smithi077.stdout:6/841: dread - d8/d52/d53/d5e/da6/de6/fef zero size 2024-04-26T04:53:18.491 INFO:tasks.workunit.client.0.smithi077.stdout:6/842: fsync d8/d2a/d86/fe0 0 2024-04-26T04:53:18.492 INFO:tasks.workunit.client.0.smithi077.stdout:6/843: chown d8/d1c/c61 105 1 2024-04-26T04:53:18.496 INFO:tasks.workunit.client.0.smithi077.stdout:6/844: mknod d8/c12d 0 2024-04-26T04:53:18.499 INFO:tasks.workunit.client.0.smithi077.stdout:6/845: fsync d8/d52/da0/f7d 0 2024-04-26T04:53:18.504 INFO:tasks.workunit.client.0.smithi077.stdout:6/846: dwrite d8/f48 [0,4194304] 0 2024-04-26T04:53:18.518 INFO:tasks.workunit.client.0.smithi077.stdout:6/847: truncate d8/f50 2258852 0 2024-04-26T04:53:18.693 INFO:tasks.workunit.client.0.smithi077.stdout:3/724: sync 2024-04-26T04:53:18.699 INFO:tasks.workunit.client.0.smithi077.stdout:3/725: write d0/d7/d9a/db3/f22 [1686186,21112] 0 2024-04-26T04:53:18.788 INFO:tasks.workunit.client.0.smithi077.stdout:0/767: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:1/784: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:9/737: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:5/790: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:8/831: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:4/824: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:6/848: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:2/915: sync 2024-04-26T04:53:18.789 INFO:tasks.workunit.client.0.smithi077.stdout:7/840: sync 2024-04-26T04:53:18.791 INFO:tasks.workunit.client.0.smithi077.stdout:1/785: read - d9/d90/fe6 zero size 2024-04-26T04:53:18.791 INFO:tasks.workunit.client.0.smithi077.stdout:5/791: readlink d1/d30/d35/l54 0 2024-04-26T04:53:18.792 INFO:tasks.workunit.client.0.smithi077.stdout:9/738: creat d9/d7d/d94/ff6 x:0 0 0 2024-04-26T04:53:18.792 INFO:tasks.workunit.client.0.smithi077.stdout:2/916: write d9/dc/d1d/d34/ffc [162409,85796] 0 2024-04-26T04:53:18.794 INFO:tasks.workunit.client.0.smithi077.stdout:5/792: fsync d1/d12/d28/f9f 0 2024-04-26T04:53:18.797 INFO:tasks.workunit.client.0.smithi077.stdout:7/841: creat da/d13/d22/d5d/dbd/f11b x:0 0 0 2024-04-26T04:53:18.799 INFO:tasks.workunit.client.0.smithi077.stdout:0/768: dwrite d0/d47/dce/ff6 [0,4194304] 0 2024-04-26T04:53:18.800 INFO:tasks.workunit.client.0.smithi077.stdout:8/832: mkdir d4/d17/d90/dfe/d114 0 2024-04-26T04:53:18.801 INFO:tasks.workunit.client.0.smithi077.stdout:1/786: dread d9/d53/d6e/d7c/fcf [0,4194304] 0 2024-04-26T04:53:18.801 INFO:tasks.workunit.client.0.smithi077.stdout:0/769: chown d0/d47/f55 525132039 1 2024-04-26T04:53:18.812 INFO:tasks.workunit.client.0.smithi077.stdout:5/793: fsync d1/d42/d62/f6f 0 2024-04-26T04:53:18.817 INFO:tasks.workunit.client.0.smithi077.stdout:7/842: rmdir da/d15/d42 39 2024-04-26T04:53:18.820 INFO:tasks.workunit.client.0.smithi077.stdout:2/917: dwrite d9/f10 [0,4194304] 0 2024-04-26T04:53:18.822 INFO:tasks.workunit.client.0.smithi077.stdout:8/833: truncate d4/df/d22/f50 1655735 0 2024-04-26T04:53:18.822 INFO:tasks.workunit.client.0.smithi077.stdout:8/834: chown d4/l95 0 1 2024-04-26T04:53:18.832 INFO:tasks.workunit.client.0.smithi077.stdout:6/849: getdents d8/d8f 0 2024-04-26T04:53:18.840 INFO:tasks.workunit.client.0.smithi077.stdout:1/787: dwrite d9/db/f39 [0,4194304] 0 2024-04-26T04:53:18.847 INFO:tasks.workunit.client.0.smithi077.stdout:4/825: truncate d4/d17/d18/d43/d51/db4/d15/d1f/d25/fcb 645924 0 2024-04-26T04:53:18.847 INFO:tasks.workunit.client.0.smithi077.stdout:3/726: sync 2024-04-26T04:53:18.849 INFO:tasks.workunit.client.0.smithi077.stdout:0/770: dread d0/d47/f55 [0,4194304] 0 2024-04-26T04:53:18.849 INFO:tasks.workunit.client.0.smithi077.stdout:4/826: fsync d4/d17/d18/d43/d51/db4/d15/d1f/fd3 0 2024-04-26T04:53:18.850 INFO:tasks.workunit.client.0.smithi077.stdout:6/850: rmdir d8/d2a/d86 39 2024-04-26T04:53:18.850 INFO:tasks.workunit.client.0.smithi077.stdout:4/827: stat d4/d17/d18/fa7 0 2024-04-26T04:53:18.851 INFO:tasks.workunit.client.0.smithi077.stdout:8/835: write d4/df/dce/f70 [1350974,58100] 0 2024-04-26T04:53:18.853 INFO:tasks.workunit.client.0.smithi077.stdout:5/794: symlink d1/d30/d33/d109/l10f 0 2024-04-26T04:53:18.853 INFO:tasks.workunit.client.0.smithi077.stdout:4/828: read d4/d17/d18/d43/d51/db4/d15/d1f/f34 [1167578,94721] 0 2024-04-26T04:53:18.855 INFO:tasks.workunit.client.0.smithi077.stdout:7/843: symlink da/d13/d22/d7d/d80/d113/d119/l11c 0 2024-04-26T04:53:18.855 INFO:tasks.workunit.client.0.smithi077.stdout:7/844: chown da/f59 6 1 2024-04-26T04:53:18.856 INFO:tasks.workunit.client.0.smithi077.stdout:1/788: symlink d9/d53/dc1/lfc 0 2024-04-26T04:53:18.858 INFO:tasks.workunit.client.0.smithi077.stdout:1/789: read d9/f73 [3042850,49206] 0 2024-04-26T04:53:18.859 INFO:tasks.workunit.client.0.smithi077.stdout:1/790: stat d9/d90/d99 0 2024-04-26T04:53:18.862 INFO:tasks.workunit.client.0.smithi077.stdout:6/851: symlink d8/d52/l12e 0 2024-04-26T04:53:18.864 INFO:tasks.workunit.client.0.smithi077.stdout:9/739: getdents d9/dd/def 0 2024-04-26T04:53:18.866 INFO:tasks.workunit.client.0.smithi077.stdout:1/791: dwrite d9/f7a [0,4194304] 0 2024-04-26T04:53:18.867 INFO:tasks.workunit.client.0.smithi077.stdout:1/792: chown d9/db/dd/d68/d81/fb5 0 1 2024-04-26T04:53:18.868 INFO:tasks.workunit.client.0.smithi077.stdout:8/836: symlink d4/df/d6f/da3/dc7/l115 0 2024-04-26T04:53:18.870 INFO:tasks.workunit.client.0.smithi077.stdout:4/829: creat d4/d4e/dad/df8/f113 x:0 0 0 2024-04-26T04:53:18.871 INFO:tasks.workunit.client.0.smithi077.stdout:7/845: creat da/d13/d118/f11d x:0 0 0 2024-04-26T04:53:18.877 INFO:tasks.workunit.client.0.smithi077.stdout:3/727: rename d0/d7/d1f/d4d/dc6/de3 to d0/d9/d32/d29/db8/dec 0 2024-04-26T04:53:18.878 INFO:tasks.workunit.client.0.smithi077.stdout:6/852: rmdir d8/d52/da0/dad 39 2024-04-26T04:53:18.879 INFO:tasks.workunit.client.0.smithi077.stdout:7/846: dwrite da/d15/d1a/dcc/ff0 [0,4194304] 0 2024-04-26T04:53:18.879 INFO:tasks.workunit.client.0.smithi077.stdout:3/728: write d0/d9/f84 [2939244,1249] 0 2024-04-26T04:53:18.888 INFO:tasks.workunit.client.0.smithi077.stdout:1/793: truncate d9/d53/d8b/fa2 422548 0 2024-04-26T04:53:18.889 INFO:tasks.workunit.client.0.smithi077.stdout:3/729: dwrite d0/d7/fd5 [0,4194304] 0 2024-04-26T04:53:18.892 INFO:tasks.workunit.client.0.smithi077.stdout:2/918: getdents d9/dc/d1d/d34/df8/dd1/dc3 0 2024-04-26T04:53:18.892 INFO:tasks.workunit.client.0.smithi077.stdout:2/919: fdatasync d9/fe 0 2024-04-26T04:53:18.893 INFO:tasks.workunit.client.0.smithi077.stdout:2/920: chown d9/dc/d5f/d78/l80 3 1 2024-04-26T04:53:18.900 INFO:tasks.workunit.client.0.smithi077.stdout:0/771: rename d0/d47/dce to d0/d47/d10a 0 2024-04-26T04:53:18.910 INFO:tasks.workunit.client.0.smithi077.stdout:0/772: creat d0/d2c/d109/f10b x:0 0 0 2024-04-26T04:53:18.911 INFO:tasks.workunit.client.0.smithi077.stdout:1/794: creat d9/d90/df6/ffd x:0 0 0 2024-04-26T04:53:18.911 INFO:tasks.workunit.client.0.smithi077.stdout:3/730: dread d0/d9/d32/d29/f4b [0,4194304] 0 2024-04-26T04:53:18.913 INFO:tasks.workunit.client.0.smithi077.stdout:4/830: rename d4/d4e/dad/d8b/ddc/cf9 to d4/c114 0 2024-04-26T04:53:18.914 INFO:tasks.workunit.client.0.smithi077.stdout:0/773: dread d0/d47/d93/d94/fcd [0,4194304] 0 2024-04-26T04:53:18.916 INFO:tasks.workunit.client.0.smithi077.stdout:2/921: dwrite d9/dc/d5f/d8e/db9/fe5 [0,4194304] 0 2024-04-26T04:53:18.918 INFO:tasks.workunit.client.0.smithi077.stdout:1/795: unlink d9/db/dd/d34/d63/l94 0 2024-04-26T04:53:18.920 INFO:tasks.workunit.client.0.smithi077.stdout:1/796: fsync d9/d90/fad 0 2024-04-26T04:53:18.922 INFO:tasks.workunit.client.0.smithi077.stdout:8/837: dwrite d4/df/d22/d2d/d2b/d33/f39 [0,4194304] 0 2024-04-26T04:53:18.925 INFO:tasks.workunit.client.0.smithi077.stdout:4/831: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d35/d115 0 2024-04-26T04:53:18.928 INFO:tasks.workunit.client.0.smithi077.stdout:1/797: truncate d9/d90/d99/d9a/dd6/fe9 428898 0 2024-04-26T04:53:18.929 INFO:tasks.workunit.client.0.smithi077.stdout:7/847: rename da/d6b/fc3 to da/d15/f11e 0 2024-04-26T04:53:18.932 INFO:tasks.workunit.client.0.smithi077.stdout:6/853: dread d8/d52/d53/d5e/f69 [4194304,4194304] 0 2024-04-26T04:53:18.933 INFO:tasks.workunit.client.0.smithi077.stdout:7/848: chown da/ce5 9920964 1 2024-04-26T04:53:18.937 INFO:tasks.workunit.client.0.smithi077.stdout:4/832: unlink d4/d17/d18/d43/d51/db4/d15/d1f/dd8/lfe 0 2024-04-26T04:53:18.937 INFO:tasks.workunit.client.0.smithi077.stdout:9/740: dwrite d9/d7d/d94/fc3 [0,4194304] 0 2024-04-26T04:53:18.942 INFO:tasks.workunit.client.0.smithi077.stdout:3/731: rename d0/d9/d32 to d0/dae/ded 0 2024-04-26T04:53:18.947 INFO:tasks.workunit.client.0.smithi077.stdout:4/833: write d4/d17/d18/d43/d51/db4/d15/ffa [96414,91240] 0 2024-04-26T04:53:18.950 INFO:tasks.workunit.client.0.smithi077.stdout:8/838: creat d4/df/d22/d80/d86/de3/f116 x:0 0 0 2024-04-26T04:53:18.950 INFO:tasks.workunit.client.0.smithi077.stdout:9/741: dread - d9/dd/d3d/d7c/dc8/fcc zero size 2024-04-26T04:53:18.954 INFO:tasks.workunit.client.0.smithi077.stdout:0/774: rename d0/ff4 to d0/d47/d64/d97/dba/dc4/f10c 0 2024-04-26T04:53:18.955 INFO:tasks.workunit.client.0.smithi077.stdout:0/775: fdatasync d0/d19/d69/fe0 0 2024-04-26T04:53:18.959 INFO:tasks.workunit.client.0.smithi077.stdout:7/849: dread da/d15/f32 [0,4194304] 0 2024-04-26T04:53:18.962 INFO:tasks.workunit.client.0.smithi077.stdout:2/922: dwrite d9/d15/f31 [0,4194304] 0 2024-04-26T04:53:18.969 INFO:tasks.workunit.client.0.smithi077.stdout:1/798: rename c8 to d9/d90/d99/cfe 0 2024-04-26T04:53:18.970 INFO:tasks.workunit.client.0.smithi077.stdout:6/854: write d8/f17 [4555177,112950] 0 2024-04-26T04:53:18.976 INFO:tasks.workunit.client.0.smithi077.stdout:3/732: write d0/dae/ded/f35 [1666570,14996] 0 2024-04-26T04:53:18.978 INFO:tasks.workunit.client.0.smithi077.stdout:3/733: chown d0/d9/f8b 4885770 1 2024-04-26T04:53:18.978 INFO:tasks.workunit.client.0.smithi077.stdout:4/834: symlink d4/d4e/dad/d8b/dba/de5/l116 0 2024-04-26T04:53:18.979 INFO:tasks.workunit.client.0.smithi077.stdout:4/835: readlink d4/d4e/dad/d8b/dba/de5/l116 0 2024-04-26T04:53:18.981 INFO:tasks.workunit.client.0.smithi077.stdout:2/923: symlink d9/dc/d1d/d1e/d77/d9d/l13e 0 2024-04-26T04:53:18.990 INFO:tasks.workunit.client.0.smithi077.stdout:0/776: dwrite d0/d47/d93/d94/f9b [0,4194304] 0 2024-04-26T04:53:18.991 INFO:tasks.workunit.client.0.smithi077.stdout:9/742: truncate d9/dd/d3d/d7c/dc8/fda 466429 0 2024-04-26T04:53:18.991 INFO:tasks.workunit.client.0.smithi077.stdout:0/777: readlink d0/d19/d1e/d2f/da8/lb4 0 2024-04-26T04:53:18.994 INFO:tasks.workunit.client.0.smithi077.stdout:2/924: symlink d9/dc/d1d/d1e/d77/d9d/l13f 0 2024-04-26T04:53:18.996 INFO:tasks.workunit.client.0.smithi077.stdout:1/799: mkdir d9/db/dff 0 2024-04-26T04:53:18.996 INFO:tasks.workunit.client.0.smithi077.stdout:7/850: dread da/d13/d22/d34/f51 [0,4194304] 0 2024-04-26T04:53:18.996 INFO:tasks.workunit.client.0.smithi077.stdout:4/836: read d4/d17/d18/d43/d51/db4/f2b [2423114,102330] 0 2024-04-26T04:53:18.998 INFO:tasks.workunit.client.0.smithi077.stdout:6/855: creat d8/d52/d53/da7/df4/f12f x:0 0 0 2024-04-26T04:53:19.001 INFO:tasks.workunit.client.0.smithi077.stdout:7/851: chown da/d13/d22/d5d/d9f 0 1 2024-04-26T04:53:19.001 INFO:tasks.workunit.client.0.smithi077.stdout:3/734: dwrite d0/dae/ded/d29/f4b [0,4194304] 0 2024-04-26T04:53:19.002 INFO:tasks.workunit.client.0.smithi077.stdout:9/743: creat d9/d45/db1/ff7 x:0 0 0 2024-04-26T04:53:19.009 INFO:tasks.workunit.client.0.smithi077.stdout:7/852: creat da/d6b/d63/d64/f11f x:0 0 0 2024-04-26T04:53:19.011 INFO:tasks.workunit.client.0.smithi077.stdout:6/856: truncate d8/f88 901299 0 2024-04-26T04:53:19.015 INFO:tasks.workunit.client.0.smithi077.stdout:0/778: rename d0/d28/l33 to d0/d19/d1e/l10d 0 2024-04-26T04:53:19.018 INFO:tasks.workunit.client.0.smithi077.stdout:9/744: fsync d9/dd/d1b/d2b/d32/f39 0 2024-04-26T04:53:19.019 INFO:tasks.workunit.client.0.smithi077.stdout:3/735: mknod d0/d7/d9a/cee 0 2024-04-26T04:53:19.022 INFO:tasks.workunit.client.0.smithi077.stdout:4/837: creat d4/f117 x:0 0 0 2024-04-26T04:53:19.031 INFO:tasks.workunit.client.0.smithi077.stdout:0/779: creat d0/d19/d1e/d2f/d52/f10e x:0 0 0 2024-04-26T04:53:19.035 INFO:tasks.workunit.client.0.smithi077.stdout:9/745: unlink d9/dd/d1b/f23 0 2024-04-26T04:53:19.037 INFO:tasks.workunit.client.0.smithi077.stdout:3/736: symlink d0/d7/d9a/db7/lef 0 2024-04-26T04:53:19.038 INFO:tasks.workunit.client.0.smithi077.stdout:0/780: dwrite d0/d19/d1e/d59/d96/d106/f103 [0,4194304] 0 2024-04-26T04:53:19.042 INFO:tasks.workunit.client.0.smithi077.stdout:4/838: creat d4/d17/d18/d43/d51/db4/d15/d1f/d35/f118 x:0 0 0 2024-04-26T04:53:19.042 INFO:tasks.workunit.client.0.smithi077.stdout:7/853: link da/ff da/d13/d22/d5d/d9f/ddd/f120 0 2024-04-26T04:53:19.044 INFO:tasks.workunit.client.0.smithi077.stdout:1/800: link d9/db/dd/d34/d63/f76 d9/db/dd/d34/d4a/f100 0 2024-04-26T04:53:19.045 INFO:tasks.workunit.client.0.smithi077.stdout:0/781: dread d0/d19/d1e/d59/d96/d106/f103 [0,4194304] 0 2024-04-26T04:53:19.174 INFO:tasks.workunit.client.0.smithi077.stdout:9/746: fdatasync d9/d45/f7e 0 2024-04-26T04:53:19.175 INFO:tasks.workunit.client.0.smithi077.stdout:2/925: getdents d9/dc/d1d/d34/d39/d63/de9/d11e 0 2024-04-26T04:53:19.176 INFO:tasks.workunit.client.0.smithi077.stdout:2/926: fdatasync d9/dc/d1d/fac 0 2024-04-26T04:53:19.182 INFO:tasks.workunit.client.0.smithi077.stdout:1/801: mkdir d9/d53/d8b/d93/da8/dd7/d101 0 2024-04-26T04:53:19.183 INFO:tasks.workunit.client.0.smithi077.stdout:1/802: write d9/fec [602761,117811] 0 2024-04-26T04:53:19.187 INFO:tasks.workunit.client.0.smithi077.stdout:9/747: symlink d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/dd4/lf8 0 2024-04-26T04:53:19.191 INFO:tasks.workunit.client.0.smithi077.stdout:2/927: mkdir d9/d15/d91/d106/d140 0 2024-04-26T04:53:19.203 INFO:tasks.workunit.client.0.smithi077.stdout:9/748: mkdir d9/d7d/df9 0 2024-04-26T04:53:19.207 INFO:tasks.workunit.client.0.smithi077.stdout:4/839: getdents d4/d17/d18/d43/d6e/d6c 0 2024-04-26T04:53:19.208 INFO:tasks.workunit.client.0.smithi077.stdout:0/782: getdents d0/d19/d8d 0 2024-04-26T04:53:19.232 INFO:tasks.workunit.client.0.smithi077.stdout:8/839: sync 2024-04-26T04:53:19.232 INFO:tasks.workunit.client.0.smithi077.stdout:5/795: sync 2024-04-26T04:53:19.233 INFO:tasks.workunit.client.0.smithi077.stdout:5/796: write d1/d42/d62/dc8/fdc [2962816,22645] 0 2024-04-26T04:53:19.236 INFO:tasks.workunit.client.0.smithi077.stdout:8/840: chown d4/df/d22/d80/d86/dd0/lff 196 1 2024-04-26T04:53:19.237 INFO:tasks.workunit.client.0.smithi077.stdout:5/797: unlink d1/d42/da4/d25/d31/d3c/l56 0 2024-04-26T04:53:19.238 INFO:tasks.workunit.client.0.smithi077.stdout:8/841: mkdir d4/df/d22/d2d/d2b/d33/dd8/d117 0 2024-04-26T04:53:19.239 INFO:tasks.workunit.client.0.smithi077.stdout:8/842: chown d4/df/d18/f74 20897 1 2024-04-26T04:53:19.242 INFO:tasks.workunit.client.0.smithi077.stdout:8/843: fdatasync d4/df/d6f/da3/dc7/ffa 0 2024-04-26T04:53:19.243 INFO:tasks.workunit.client.0.smithi077.stdout:5/798: dread d1/d30/d33/fe4 [0,4194304] 0 2024-04-26T04:53:19.245 INFO:tasks.workunit.client.0.smithi077.stdout:8/844: getdents d4/d17/ddc 0 2024-04-26T04:53:19.246 INFO:tasks.workunit.client.0.smithi077.stdout:8/845: dread - d4/d17/ddc/f10d zero size 2024-04-26T04:53:19.246 INFO:tasks.workunit.client.0.smithi077.stdout:5/799: fdatasync d1/d42/da4/d25/d31/f92 0 2024-04-26T04:53:19.247 INFO:tasks.workunit.client.0.smithi077.stdout:5/800: readlink d1/d8/l2f 0 2024-04-26T04:53:19.290 INFO:tasks.workunit.client.0.smithi077.stdout:7/854: sync 2024-04-26T04:53:19.290 INFO:tasks.workunit.client.0.smithi077.stdout:6/857: sync 2024-04-26T04:53:19.293 INFO:tasks.workunit.client.0.smithi077.stdout:7/855: read da/d13/d22/d7d/f85 [1733778,35405] 0 2024-04-26T04:53:19.294 INFO:tasks.workunit.client.0.smithi077.stdout:6/858: creat d8/d52/d53/d5e/d6c/d102/f130 x:0 0 0 2024-04-26T04:53:19.297 INFO:tasks.workunit.client.0.smithi077.stdout:6/859: rename d8/d1c/f118 to d8/d2a/d10b/f131 0 2024-04-26T04:53:19.299 INFO:tasks.workunit.client.0.smithi077.stdout:7/856: link da/d13/d22/d5d/dbd/fdc da/d13/d22/d5d/d9f/ddd/f121 0 2024-04-26T04:53:19.304 INFO:tasks.workunit.client.0.smithi077.stdout:3/737: write d0/d7/d1f/f60 [1155789,129040] 0 2024-04-26T04:53:19.307 INFO:tasks.workunit.client.0.smithi077.stdout:3/738: truncate d0/d9/f2b 1932175 0 2024-04-26T04:53:19.312 INFO:tasks.workunit.client.0.smithi077.stdout:3/739: getdents d0/d7/d1f/d81 0 2024-04-26T04:53:19.315 INFO:tasks.workunit.client.0.smithi077.stdout:3/740: rename d0/dae/ded/d29/fd2 to d0/d7/d1f/d4d/dbe/ff0 0 2024-04-26T04:53:19.315 INFO:tasks.workunit.client.0.smithi077.stdout:8/846: rmdir d4/df/d22/d75 39 2024-04-26T04:53:19.318 INFO:tasks.workunit.client.0.smithi077.stdout:3/741: rename d0/dae/ded/d29/ca7 to d0/d7/d9a/cf1 0 2024-04-26T04:53:19.320 INFO:tasks.workunit.client.0.smithi077.stdout:8/847: mknod d4/d17/d90/dfe/d114/c118 0 2024-04-26T04:53:19.326 INFO:tasks.workunit.client.0.smithi077.stdout:4/840: write d4/d17/d18/fd5 [375545,55833] 0 2024-04-26T04:53:19.328 INFO:tasks.workunit.client.0.smithi077.stdout:0/783: write d0/d47/d93/d94/faa [1807648,66944] 0 2024-04-26T04:53:19.334 INFO:tasks.workunit.client.0.smithi077.stdout:3/742: rmdir d0/d7/d1f/d4d 39 2024-04-26T04:53:19.335 INFO:tasks.workunit.client.0.smithi077.stdout:0/784: dwrite d0/d47/d64/d97/ffb [0,4194304] 0 2024-04-26T04:53:19.336 INFO:tasks.workunit.client.0.smithi077.stdout:0/785: write d0/d47/d64/dfe/f104 [172312,92204] 0 2024-04-26T04:53:19.338 INFO:tasks.workunit.client.0.smithi077.stdout:3/743: creat d0/d7/dce/ff2 x:0 0 0 2024-04-26T04:53:19.339 INFO:tasks.workunit.client.0.smithi077.stdout:2/928: creat d9/dc/d1d/d34/df8/dd1/dc3/f141 x:0 0 0 2024-04-26T04:53:19.347 INFO:tasks.workunit.client.0.smithi077.stdout:0/786: getdents d0/d19/d1e/d2f/d52 0 2024-04-26T04:53:19.347 INFO:tasks.workunit.client.0.smithi077.stdout:4/841: dread d4/d17/d18/d43/d63/fb3 [0,4194304] 0 2024-04-26T04:53:19.352 INFO:tasks.workunit.client.0.smithi077.stdout:4/842: rename d4/d17/d18/d43/d51/db4/d15/d1f/dd8/le6 to d4/d10e/l119 0 2024-04-26T04:53:19.353 INFO:tasks.workunit.client.0.smithi077.stdout:6/860: creat d8/f132 x:0 0 0 2024-04-26T04:53:19.354 INFO:tasks.workunit.client.0.smithi077.stdout:4/843: chown d4/d4e/dad/d8b/ddc/dec 56561466 1 2024-04-26T04:53:19.354 INFO:tasks.workunit.client.0.smithi077.stdout:0/787: dwrite d0/d19/f98 [0,4194304] 0 2024-04-26T04:53:19.355 INFO:tasks.workunit.client.0.smithi077.stdout:0/788: chown d0/d28/le1 3 1 2024-04-26T04:53:19.357 INFO:tasks.workunit.client.0.smithi077.stdout:1/803: unlink d9/f1b 0 2024-04-26T04:53:19.359 INFO:tasks.workunit.client.0.smithi077.stdout:5/801: write d1/d8/ff [946882,106367] 0 2024-04-26T04:53:19.360 INFO:tasks.workunit.client.0.smithi077.stdout:0/789: fdatasync d0/d19/d1e/d2f/d52/f76 0 2024-04-26T04:53:19.363 INFO:tasks.workunit.client.0.smithi077.stdout:5/802: rmdir d1/d42/d62/dc8 39 2024-04-26T04:53:19.364 INFO:tasks.workunit.client.0.smithi077.stdout:1/804: dwrite d9/db/fd1 [0,4194304] 0 2024-04-26T04:53:19.366 INFO:tasks.workunit.client.0.smithi077.stdout:0/790: dread d0/d47/d68/fe5 [0,4194304] 0 2024-04-26T04:53:19.369 INFO:tasks.workunit.client.0.smithi077.stdout:1/805: creat d9/d53/d8b/d93/da8/f102 x:0 0 0 2024-04-26T04:53:19.370 INFO:tasks.workunit.client.0.smithi077.stdout:1/806: write d9/db/f4c [3038660,129229] 0 2024-04-26T04:53:19.371 INFO:tasks.workunit.client.0.smithi077.stdout:6/861: mkdir d8/d52/d53/d5e/da6/d133 0 2024-04-26T04:53:19.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/791: dwrite d0/d47/d64/d97/dba/fca [4194304,4194304] 0 2024-04-26T04:53:19.380 INFO:tasks.workunit.client.0.smithi077.stdout:1/807: mknod d9/d53/d6e/c103 0 2024-04-26T04:53:19.382 INFO:tasks.workunit.client.0.smithi077.stdout:1/808: truncate d9/d53/d6e/d7c/fe1 688024 0 2024-04-26T04:53:19.384 INFO:tasks.workunit.client.0.smithi077.stdout:9/749: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dfa 0 2024-04-26T04:53:19.384 INFO:tasks.workunit.client.0.smithi077.stdout:0/792: rmdir d0/d47/d93 39 2024-04-26T04:53:19.388 INFO:tasks.workunit.client.0.smithi077.stdout:4/844: mknod d4/d4e/dad/d8b/c11a 0 2024-04-26T04:53:19.388 INFO:tasks.workunit.client.0.smithi077.stdout:5/803: dread d1/d12/f2a [0,4194304] 0 2024-04-26T04:53:19.390 INFO:tasks.workunit.client.0.smithi077.stdout:9/750: rmdir d9/dd/d1b/d2b 39 2024-04-26T04:53:19.392 INFO:tasks.workunit.client.0.smithi077.stdout:6/862: rmdir d8/d1c/d108/d12a 0 2024-04-26T04:53:19.395 INFO:tasks.workunit.client.0.smithi077.stdout:4/845: rename d4/d17/d18/d43/d53/d64/f81 to d4/d17/d18/d43/d53/d64/f11b 0 2024-04-26T04:53:19.396 INFO:tasks.workunit.client.0.smithi077.stdout:9/751: rmdir d9/d7d 39 2024-04-26T04:53:19.398 INFO:tasks.workunit.client.0.smithi077.stdout:6/863: dwrite d8/d2a/fec [0,4194304] 0 2024-04-26T04:53:19.403 INFO:tasks.workunit.client.0.smithi077.stdout:4/846: creat d4/d17/d18/d43/d6e/d6c/dae/dc5/f11c x:0 0 0 2024-04-26T04:53:19.404 INFO:tasks.workunit.client.0.smithi077.stdout:4/847: dread - d4/d17/d18/d43/d6e/d6c/fd6 zero size 2024-04-26T04:53:19.406 INFO:tasks.workunit.client.0.smithi077.stdout:9/752: chown d9/dd/d1b/d2b/d32/d36/fb6 2 1 2024-04-26T04:53:19.409 INFO:tasks.workunit.client.0.smithi077.stdout:5/804: getdents d1/d8c 0 2024-04-26T04:53:19.414 INFO:tasks.workunit.client.0.smithi077.stdout:6/864: creat d8/d2a/f134 x:0 0 0 2024-04-26T04:53:19.415 INFO:tasks.workunit.client.0.smithi077.stdout:6/865: stat d8/d2a/ddd/fe3 0 2024-04-26T04:53:19.417 INFO:tasks.workunit.client.0.smithi077.stdout:4/848: getdents d4/d4e/dad 0 2024-04-26T04:53:19.420 INFO:tasks.workunit.client.0.smithi077.stdout:9/753: link d9/c81 d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/cfb 0 2024-04-26T04:53:19.421 INFO:tasks.workunit.client.0.smithi077.stdout:9/754: dread - d9/d3a/fd6 zero size 2024-04-26T04:53:19.421 INFO:tasks.workunit.client.0.smithi077.stdout:9/755: chown d9/dd/d3d/d7c/c97 371661 1 2024-04-26T04:53:19.422 INFO:tasks.workunit.client.0.smithi077.stdout:4/849: unlink d4/d17/d18/d43/d51/db4/d15/d1f/d25/c68 0 2024-04-26T04:53:19.426 INFO:tasks.workunit.client.0.smithi077.stdout:9/756: dread - d9/dd/d1b/d2b/d32/d36/d7f/dae/fc9 zero size 2024-04-26T04:53:19.427 INFO:tasks.workunit.client.0.smithi077.stdout:4/850: creat d4/d4e/dad/d8b/dba/f11d x:0 0 0 2024-04-26T04:53:19.430 INFO:tasks.workunit.client.0.smithi077.stdout:4/851: unlink d4/d17/f96 0 2024-04-26T04:53:19.436 INFO:tasks.workunit.client.0.smithi077.stdout:8/848: dwrite d4/df/d22/d80/fae [0,4194304] 0 2024-04-26T04:53:19.440 INFO:tasks.workunit.client.0.smithi077.stdout:8/849: unlink d4/df/d22/c3b 0 2024-04-26T04:53:19.441 INFO:tasks.workunit.client.0.smithi077.stdout:2/929: write d9/dc/d1d/d34/d39/d63/d79/fd6 [984516,77949] 0 2024-04-26T04:53:19.442 INFO:tasks.workunit.client.0.smithi077.stdout:8/850: read - d4/df/dce/d5b/d8d/ff9 zero size 2024-04-26T04:53:19.447 INFO:tasks.workunit.client.0.smithi077.stdout:4/852: dread d4/d17/d18/d43/d53/f79 [0,4194304] 0 2024-04-26T04:53:19.450 INFO:tasks.workunit.client.0.smithi077.stdout:8/851: dwrite d4/df/d22/d80/d86/d103/f10a [0,4194304] 0 2024-04-26T04:53:19.457 INFO:tasks.workunit.client.0.smithi077.stdout:4/853: symlink d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/l11e 0 2024-04-26T04:53:19.459 INFO:tasks.workunit.client.0.smithi077.stdout:4/854: fdatasync d4/d17/d18/d43/d6e/d6c/fd6 0 2024-04-26T04:53:19.460 INFO:tasks.workunit.client.0.smithi077.stdout:1/809: write d9/db/dd/f2f [3615136,26810] 0 2024-04-26T04:53:19.464 INFO:tasks.workunit.client.0.smithi077.stdout:8/852: creat d4/df/d22/d75/f119 x:0 0 0 2024-04-26T04:53:19.465 INFO:tasks.workunit.client.0.smithi077.stdout:4/855: stat d4/d4e/dad/d8b/ddc/l111 0 2024-04-26T04:53:19.469 INFO:tasks.workunit.client.0.smithi077.stdout:8/853: creat d4/df/dce/d5b/f11a x:0 0 0 2024-04-26T04:53:19.469 INFO:tasks.workunit.client.0.smithi077.stdout:1/810: dread d9/f44 [0,4194304] 0 2024-04-26T04:53:19.470 INFO:tasks.workunit.client.0.smithi077.stdout:2/930: dread d9/dc/d1d/fac [0,4194304] 0 2024-04-26T04:53:19.479 INFO:tasks.workunit.client.0.smithi077.stdout:8/854: dread d4/df/d22/d80/d86/fe8 [0,4194304] 0 2024-04-26T04:53:19.482 INFO:tasks.workunit.client.0.smithi077.stdout:2/931: dwrite d9/dc/d1d/d34/d39/d63/d79/f128 [0,4194304] 0 2024-04-26T04:53:19.483 INFO:tasks.workunit.client.0.smithi077.stdout:8/855: chown d4/df/d22/f43 155477 1 2024-04-26T04:53:19.483 INFO:tasks.workunit.client.0.smithi077.stdout:2/932: fsync d9/dc/f123 0 2024-04-26T04:53:19.486 INFO:tasks.workunit.client.0.smithi077.stdout:8/856: chown d4/df/d22/d2d/f94 978320 1 2024-04-26T04:53:19.488 INFO:tasks.workunit.client.0.smithi077.stdout:5/805: dwrite d1/d12/d28/d38/fee [0,4194304] 0 2024-04-26T04:53:19.494 INFO:tasks.workunit.client.0.smithi077.stdout:6/866: truncate d8/d52/d53/d5e/f8c 4343947 0 2024-04-26T04:53:19.496 INFO:tasks.workunit.client.0.smithi077.stdout:5/806: mknod d1/d42/da4/dd3/c110 0 2024-04-26T04:53:19.500 INFO:tasks.workunit.client.0.smithi077.stdout:2/933: mknod d9/dc/d1d/d34/dd4/d139/c142 0 2024-04-26T04:53:19.504 INFO:tasks.workunit.client.0.smithi077.stdout:4/856: dread d4/d17/d18/d43/d6e/f95 [0,4194304] 0 2024-04-26T04:53:19.505 INFO:tasks.workunit.client.0.smithi077.stdout:5/807: creat d1/d42/da4/d25/d31/d3c/dec/f111 x:0 0 0 2024-04-26T04:53:19.506 INFO:tasks.workunit.client.0.smithi077.stdout:5/808: chown d1/d12/d87/fe5 3004 1 2024-04-26T04:53:19.508 INFO:tasks.workunit.client.0.smithi077.stdout:9/757: dwrite d9/f82 [0,4194304] 0 2024-04-26T04:53:19.510 INFO:tasks.workunit.client.0.smithi077.stdout:8/857: link d4/df/d6f/da3/dc7/ffa d4/d69/d73/f11b 0 2024-04-26T04:53:19.513 INFO:tasks.workunit.client.0.smithi077.stdout:4/857: rmdir d4/d17/d18/d43/d6e/d6c/dae 39 2024-04-26T04:53:19.515 INFO:tasks.workunit.client.0.smithi077.stdout:4/858: write d4/d4e/dad/d8b/f102 [346466,48544] 0 2024-04-26T04:53:19.516 INFO:tasks.workunit.client.0.smithi077.stdout:4/859: readlink d4/d17/d18/d43/d51/db4/l13 0 2024-04-26T04:53:19.519 INFO:tasks.workunit.client.0.smithi077.stdout:5/809: rename d1/d8c/l93 to d1/d8/d72/l112 0 2024-04-26T04:53:19.522 INFO:tasks.workunit.client.0.smithi077.stdout:4/860: creat d4/d4e/dad/d8b/dba/f11f x:0 0 0 2024-04-26T04:53:19.525 INFO:tasks.workunit.client.0.smithi077.stdout:8/858: creat d4/df/dce/f11c x:0 0 0 2024-04-26T04:53:19.526 INFO:tasks.workunit.client.0.smithi077.stdout:6/867: getdents d8/d52/da0/dad 0 2024-04-26T04:53:19.526 INFO:tasks.workunit.client.0.smithi077.stdout:6/868: stat d8/d2a/c8e 0 2024-04-26T04:53:19.559 INFO:tasks.workunit.client.0.smithi077.stdout:5/810: mkdir d1/d42/d62/dc8/dd5/ddb/d113 0 2024-04-26T04:53:19.560 INFO:tasks.workunit.client.0.smithi077.stdout:6/869: mkdir d8/d52/d53/d5e/d6c/d135 0 2024-04-26T04:53:19.561 INFO:tasks.workunit.client.0.smithi077.stdout:8/859: creat d4/df/d22/d2d/d2b/d33/dd8/d117/f11d x:0 0 0 2024-04-26T04:53:19.563 INFO:tasks.workunit.client.0.smithi077.stdout:5/811: dread d1/d12/f2a [0,4194304] 0 2024-04-26T04:53:19.655 INFO:tasks.workunit.client.0.smithi077.stdout:8/860: dread d4/df/d22/d2d/d2b/d33/fe1 [0,4194304] 0 2024-04-26T04:53:19.660 INFO:tasks.workunit.client.0.smithi077.stdout:8/861: dwrite d4/df/d18/fb0 [4194304,4194304] 0 2024-04-26T04:53:19.664 INFO:tasks.workunit.client.0.smithi077.stdout:4/861: rename d4/d17/d18/d43/d51/cb0 to d4/d17/d18/d43/d51/db4/d15/c120 0 2024-04-26T04:53:19.667 INFO:tasks.workunit.client.0.smithi077.stdout:5/812: fdatasync d1/d8d/da7/fb4 0 2024-04-26T04:53:19.669 INFO:tasks.workunit.client.0.smithi077.stdout:5/813: write d1/d42/d62/dc8/fdc [5093744,26850] 0 2024-04-26T04:53:19.714 INFO:tasks.workunit.client.0.smithi077.stdout:6/870: rename d8/d52/d53/d5e/fa5 to d8/d2a/ddd/f136 0 2024-04-26T04:53:19.715 INFO:tasks.workunit.client.0.smithi077.stdout:6/871: chown d8/d8f/fe5 259655173 1 2024-04-26T04:53:19.715 INFO:tasks.workunit.client.0.smithi077.stdout:6/872: write d8/d52/da0/dad/ff3 [1056851,106592] 0 2024-04-26T04:53:19.718 INFO:tasks.workunit.client.0.smithi077.stdout:1/811: link d9/d53/d8b/d93/da8/da7/fb3 d9/db/dd/d34/d63/f104 0 2024-04-26T04:53:19.719 INFO:tasks.workunit.client.0.smithi077.stdout:4/862: mknod d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/c121 0 2024-04-26T04:53:19.722 INFO:tasks.workunit.client.0.smithi077.stdout:8/862: dread d4/df/d22/d2d/f1c [0,4194304] 0 2024-04-26T04:53:19.727 INFO:tasks.workunit.client.0.smithi077.stdout:4/863: symlink d4/d4e/dad/d8b/l122 0 2024-04-26T04:53:19.727 INFO:tasks.workunit.client.0.smithi077.stdout:5/814: dread d1/fe3 [0,4194304] 0 2024-04-26T04:53:19.727 INFO:tasks.workunit.client.0.smithi077.stdout:2/934: dwrite d9/d15/d91/f96 [4194304,4194304] 0 2024-04-26T04:53:19.734 INFO:tasks.workunit.client.0.smithi077.stdout:1/812: mkdir d9/d53/d8b/d93/da8/dd7/ddc/d105 0 2024-04-26T04:53:19.743 INFO:tasks.workunit.client.0.smithi077.stdout:4/864: mkdir d4/d17/d18/d43/d123 0 2024-04-26T04:53:19.744 INFO:tasks.workunit.client.0.smithi077.stdout:4/865: write d4/d4e/dad/df8/f10b [50852,125594] 0 2024-04-26T04:53:19.747 INFO:tasks.workunit.client.0.smithi077.stdout:4/866: write d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/f8e [1833066,23110] 0 2024-04-26T04:53:19.749 INFO:tasks.workunit.client.0.smithi077.stdout:2/935: truncate d9/dc/f5c 4204716 0 2024-04-26T04:53:19.750 INFO:tasks.workunit.client.0.smithi077.stdout:2/936: chown d9/dc/d1d/d1e/c49 6 1 2024-04-26T04:53:19.752 INFO:tasks.workunit.client.0.smithi077.stdout:6/873: creat d8/d52/da0/f137 x:0 0 0 2024-04-26T04:53:19.759 INFO:tasks.workunit.client.0.smithi077.stdout:1/813: dread d9/db/f4c [0,4194304] 0 2024-04-26T04:53:19.759 INFO:tasks.workunit.client.0.smithi077.stdout:4/867: symlink d4/d4e/dad/d8b/ddc/l124 0 2024-04-26T04:53:19.760 INFO:tasks.workunit.client.0.smithi077.stdout:8/863: dwrite d4/df/d22/d2d/d2b/d33/fe1 [0,4194304] 0 2024-04-26T04:53:19.762 INFO:tasks.workunit.client.0.smithi077.stdout:2/937: creat d9/dc/d1d/d34/d39/d63/df6/f143 x:0 0 0 2024-04-26T04:53:19.767 INFO:tasks.workunit.client.0.smithi077.stdout:6/874: dread - d8/d52/d53/d5e/d6c/fdc zero size 2024-04-26T04:53:19.775 INFO:tasks.workunit.client.0.smithi077.stdout:8/864: creat d4/df/d6f/f11e x:0 0 0 2024-04-26T04:53:19.776 INFO:tasks.workunit.client.0.smithi077.stdout:2/938: rename d9/d15/f37 to d9/dc/d1d/d34/df8/f144 0 2024-04-26T04:53:19.778 INFO:tasks.workunit.client.0.smithi077.stdout:7/857: sync 2024-04-26T04:53:19.778 INFO:tasks.workunit.client.0.smithi077.stdout:6/875: symlink d8/d52/l138 0 2024-04-26T04:53:19.778 INFO:tasks.workunit.client.0.smithi077.stdout:5/815: getdents d1/d42/da4/d25/d31/d3c/dec 0 2024-04-26T04:53:19.780 INFO:tasks.workunit.client.0.smithi077.stdout:1/814: creat d9/db/dff/f106 x:0 0 0 2024-04-26T04:53:19.780 INFO:tasks.workunit.client.0.smithi077.stdout:1/815: stat d9/f21 0 2024-04-26T04:53:19.782 INFO:tasks.workunit.client.0.smithi077.stdout:8/865: creat d4/df/d6f/da3/dc7/dde/f11f x:0 0 0 2024-04-26T04:53:19.782 INFO:tasks.workunit.client.0.smithi077.stdout:2/939: rmdir d9/d15 39 2024-04-26T04:53:19.783 INFO:tasks.workunit.client.0.smithi077.stdout:8/866: fdatasync d4/df/d22/d75/fcf 0 2024-04-26T04:53:19.783 INFO:tasks.workunit.client.0.smithi077.stdout:2/940: write d9/f2c [2681088,107172] 0 2024-04-26T04:53:19.785 INFO:tasks.workunit.client.0.smithi077.stdout:8/867: dread - d4/df/dce/d5b/d8d/ff9 zero size 2024-04-26T04:53:19.786 INFO:tasks.workunit.client.0.smithi077.stdout:4/868: creat d4/d17/d18/f125 x:0 0 0 2024-04-26T04:53:19.788 INFO:tasks.workunit.client.0.smithi077.stdout:4/869: write d4/d17/d18/d43/d6e/f84 [86167,20388] 0 2024-04-26T04:53:19.790 INFO:tasks.workunit.client.0.smithi077.stdout:4/870: truncate d4/d4e/dad/df8/f113 303506 0 2024-04-26T04:53:19.791 INFO:tasks.workunit.client.0.smithi077.stdout:1/816: mkdir d9/d90/de0/d107 0 2024-04-26T04:53:19.797 INFO:tasks.workunit.client.0.smithi077.stdout:8/868: fdatasync d4/d69/d73/fe5 0 2024-04-26T04:53:19.798 INFO:tasks.workunit.client.0.smithi077.stdout:2/941: truncate d9/dc/d1d/d34/d39/d63/de9/fea 919453 0 2024-04-26T04:53:19.799 INFO:tasks.workunit.client.0.smithi077.stdout:6/876: dread d8/d2a/d86/fe0 [0,4194304] 0 2024-04-26T04:53:19.803 INFO:tasks.workunit.client.0.smithi077.stdout:5/816: fdatasync d1/d8d/da7/f10c 0 2024-04-26T04:53:19.805 INFO:tasks.workunit.client.0.smithi077.stdout:6/877: dwrite d8/d52/da0/f137 [0,4194304] 0 2024-04-26T04:53:19.807 INFO:tasks.workunit.client.0.smithi077.stdout:7/858: link da/d13/d22/d5d/d9f/fdf da/d13/d22/d5d/d9f/f122 0 2024-04-26T04:53:19.810 INFO:tasks.workunit.client.0.smithi077.stdout:4/871: rename d4/d17/d18/d43/d63/d80/c85 to d4/d17/d18/d43/c126 0 2024-04-26T04:53:19.814 INFO:tasks.workunit.client.0.smithi077.stdout:1/817: chown d9/db/d2b/d75/l9d 306899 1 2024-04-26T04:53:19.817 INFO:tasks.workunit.client.0.smithi077.stdout:2/942: symlink d9/dc/d1d/d34/df8/dd1/dc3/l145 0 2024-04-26T04:53:19.821 INFO:tasks.workunit.client.0.smithi077.stdout:6/878: rename d8/d52/d53/d5e/d6c/d102 to d8/d52/d53/d5e/da6/d133/d139 0 2024-04-26T04:53:19.821 INFO:tasks.workunit.client.0.smithi077.stdout:6/879: stat d8/d52/d53/da7/df4/f128 0 2024-04-26T04:53:19.823 INFO:tasks.workunit.client.0.smithi077.stdout:2/943: mkdir d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/d146 0 2024-04-26T04:53:19.825 INFO:tasks.workunit.client.0.smithi077.stdout:4/872: chown d4/d17/d18/d43/d6e/d6c/dae/dc5/cf1 16 1 2024-04-26T04:53:19.826 INFO:tasks.workunit.client.0.smithi077.stdout:6/880: unlink d8/d52/c99 0 2024-04-26T04:53:19.829 INFO:tasks.workunit.client.0.smithi077.stdout:2/944: dwrite d9/dc/d1d/f24 [0,4194304] 0 2024-04-26T04:53:19.833 INFO:tasks.workunit.client.0.smithi077.stdout:4/873: unlink d4/d17/d18/l73 0 2024-04-26T04:53:19.834 INFO:tasks.workunit.client.0.smithi077.stdout:5/817: getdents d1/d8c 0 2024-04-26T04:53:19.836 INFO:tasks.workunit.client.0.smithi077.stdout:4/874: dread d4/d17/d18/d43/d51/db4/d15/d1f/d35/fc1 [0,4194304] 0 2024-04-26T04:53:19.837 INFO:tasks.workunit.client.0.smithi077.stdout:8/869: write d4/df/d22/f9e [433824,108022] 0 2024-04-26T04:53:19.838 INFO:tasks.workunit.client.0.smithi077.stdout:1/818: link d9/db/dd/c57 d9/d53/c108 0 2024-04-26T04:53:19.841 INFO:tasks.workunit.client.0.smithi077.stdout:5/818: truncate d1/d42/f69 1032750 0 2024-04-26T04:53:19.845 INFO:tasks.workunit.client.0.smithi077.stdout:4/875: truncate d4/d17/d18/d43/d51/db4/fa6 1545769 0 2024-04-26T04:53:19.850 INFO:tasks.workunit.client.0.smithi077.stdout:8/870: rename d4/df/d6f to d4/df/d22/d2d/d2b/d33/dd8/d117/d120 0 2024-04-26T04:53:19.851 INFO:tasks.workunit.client.0.smithi077.stdout:8/871: chown d4/df/l6c 367417 1 2024-04-26T04:53:19.852 INFO:tasks.workunit.client.0.smithi077.stdout:1/819: mkdir d9/d53/dba/d109 0 2024-04-26T04:53:19.857 INFO:tasks.workunit.client.0.smithi077.stdout:8/872: dread d4/df/d22/f9e [0,4194304] 0 2024-04-26T04:53:19.858 INFO:tasks.workunit.client.0.smithi077.stdout:6/881: truncate d8/d52/d53/da7/fea 3492939 0 2024-04-26T04:53:19.859 INFO:tasks.workunit.client.0.smithi077.stdout:1/820: mknod d9/d90/d99/d9a/c10a 0 2024-04-26T04:53:19.860 INFO:tasks.workunit.client.0.smithi077.stdout:5/819: dread d1/d42/da4/d25/d31/f71 [0,4194304] 0 2024-04-26T04:53:19.862 INFO:tasks.workunit.client.0.smithi077.stdout:2/945: dwrite d9/dc/d1d/d34/d39/d63/fca [4194304,4194304] 0 2024-04-26T04:53:19.863 INFO:tasks.workunit.client.0.smithi077.stdout:5/820: write d1/d12/d28/ffc [753233,2724] 0 2024-04-26T04:53:19.871 INFO:tasks.workunit.client.0.smithi077.stdout:1/821: unlink d9/db/d2b/d75/f91 0 2024-04-26T04:53:19.872 INFO:tasks.workunit.client.0.smithi077.stdout:1/822: stat d9/d53/dc1/lc8 0 2024-04-26T04:53:19.879 INFO:tasks.workunit.client.0.smithi077.stdout:4/876: truncate d4/d17/d18/d43/d51/db4/d15/d1f/fe7 8330847 0 2024-04-26T04:53:19.883 INFO:tasks.workunit.client.0.smithi077.stdout:2/946: mknod d9/dc/d1d/d1e/d77/df9/c147 0 2024-04-26T04:53:19.892 INFO:tasks.workunit.client.0.smithi077.stdout:2/947: unlink d9/dc/d1d/d34/d39/d63/de9/d11e/le8 0 2024-04-26T04:53:19.893 INFO:tasks.workunit.client.0.smithi077.stdout:2/948: dread - d9/dc/d1d/d34/d39/d63/d79/d5b/fcf zero size 2024-04-26T04:53:19.900 INFO:tasks.workunit.client.0.smithi077.stdout:8/873: truncate d4/d69/d73/fef 1567224 0 2024-04-26T04:53:19.902 INFO:tasks.workunit.client.0.smithi077.stdout:4/877: write d4/d17/d18/fe2 [878520,15500] 0 2024-04-26T04:53:19.905 INFO:tasks.workunit.client.0.smithi077.stdout:2/949: dread - d9/d15/d44/d4e/f13c zero size 2024-04-26T04:53:19.906 INFO:tasks.workunit.client.0.smithi077.stdout:1/823: link d9/db/d2b/d75/fa6 d9/d90/d99/d9a/f10b 0 2024-04-26T04:53:19.909 INFO:tasks.workunit.client.0.smithi077.stdout:1/824: write d9/d53/d8b/d93/da8/db8/fc5 [177722,11710] 0 2024-04-26T04:53:19.909 INFO:tasks.workunit.client.0.smithi077.stdout:5/821: dwrite d1/d42/f69 [0,4194304] 0 2024-04-26T04:53:19.910 INFO:tasks.workunit.client.0.smithi077.stdout:1/825: chown d9/d53/f6f 812 1 2024-04-26T04:53:19.911 INFO:tasks.workunit.client.0.smithi077.stdout:4/878: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/d127 0 2024-04-26T04:53:19.918 INFO:tasks.workunit.client.0.smithi077.stdout:5/822: dwrite d1/d12/d45/f63 [0,4194304] 0 2024-04-26T04:53:19.919 INFO:tasks.workunit.client.0.smithi077.stdout:5/823: truncate d1/d12/d28/fff 981856 0 2024-04-26T04:53:19.926 INFO:tasks.workunit.client.0.smithi077.stdout:1/826: mkdir d9/d53/d8b/d10c 0 2024-04-26T04:53:19.927 INFO:tasks.workunit.client.0.smithi077.stdout:4/879: unlink d4/d17/d18/d43/d53/lfc 0 2024-04-26T04:53:19.929 INFO:tasks.workunit.client.0.smithi077.stdout:1/827: mknod d9/d90/de0/c10d 0 2024-04-26T04:53:19.931 INFO:tasks.workunit.client.0.smithi077.stdout:8/874: getdents d4/df 0 2024-04-26T04:53:19.934 INFO:tasks.workunit.client.0.smithi077.stdout:4/880: symlink d4/d17/d18/d43/d63/l128 0 2024-04-26T04:53:19.937 INFO:tasks.workunit.client.0.smithi077.stdout:5/824: getdents d1/d42/da4/d25/d31 0 2024-04-26T04:53:19.939 INFO:tasks.workunit.client.0.smithi077.stdout:5/825: chown d1/d42/da4/d25/l26 18 1 2024-04-26T04:53:19.939 INFO:tasks.workunit.client.0.smithi077.stdout:4/881: mkdir d4/d17/d18/d43/d53/d64/d129 0 2024-04-26T04:53:19.940 INFO:tasks.workunit.client.0.smithi077.stdout:9/758: sync 2024-04-26T04:53:19.941 INFO:tasks.workunit.client.0.smithi077.stdout:3/744: sync 2024-04-26T04:53:19.941 INFO:tasks.workunit.client.0.smithi077.stdout:0/793: sync 2024-04-26T04:53:19.942 INFO:tasks.workunit.client.0.smithi077.stdout:2/950: dwrite d9/dc/d1d/d34/df8/dd1/dc3/fa0 [0,4194304] 0 2024-04-26T04:53:19.942 INFO:tasks.workunit.client.0.smithi077.stdout:0/794: fsync d0/f1 0 2024-04-26T04:53:19.943 INFO:tasks.workunit.client.0.smithi077.stdout:1/828: getdents d9/d53/dc1 0 2024-04-26T04:53:19.947 INFO:tasks.workunit.client.0.smithi077.stdout:0/795: write d0/d28/d40/f85 [2147455,87806] 0 2024-04-26T04:53:19.950 INFO:tasks.workunit.client.0.smithi077.stdout:3/745: write d0/d9/f78 [2864080,51674] 0 2024-04-26T04:53:19.950 INFO:tasks.workunit.client.0.smithi077.stdout:4/882: read d4/d17/d18/d43/d53/fbe [4125552,3717] 0 2024-04-26T04:53:19.952 INFO:tasks.workunit.client.0.smithi077.stdout:9/759: truncate d9/d45/f71 4981433 0 2024-04-26T04:53:19.954 INFO:tasks.workunit.client.0.smithi077.stdout:4/883: write d4/d17/f57 [4534362,7853] 0 2024-04-26T04:53:19.955 INFO:tasks.workunit.client.0.smithi077.stdout:8/875: dwrite d4/df/d22/d2d/d2b/d33/fda [0,4194304] 0 2024-04-26T04:53:19.959 INFO:tasks.workunit.client.0.smithi077.stdout:2/951: symlink d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/l148 0 2024-04-26T04:53:19.960 INFO:tasks.workunit.client.0.smithi077.stdout:5/826: dread d1/d42/d62/dc8/fdc [0,4194304] 0 2024-04-26T04:53:19.962 INFO:tasks.workunit.client.0.smithi077.stdout:0/796: chown d0/l10 43300197 1 2024-04-26T04:53:19.964 INFO:tasks.workunit.client.0.smithi077.stdout:3/746: dwrite d0/dae/ded/d2a/f4e [0,4194304] 0 2024-04-26T04:53:19.964 INFO:tasks.workunit.client.0.smithi077.stdout:9/760: fdatasync d9/d3a/d79/f33 0 2024-04-26T04:53:19.969 INFO:tasks.workunit.client.0.smithi077.stdout:5/827: truncate d1/d42/da4/d25/d31/d3c/f68 4972611 0 2024-04-26T04:53:19.969 INFO:tasks.workunit.client.0.smithi077.stdout:0/797: write d0/d47/d93/d94/faa [1230775,61015] 0 2024-04-26T04:53:19.970 INFO:tasks.workunit.client.0.smithi077.stdout:0/798: dread - d0/d2c/d109/dd7/f107 zero size 2024-04-26T04:53:19.970 INFO:tasks.workunit.client.0.smithi077.stdout:2/952: dread d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f135 [0,4194304] 0 2024-04-26T04:53:19.972 INFO:tasks.workunit.client.0.smithi077.stdout:4/884: getdents d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/d127 0 2024-04-26T04:53:19.973 INFO:tasks.workunit.client.0.smithi077.stdout:1/829: link d9/db/dd/l26 d9/db/dd/d9c/dab/de8/def/l10e 0 2024-04-26T04:53:19.974 INFO:tasks.workunit.client.0.smithi077.stdout:1/830: fdatasync d9/db/f39 0 2024-04-26T04:53:19.978 INFO:tasks.workunit.client.0.smithi077.stdout:4/885: dread d4/d17/d18/d43/d63/fb3 [0,4194304] 0 2024-04-26T04:53:19.979 INFO:tasks.workunit.client.0.smithi077.stdout:3/747: mknod d0/d7/d1f/d81/cf3 0 2024-04-26T04:53:19.980 INFO:tasks.workunit.client.0.smithi077.stdout:3/748: chown d0/feb 1118159 1 2024-04-26T04:53:19.986 INFO:tasks.workunit.client.0.smithi077.stdout:1/831: creat d9/d53/d8b/f10f x:0 0 0 2024-04-26T04:53:19.989 INFO:tasks.workunit.client.0.smithi077.stdout:0/799: dread d0/d47/d68/f86 [0,4194304] 0 2024-04-26T04:53:19.990 INFO:tasks.workunit.client.0.smithi077.stdout:9/761: symlink d9/d3a/d79/d95/dad/de7/lfc 0 2024-04-26T04:53:19.991 INFO:tasks.workunit.client.0.smithi077.stdout:4/886: creat d4/d17/d18/d43/d63/d80/f12a x:0 0 0 2024-04-26T04:53:19.996 INFO:tasks.workunit.client.0.smithi077.stdout:8/876: dread d4/d87/fb2 [0,4194304] 0 2024-04-26T04:53:19.996 INFO:tasks.workunit.client.0.smithi077.stdout:0/800: dwrite d0/d47/ff [0,4194304] 0 2024-04-26T04:53:19.997 INFO:tasks.workunit.client.0.smithi077.stdout:0/801: chown d0/fe 209960300 1 2024-04-26T04:53:20.008 INFO:tasks.workunit.client.0.smithi077.stdout:5/828: dwrite d1/d30/d35/f5a [4194304,4194304] 0 2024-04-26T04:53:20.008 INFO:tasks.workunit.client.0.smithi077.stdout:4/887: dread d4/d17/d7f/feb [0,4194304] 0 2024-04-26T04:53:20.015 INFO:tasks.workunit.client.0.smithi077.stdout:3/749: truncate d0/d7/d1f/d4d/f56 885188 0 2024-04-26T04:53:20.019 INFO:tasks.workunit.client.0.smithi077.stdout:2/953: truncate d9/d15/d91/f96 1981810 0 2024-04-26T04:53:20.020 INFO:tasks.workunit.client.0.smithi077.stdout:0/802: symlink d0/d47/d93/l10f 0 2024-04-26T04:53:20.022 INFO:tasks.workunit.client.0.smithi077.stdout:2/954: truncate d9/dc/d1d/d34/d39/d63/f104 232425 0 2024-04-26T04:53:20.025 INFO:tasks.workunit.client.0.smithi077.stdout:1/832: rename d9/db/dd/l26 to d9/db/dd/d34/d63/l110 0 2024-04-26T04:53:20.029 INFO:tasks.workunit.client.0.smithi077.stdout:5/829: dread d1/f5 [0,4194304] 0 2024-04-26T04:53:20.030 INFO:tasks.workunit.client.0.smithi077.stdout:9/762: symlink d9/d45/lfd 0 2024-04-26T04:53:20.031 INFO:tasks.workunit.client.0.smithi077.stdout:8/877: write d4/df/d22/d2d/d2b/d33/dd8/d117/d120/f7e [9029,6596] 0 2024-04-26T04:53:20.035 INFO:tasks.workunit.client.0.smithi077.stdout:8/878: readlink d4/l29 0 2024-04-26T04:53:20.079 INFO:tasks.workunit.client.0.smithi077.stdout:7/859: sync 2024-04-26T04:53:20.079 INFO:tasks.workunit.client.0.smithi077.stdout:7/860: stat da/d13/d22/d5d/d8d/d10c 0 2024-04-26T04:53:20.140 INFO:tasks.workunit.client.0.smithi077.stdout:0/803: creat d0/d19/d1e/d59/d96/f110 x:0 0 0 2024-04-26T04:53:20.143 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:19 smithi077 ceph-mon[26311]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 104 MiB/s rd, 241 MiB/s wr, 549 op/s 2024-04-26T04:53:20.146 INFO:tasks.workunit.client.0.smithi077.stdout:4/888: mkdir d4/d12b 0 2024-04-26T04:53:20.148 INFO:tasks.workunit.client.0.smithi077.stdout:1/833: write d9/d53/d6e/f87 [6012301,73973] 0 2024-04-26T04:53:20.172 INFO:tasks.workunit.client.0.smithi077.stdout:7/861: unlink da/d13/d22/d5d/dbd/f11b 0 2024-04-26T04:53:20.178 INFO:tasks.workunit.client.0.smithi077.stdout:1/834: creat d9/db/d2b/d3f/f111 x:0 0 0 2024-04-26T04:53:20.179 INFO:tasks.workunit.client.0.smithi077.stdout:8/879: getdents d4/df/d22/d2d/d2b/d33/dd8/d117/d120/db8 0 2024-04-26T04:53:20.180 INFO:tasks.workunit.client.0.smithi077.stdout:8/880: chown d4/d69/d73/f11b 5 1 2024-04-26T04:53:20.180 INFO:tasks.workunit.client.0.smithi077.stdout:8/881: chown d4/d69/d73/fb5 23588076 1 2024-04-26T04:53:20.184 INFO:tasks.workunit.client.0.smithi077.stdout:2/955: creat d9/dc/d1d/d34/df8/dd1/f149 x:0 0 0 2024-04-26T04:53:20.184 INFO:tasks.workunit.client.0.smithi077.stdout:2/956: readlink d9/d15/d91/leb 0 2024-04-26T04:53:20.185 INFO:tasks.workunit.client.0.smithi077.stdout:3/750: rename d0/d7/d1f/d81/f89 to d0/d9/ff4 0 2024-04-26T04:53:20.187 INFO:tasks.workunit.client.0.smithi077.stdout:5/830: creat d1/d30/f114 x:0 0 0 2024-04-26T04:53:20.188 INFO:tasks.workunit.client.0.smithi077.stdout:7/862: dread da/d15/f94 [0,4194304] 0 2024-04-26T04:53:20.188 INFO:tasks.workunit.client.0.smithi077.stdout:5/831: fdatasync d1/d12/d28/ffc 0 2024-04-26T04:53:20.189 INFO:tasks.workunit.client.0.smithi077.stdout:5/832: truncate d1/d12/d28/fff 1570173 0 2024-04-26T04:53:20.190 INFO:tasks.workunit.client.0.smithi077.stdout:8/882: mknod d4/df/d22/d2d/d2b/d33/dd8/d117/d120/db8/c121 0 2024-04-26T04:53:20.192 INFO:tasks.workunit.client.0.smithi077.stdout:0/804: rmdir d0/d19/d1e/d2f/da8/dd4 0 2024-04-26T04:53:20.193 INFO:tasks.workunit.client.0.smithi077.stdout:0/805: fdatasync d0/d19/d69/f84 0 2024-04-26T04:53:20.193 INFO:tasks.workunit.client.0.smithi077.stdout:2/957: mknod d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/c14a 0 2024-04-26T04:53:20.194 INFO:tasks.workunit.client.0.smithi077.stdout:3/751: fsync d0/dae/ded/d2a/f4c 0 2024-04-26T04:53:20.198 INFO:tasks.workunit.client.0.smithi077.stdout:7/863: mkdir da/d13/d22/d7d/d80/d113/d119/d123 0 2024-04-26T04:53:20.210 INFO:tasks.workunit.client.0.smithi077.stdout:2/958: read d9/dc/d1d/d34/df8/f144 [3193667,130915] 0 2024-04-26T04:53:20.210 INFO:tasks.workunit.client.0.smithi077.stdout:3/752: dread d0/dae/ded/d2a/faa [0,4194304] 0 2024-04-26T04:53:20.212 INFO:tasks.workunit.client.0.smithi077.stdout:2/959: dread - d9/dc/d1d/d34/d39/d63/df6/f143 zero size 2024-04-26T04:53:20.213 INFO:tasks.workunit.client.0.smithi077.stdout:4/889: truncate d4/d17/d18/d43/d63/fb3 3305356 0 2024-04-26T04:53:20.214 INFO:tasks.workunit.client.0.smithi077.stdout:7/864: dread da/d13/d22/d5d/d9f/ddd/f120 [0,4194304] 0 2024-04-26T04:53:20.216 INFO:tasks.workunit.client.0.smithi077.stdout:3/753: mknod d0/d7/d1f/d81/dc5/cf5 0 2024-04-26T04:53:20.220 INFO:tasks.workunit.client.0.smithi077.stdout:0/806: creat d0/d2c/f111 x:0 0 0 2024-04-26T04:53:20.220 INFO:tasks.workunit.client.0.smithi077.stdout:1/835: truncate d9/db/f4c 3878465 0 2024-04-26T04:53:20.221 INFO:tasks.workunit.client.0.smithi077.stdout:4/890: readlink d4/d17/d7f/da3/ld1 0 2024-04-26T04:53:20.223 INFO:tasks.workunit.client.0.smithi077.stdout:3/754: dread d0/dae/ded/f35 [0,4194304] 0 2024-04-26T04:53:20.225 INFO:tasks.workunit.client.0.smithi077.stdout:7/865: creat da/d13/d22/d5d/d9f/da7/de0/f124 x:0 0 0 2024-04-26T04:53:20.227 INFO:tasks.workunit.client.0.smithi077.stdout:2/960: mkdir d9/dc/d1d/d1e/d77/d11b/d14b 0 2024-04-26T04:53:20.227 INFO:tasks.workunit.client.0.smithi077.stdout:5/833: dwrite d1/d12/d28/d38/fb5 [0,4194304] 0 2024-04-26T04:53:20.232 INFO:tasks.workunit.client.0.smithi077.stdout:4/891: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d76/d12c 0 2024-04-26T04:53:20.233 INFO:tasks.workunit.client.0.smithi077.stdout:7/866: truncate da/d6b/d63/d64/fc1 322298 0 2024-04-26T04:53:20.238 INFO:tasks.workunit.client.0.smithi077.stdout:0/807: symlink d0/d47/d64/d97/df9/l112 0 2024-04-26T04:53:20.240 INFO:tasks.workunit.client.0.smithi077.stdout:2/961: dread d9/d15/d91/f96 [0,4194304] 0 2024-04-26T04:53:20.243 INFO:tasks.workunit.client.0.smithi077.stdout:3/755: mknod d0/d9/cf6 0 2024-04-26T04:53:20.248 INFO:tasks.workunit.client.0.smithi077.stdout:0/808: symlink d0/d47/d64/d97/df9/l113 0 2024-04-26T04:53:20.251 INFO:tasks.workunit.client.0.smithi077.stdout:2/962: unlink d9/dc/d1d/d34/l97 0 2024-04-26T04:53:20.251 INFO:tasks.workunit.client.0.smithi077.stdout:1/836: link d9/d53/d8b/d93/da8/dd7/ce2 d9/db/dd/d9c/dab/de8/c112 0 2024-04-26T04:53:20.256 INFO:tasks.workunit.client.0.smithi077.stdout:0/809: symlink d0/d47/d68/d9f/l114 0 2024-04-26T04:53:20.257 INFO:tasks.workunit.client.0.smithi077.stdout:0/810: write d0/d2c/f88 [4118609,100747] 0 2024-04-26T04:53:20.264 INFO:tasks.workunit.client.0.smithi077.stdout:4/892: rmdir d4/d17/d18/d43/d51/db4/d15/d1f/d25/d10c 0 2024-04-26T04:53:20.267 INFO:tasks.workunit.client.0.smithi077.stdout:0/811: creat d0/d2c/d109/f115 x:0 0 0 2024-04-26T04:53:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:19 smithi078 ceph-mon[33746]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 104 MiB/s rd, 241 MiB/s wr, 549 op/s 2024-04-26T04:53:20.271 INFO:tasks.workunit.client.0.smithi077.stdout:3/756: dwrite d0/d7/d1f/d76/f8d [0,4194304] 0 2024-04-26T04:53:20.271 INFO:tasks.workunit.client.0.smithi077.stdout:1/837: mkdir d9/d53/d6e/dd3/d113 0 2024-04-26T04:53:20.274 INFO:tasks.workunit.client.0.smithi077.stdout:0/812: dwrite d0/d2c/d109/dd7/fe6 [0,4194304] 0 2024-04-26T04:53:20.277 INFO:tasks.workunit.client.0.smithi077.stdout:4/893: rmdir d4/d17/d18/d43 39 2024-04-26T04:53:20.284 INFO:tasks.workunit.client.0.smithi077.stdout:0/813: symlink d0/d19/d1e/d59/l116 0 2024-04-26T04:53:20.285 INFO:tasks.workunit.client.0.smithi077.stdout:0/814: write d0/d47/d64/d97/dba/fca [1979287,61012] 0 2024-04-26T04:53:20.286 INFO:tasks.workunit.client.0.smithi077.stdout:2/963: getdents d9/d15/d56/de6 0 2024-04-26T04:53:20.286 INFO:tasks.workunit.client.0.smithi077.stdout:4/894: chown d4/d17/d18/d43/d51/db4/d15/d1f/d25 33222 1 2024-04-26T04:53:20.287 INFO:tasks.workunit.client.0.smithi077.stdout:1/838: creat d9/d90/de0/d107/f114 x:0 0 0 2024-04-26T04:53:20.294 INFO:tasks.workunit.client.0.smithi077.stdout:1/839: unlink d9/db/dd/d34/d63/f76 0 2024-04-26T04:53:20.295 INFO:tasks.workunit.client.0.smithi077.stdout:4/895: creat d4/f12d x:0 0 0 2024-04-26T04:53:20.296 INFO:tasks.workunit.client.0.smithi077.stdout:1/840: getdents d9/d53/dc1 0 2024-04-26T04:53:20.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/896: fsync d4/d17/d18/d43/d51/db4/f44 0 2024-04-26T04:53:20.300 INFO:tasks.workunit.client.0.smithi077.stdout:4/897: creat d4/d4e/dad/d8b/ddc/f12e x:0 0 0 2024-04-26T04:53:20.301 INFO:tasks.workunit.client.0.smithi077.stdout:1/841: mkdir d9/db/dd/d34/d4a/dbb/d115 0 2024-04-26T04:53:20.301 INFO:tasks.workunit.client.0.smithi077.stdout:4/898: dread - d4/d4e/dad/d8b/ddc/f12e zero size 2024-04-26T04:53:20.302 INFO:tasks.workunit.client.0.smithi077.stdout:4/899: write d4/d17/d18/d43/d51/db4/f44 [4866396,7964] 0 2024-04-26T04:53:20.303 INFO:tasks.workunit.client.0.smithi077.stdout:3/757: write d0/d7/d1f/f4a [1400310,1606] 0 2024-04-26T04:53:20.310 INFO:tasks.workunit.client.0.smithi077.stdout:3/758: dwrite d0/feb [0,4194304] 0 2024-04-26T04:53:20.311 INFO:tasks.workunit.client.0.smithi077.stdout:2/964: truncate d9/dc/f41 2250130 0 2024-04-26T04:53:20.313 INFO:tasks.workunit.client.0.smithi077.stdout:0/815: dwrite d0/d2c/d4e/f14 [0,4194304] 0 2024-04-26T04:53:20.314 INFO:tasks.workunit.client.0.smithi077.stdout:0/816: fdatasync d0/d47/d93/d94/faa 0 2024-04-26T04:53:20.316 INFO:tasks.workunit.client.0.smithi077.stdout:1/842: getdents d9/d53/d8b/d93/da8/dd7 0 2024-04-26T04:53:20.316 INFO:tasks.workunit.client.0.smithi077.stdout:1/843: chown d9/db/d2b/d75 43 1 2024-04-26T04:53:20.322 INFO:tasks.workunit.client.0.smithi077.stdout:2/965: chown d9/d15/d44/c8f 244395 1 2024-04-26T04:53:20.325 INFO:tasks.workunit.client.0.smithi077.stdout:0/817: mkdir d0/d47/d68/d117 0 2024-04-26T04:53:20.325 INFO:tasks.workunit.client.0.smithi077.stdout:1/844: fsync d9/d90/d99/fd5 0 2024-04-26T04:53:20.330 INFO:tasks.workunit.client.0.smithi077.stdout:3/759: link d0/d7/d1f/f72 d0/d7/d1f/d81/dc5/dc7/ff7 0 2024-04-26T04:53:20.334 INFO:tasks.workunit.client.0.smithi077.stdout:4/900: getdents d4/d17/d18/d43/d6e/d6c/dda 0 2024-04-26T04:53:20.335 INFO:tasks.workunit.client.0.smithi077.stdout:0/818: dwrite d0/d2c/d109/f10b [0,4194304] 0 2024-04-26T04:53:20.335 INFO:tasks.workunit.client.0.smithi077.stdout:1/845: dwrite d9/db/dd/d9c/dab/fda [0,4194304] 0 2024-04-26T04:53:20.337 INFO:tasks.workunit.client.0.smithi077.stdout:2/966: rename d9/dc/d1d/d34/f36 to d9/dc/d1d/d34/df8/dd1/dc3/dc4/de0/f14c 0 2024-04-26T04:53:20.339 INFO:tasks.workunit.client.0.smithi077.stdout:1/846: truncate d9/db/dd/f2f 4927377 0 2024-04-26T04:53:20.341 INFO:tasks.workunit.client.0.smithi077.stdout:1/847: fsync d9/d53/f86 0 2024-04-26T04:53:20.342 INFO:tasks.workunit.client.0.smithi077.stdout:1/848: chown d9/d53/d6e/c6b 1159839 1 2024-04-26T04:53:20.343 INFO:tasks.workunit.client.0.smithi077.stdout:4/901: dwrite d4/d17/d18/d43/d63/d80/db6/fee [0,4194304] 0 2024-04-26T04:53:20.344 INFO:tasks.workunit.client.0.smithi077.stdout:1/849: write d9/d90/fbf [1275432,99032] 0 2024-04-26T04:53:20.353 INFO:tasks.workunit.client.0.smithi077.stdout:1/850: dwrite d9/db/d2b/d3f/f111 [0,4194304] 0 2024-04-26T04:53:20.358 INFO:tasks.workunit.client.0.smithi077.stdout:4/902: symlink d4/d17/d18/d43/d51/db4/d15/d1f/d25/l12f 0 2024-04-26T04:53:20.362 INFO:tasks.workunit.client.0.smithi077.stdout:0/819: creat d0/d19/d1e/d2f/da8/d101/f118 x:0 0 0 2024-04-26T04:53:20.366 INFO:tasks.workunit.client.0.smithi077.stdout:4/903: creat d4/d17/d18/d43/df7/f130 x:0 0 0 2024-04-26T04:53:20.367 INFO:tasks.workunit.client.0.smithi077.stdout:0/820: creat d0/d19/d1e/d2f/da8/f119 x:0 0 0 2024-04-26T04:53:20.369 INFO:tasks.workunit.client.0.smithi077.stdout:2/967: write d9/dc/d1d/d34/d39/fa3 [71174,78173] 0 2024-04-26T04:53:20.371 INFO:tasks.workunit.client.0.smithi077.stdout:0/821: unlink d0/d28/ceb 0 2024-04-26T04:53:20.373 INFO:tasks.workunit.client.0.smithi077.stdout:3/760: dwrite d0/dae/ded/d2a/f4c [0,4194304] 0 2024-04-26T04:53:20.374 INFO:tasks.workunit.client.0.smithi077.stdout:3/761: stat d0/d7/d9a/db3/f22 0 2024-04-26T04:53:20.375 INFO:tasks.workunit.client.0.smithi077.stdout:0/822: unlink d0/d19/ccc 0 2024-04-26T04:53:20.376 INFO:tasks.workunit.client.0.smithi077.stdout:4/904: rename d4/d17/d18/d43/d51/db4/d15/d1f/l41 to d4/d17/d18/d43/d51/db4/d15/d1f/d35/l131 0 2024-04-26T04:53:20.378 INFO:tasks.workunit.client.0.smithi077.stdout:2/968: creat d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/f14d x:0 0 0 2024-04-26T04:53:20.384 INFO:tasks.workunit.client.0.smithi077.stdout:1/851: dwrite d9/db/dd/f11 [0,4194304] 0 2024-04-26T04:53:20.391 INFO:tasks.workunit.client.0.smithi077.stdout:0/823: unlink d0/d47/f55 0 2024-04-26T04:53:20.396 INFO:tasks.workunit.client.0.smithi077.stdout:1/852: mknod d9/db/dd/d9c/c116 0 2024-04-26T04:53:20.399 INFO:tasks.workunit.client.0.smithi077.stdout:2/969: symlink d9/dc/d1d/d34/l14e 0 2024-04-26T04:53:20.400 INFO:tasks.workunit.client.0.smithi077.stdout:3/762: dwrite d0/d7/d1f/d81/fa3 [0,4194304] 0 2024-04-26T04:53:20.409 INFO:tasks.workunit.client.0.smithi077.stdout:3/763: creat d0/d7/d1f/d81/dc5/ff8 x:0 0 0 2024-04-26T04:53:20.410 INFO:tasks.workunit.client.0.smithi077.stdout:4/905: getdents d4/d17/d18/d43/d53/d56 0 2024-04-26T04:53:20.410 INFO:tasks.workunit.client.0.smithi077.stdout:0/824: dwrite d0/d2c/d4e/f66 [0,4194304] 0 2024-04-26T04:53:20.412 INFO:tasks.workunit.client.0.smithi077.stdout:3/764: read d0/dae/ded/f21 [2351244,41389] 0 2024-04-26T04:53:20.414 INFO:tasks.workunit.client.0.smithi077.stdout:4/906: creat d4/d4e/dad/d8b/dba/f132 x:0 0 0 2024-04-26T04:53:20.415 INFO:tasks.workunit.client.0.smithi077.stdout:3/765: chown d0/d7/d9a/fd6 1610994496 1 2024-04-26T04:53:20.416 INFO:tasks.workunit.client.0.smithi077.stdout:0/825: creat d0/d19/d1e/d59/f11a x:0 0 0 2024-04-26T04:53:20.417 INFO:tasks.workunit.client.0.smithi077.stdout:0/826: fdatasync d0/d2c/f88 0 2024-04-26T04:53:20.418 INFO:tasks.workunit.client.0.smithi077.stdout:0/827: stat d0/d47/d64/d97/dba/dc4/ld0 0 2024-04-26T04:53:20.421 INFO:tasks.workunit.client.0.smithi077.stdout:3/766: creat d0/d7/d9a/db7/ff9 x:0 0 0 2024-04-26T04:53:20.423 INFO:tasks.workunit.client.0.smithi077.stdout:1/853: dread f7 [0,4194304] 0 2024-04-26T04:53:20.424 INFO:tasks.workunit.client.0.smithi077.stdout:6/882: sync 2024-04-26T04:53:20.428 INFO:tasks.workunit.client.0.smithi077.stdout:3/767: symlink d0/dae/lfa 0 2024-04-26T04:53:20.429 INFO:tasks.workunit.client.0.smithi077.stdout:2/970: truncate d9/dc/d5f/d8e/db9/fe5 1686671 0 2024-04-26T04:53:20.432 INFO:tasks.workunit.client.0.smithi077.stdout:4/907: unlink d4/d17/d18/d43/d51/db4/d15/d1f/cc0 0 2024-04-26T04:53:20.433 INFO:tasks.workunit.client.0.smithi077.stdout:4/908: read - d4/d17/d18/d43/d6e/d6c/dda/f112 zero size 2024-04-26T04:53:20.435 INFO:tasks.workunit.client.0.smithi077.stdout:6/883: getdents d8/d52/d53/d5e/d6c/d135 0 2024-04-26T04:53:20.435 INFO:tasks.workunit.client.0.smithi077.stdout:4/909: write d4/d17/d18/d43/d6e/f86 [4964573,77522] 0 2024-04-26T04:53:20.437 INFO:tasks.workunit.client.0.smithi077.stdout:6/884: chown d8/d52/d53/lbc 15975941 1 2024-04-26T04:53:20.438 INFO:tasks.workunit.client.0.smithi077.stdout:6/885: stat d8/d1c/l35 0 2024-04-26T04:53:20.439 INFO:tasks.workunit.client.0.smithi077.stdout:6/886: chown d8/f5a 41317915 1 2024-04-26T04:53:20.440 INFO:tasks.workunit.client.0.smithi077.stdout:4/910: rename d4/d4e/dad/df8/f113 to d4/d17/d18/d43/d53/f133 0 2024-04-26T04:53:20.443 INFO:tasks.workunit.client.0.smithi077.stdout:3/768: dread d0/d7/f34 [0,4194304] 0 2024-04-26T04:53:20.443 INFO:tasks.workunit.client.0.smithi077.stdout:4/911: truncate d4/d17/d18/d43/d63/d80/f12a 367189 0 2024-04-26T04:53:20.444 INFO:tasks.workunit.client.0.smithi077.stdout:6/887: mkdir d8/d2a/d13a 0 2024-04-26T04:53:20.446 INFO:tasks.workunit.client.0.smithi077.stdout:6/888: fdatasync d8/d52/d53/d5e/fb0 0 2024-04-26T04:53:20.449 INFO:tasks.workunit.client.0.smithi077.stdout:3/769: dwrite d0/d7/d1f/d4d/f92 [0,4194304] 0 2024-04-26T04:53:20.449 INFO:tasks.workunit.client.0.smithi077.stdout:6/889: truncate d8/d52/d53/da7/df4/f128 509421 0 2024-04-26T04:53:20.450 INFO:tasks.workunit.client.0.smithi077.stdout:0/828: write d0/d19/d1e/d2f/f3b [3290649,47161] 0 2024-04-26T04:53:20.451 INFO:tasks.workunit.client.0.smithi077.stdout:4/912: link d4/l5f d4/d17/d18/d43/l134 0 2024-04-26T04:53:20.453 INFO:tasks.workunit.client.0.smithi077.stdout:4/913: write d4/d17/d7f/f9e [1553931,12958] 0 2024-04-26T04:53:20.454 INFO:tasks.workunit.client.0.smithi077.stdout:6/890: fsync d8/f22 0 2024-04-26T04:53:20.460 INFO:tasks.workunit.client.0.smithi077.stdout:3/770: symlink d0/dae/ded/lfb 0 2024-04-26T04:53:20.463 INFO:tasks.workunit.client.0.smithi077.stdout:4/914: mknod d4/d4e/dad/d8b/dba/c135 0 2024-04-26T04:53:20.463 INFO:tasks.workunit.client.0.smithi077.stdout:1/854: write f6 [3407913,59677] 0 2024-04-26T04:53:20.464 INFO:tasks.workunit.client.0.smithi077.stdout:3/771: dread d0/d7/d1f/d81/fa3 [0,4194304] 0 2024-04-26T04:53:20.465 INFO:tasks.workunit.client.0.smithi077.stdout:6/891: symlink d8/d52/d53/da7/dfa/l13b 0 2024-04-26T04:53:20.466 INFO:tasks.workunit.client.0.smithi077.stdout:6/892: read d8/d2a/d86/f94 [686546,25807] 0 2024-04-26T04:53:20.468 INFO:tasks.workunit.client.0.smithi077.stdout:4/915: mkdir d4/d17/d136 0 2024-04-26T04:53:20.468 INFO:tasks.workunit.client.0.smithi077.stdout:4/916: chown d4/d17/d18/d43/d53/fbe 107999829 1 2024-04-26T04:53:20.469 INFO:tasks.workunit.client.0.smithi077.stdout:1/855: mkdir d9/db/dd/d9c/dab/d117 0 2024-04-26T04:53:20.472 INFO:tasks.workunit.client.0.smithi077.stdout:1/856: symlink d9/db/dd/l118 0 2024-04-26T04:53:20.474 INFO:tasks.workunit.client.0.smithi077.stdout:3/772: rename d0/d7/dce/le7 to d0/d7/d1f/d81/dc5/dc7/lfc 0 2024-04-26T04:53:20.476 INFO:tasks.workunit.client.0.smithi077.stdout:3/773: chown d0/dae/ded/l73 91 1 2024-04-26T04:53:20.478 INFO:tasks.workunit.client.0.smithi077.stdout:1/857: mknod d9/db/dd/d9c/dab/d117/c119 0 2024-04-26T04:53:20.480 INFO:tasks.workunit.client.0.smithi077.stdout:3/774: creat d0/dae/ded/d29/db8/dec/ffd x:0 0 0 2024-04-26T04:53:20.480 INFO:tasks.workunit.client.0.smithi077.stdout:6/893: link l6 d8/d2a/l13c 0 2024-04-26T04:53:20.482 INFO:tasks.workunit.client.0.smithi077.stdout:1/858: mkdir d9/d90/d99/d9a/d11a 0 2024-04-26T04:53:20.483 INFO:tasks.workunit.client.0.smithi077.stdout:3/775: mknod d0/dae/cfe 0 2024-04-26T04:53:20.484 INFO:tasks.workunit.client.0.smithi077.stdout:1/859: dread - d9/d53/d6e/d7c/f82 zero size 2024-04-26T04:53:20.485 INFO:tasks.workunit.client.0.smithi077.stdout:1/860: chown d9/d90/d99/d9a/dd6 1185 1 2024-04-26T04:53:20.489 INFO:tasks.workunit.client.0.smithi077.stdout:6/894: link d8/d52/d53/d5e/l109 d8/d52/d53/d5e/da6/de6/l13d 0 2024-04-26T04:53:20.504 INFO:tasks.workunit.client.0.smithi077.stdout:3/776: write d0/dae/ded/f35 [159550,14627] 0 2024-04-26T04:53:20.509 INFO:tasks.workunit.client.0.smithi077.stdout:6/895: write d8/d1c/f79 [2748738,74117] 0 2024-04-26T04:53:20.512 INFO:tasks.workunit.client.0.smithi077.stdout:1/861: dwrite d9/db/dd/f74 [0,4194304] 0 2024-04-26T04:53:20.515 INFO:tasks.workunit.client.0.smithi077.stdout:1/862: chown d9/db/d2b/c65 23704868 1 2024-04-26T04:53:20.516 INFO:tasks.workunit.client.0.smithi077.stdout:1/863: readlink d9/db/dd/lc6 0 2024-04-26T04:53:20.535 INFO:tasks.workunit.client.0.smithi077.stdout:6/896: dwrite d8/d8f/fe5 [0,4194304] 0 2024-04-26T04:53:20.539 INFO:tasks.workunit.client.0.smithi077.stdout:6/897: symlink d8/d52/d53/d5e/l13e 0 2024-04-26T04:53:20.541 INFO:tasks.workunit.client.0.smithi077.stdout:6/898: symlink d8/d52/d11c/l13f 0 2024-04-26T04:53:20.543 INFO:tasks.workunit.client.0.smithi077.stdout:9/763: sync 2024-04-26T04:53:20.545 INFO:tasks.workunit.client.0.smithi077.stdout:9/764: dread - d9/dd/d1b/d2b/d32/d36/d7f/dae/fc9 zero size 2024-04-26T04:53:20.547 INFO:tasks.workunit.client.0.smithi077.stdout:9/765: chown d9/dd/d1b/d2b/d32/fce 1594 1 2024-04-26T04:53:20.549 INFO:tasks.workunit.client.0.smithi077.stdout:6/899: link d8/f48 d8/d1c/f140 0 2024-04-26T04:53:20.550 INFO:tasks.workunit.client.0.smithi077.stdout:3/777: dread d0/dae/ded/d29/f31 [4194304,4194304] 0 2024-04-26T04:53:20.550 INFO:tasks.workunit.client.0.smithi077.stdout:9/766: write d9/d3a/fea [620342,16814] 0 2024-04-26T04:53:20.551 INFO:tasks.workunit.client.0.smithi077.stdout:1/864: dread d9/d53/f6f [0,4194304] 0 2024-04-26T04:53:20.554 INFO:tasks.workunit.client.0.smithi077.stdout:6/900: read d8/f11 [1088992,64140] 0 2024-04-26T04:53:20.555 INFO:tasks.workunit.client.0.smithi077.stdout:3/778: symlink d0/d7/dce/lff 0 2024-04-26T04:53:20.556 INFO:tasks.workunit.client.0.smithi077.stdout:9/767: chown d9/dd/d3d/l80 119618 1 2024-04-26T04:53:20.559 INFO:tasks.workunit.client.0.smithi077.stdout:3/779: fdatasync d0/dae/ded/d2a/f40 0 2024-04-26T04:53:20.564 INFO:tasks.workunit.client.0.smithi077.stdout:6/901: mkdir d8/d52/d53/d141 0 2024-04-26T04:53:20.566 INFO:tasks.workunit.client.0.smithi077.stdout:9/768: creat d9/d45/dcd/ffe x:0 0 0 2024-04-26T04:53:20.570 INFO:tasks.workunit.client.0.smithi077.stdout:3/780: dwrite d0/d9/f84 [0,4194304] 0 2024-04-26T04:53:20.575 INFO:tasks.workunit.client.0.smithi077.stdout:3/781: creat d0/dae/ded/d29/db8/f100 x:0 0 0 2024-04-26T04:53:20.578 INFO:tasks.workunit.client.0.smithi077.stdout:6/902: dwrite d8/d52/da0/dad/ff3 [0,4194304] 0 2024-04-26T04:53:20.578 INFO:tasks.workunit.client.0.smithi077.stdout:3/782: fdatasync d0/dae/ded/d29/fc2 0 2024-04-26T04:53:20.581 INFO:tasks.workunit.client.0.smithi077.stdout:3/783: stat d0/d7/d1f/d81/fde 0 2024-04-26T04:53:20.583 INFO:tasks.workunit.client.0.smithi077.stdout:9/769: getdents d9/d35 0 2024-04-26T04:53:20.584 INFO:tasks.workunit.client.0.smithi077.stdout:9/770: write d9/f46 [4074401,21244] 0 2024-04-26T04:53:20.585 INFO:tasks.workunit.client.0.smithi077.stdout:3/784: creat d0/d7/d1f/d4d/dc6/f101 x:0 0 0 2024-04-26T04:53:20.588 INFO:tasks.workunit.client.0.smithi077.stdout:1/865: dread f6 [0,4194304] 0 2024-04-26T04:53:20.590 INFO:tasks.workunit.client.0.smithi077.stdout:9/771: rmdir d9/dd/d1b/d4e 39 2024-04-26T04:53:20.591 INFO:tasks.workunit.client.0.smithi077.stdout:9/772: chown d9/dd/dec 328429 1 2024-04-26T04:53:20.595 INFO:tasks.workunit.client.0.smithi077.stdout:1/866: dwrite d9/d90/fbf [0,4194304] 0 2024-04-26T04:53:20.595 INFO:tasks.workunit.client.0.smithi077.stdout:1/867: stat d9/db/dd/f49 0 2024-04-26T04:53:20.596 INFO:tasks.workunit.client.0.smithi077.stdout:9/773: mkdir d9/dd/d1b/d4e/dff 0 2024-04-26T04:53:20.599 INFO:tasks.workunit.client.0.smithi077.stdout:3/785: getdents d0 0 2024-04-26T04:53:20.605 INFO:tasks.workunit.client.0.smithi077.stdout:1/868: getdents d9/d90/d99 0 2024-04-26T04:53:20.606 INFO:tasks.workunit.client.0.smithi077.stdout:9/774: dwrite d9/d45/dcd/ffe [0,4194304] 0 2024-04-26T04:53:20.607 INFO:tasks.workunit.client.0.smithi077.stdout:1/869: write d9/d90/de0/d107/f114 [362920,28279] 0 2024-04-26T04:53:20.611 INFO:tasks.workunit.client.0.smithi077.stdout:1/870: dread - d9/db/dd/d68/d81/fb5 zero size 2024-04-26T04:53:20.613 INFO:tasks.workunit.client.0.smithi077.stdout:1/871: readlink d9/db/dd/lfb 0 2024-04-26T04:53:20.619 INFO:tasks.workunit.client.0.smithi077.stdout:9/775: dwrite d9/d7d/d94/ff6 [0,4194304] 0 2024-04-26T04:53:20.620 INFO:tasks.workunit.client.0.smithi077.stdout:9/776: fsync d9/d3a/d79/f33 0 2024-04-26T04:53:20.621 INFO:tasks.workunit.client.0.smithi077.stdout:1/872: dwrite d9/f43 [0,4194304] 0 2024-04-26T04:53:20.630 INFO:tasks.workunit.client.0.smithi077.stdout:3/786: dread d0/d7/d1f/d76/f7e [0,4194304] 0 2024-04-26T04:53:20.630 INFO:tasks.workunit.client.0.smithi077.stdout:3/787: chown d0/dae/ded/d2a/l2d 26979 1 2024-04-26T04:53:20.630 INFO:tasks.workunit.client.0.smithi077.stdout:1/873: link d9/d53/d8b/cae d9/db/dd/d9c/dab/de8/c11b 0 2024-04-26T04:53:20.632 INFO:tasks.workunit.client.0.smithi077.stdout:3/788: rmdir d0/d7/d1f/d81/dc5/dc7 39 2024-04-26T04:53:20.637 INFO:tasks.workunit.client.0.smithi077.stdout:1/874: getdents d9/db/dd/d34/d63 0 2024-04-26T04:53:20.641 INFO:tasks.workunit.client.0.smithi077.stdout:9/777: dread d9/dd/d3d/f5a [0,4194304] 0 2024-04-26T04:53:20.641 INFO:tasks.workunit.client.0.smithi077.stdout:3/789: dwrite d0/d7/d1f/d81/fbb [0,4194304] 0 2024-04-26T04:53:20.645 INFO:tasks.workunit.client.0.smithi077.stdout:9/778: fdatasync d9/dd/d3d/f88 0 2024-04-26T04:53:20.645 INFO:tasks.workunit.client.0.smithi077.stdout:9/779: fdatasync d9/f22 0 2024-04-26T04:53:20.649 INFO:tasks.workunit.client.0.smithi077.stdout:3/790: creat d0/d7/d1f/d81/dc5/dd1/f102 x:0 0 0 2024-04-26T04:53:20.652 INFO:tasks.workunit.client.0.smithi077.stdout:3/791: truncate d0/d7/d1f/d76/f9f 432045 0 2024-04-26T04:53:20.653 INFO:tasks.workunit.client.0.smithi077.stdout:3/792: write d0/dae/ded/d2a/f4e [4679965,26432] 0 2024-04-26T04:53:20.654 INFO:tasks.workunit.client.0.smithi077.stdout:3/793: chown d0/d7/d1f/d4d/dbe 122422475 1 2024-04-26T04:53:20.655 INFO:tasks.workunit.client.0.smithi077.stdout:3/794: dread - d0/d7/d9a/fd6 zero size 2024-04-26T04:53:20.656 INFO:tasks.workunit.client.0.smithi077.stdout:1/875: dread d9/d90/d99/fd5 [0,4194304] 0 2024-04-26T04:53:20.658 INFO:tasks.workunit.client.0.smithi077.stdout:1/876: mknod d9/d90/de0/c11c 0 2024-04-26T04:53:20.659 INFO:tasks.workunit.client.0.smithi077.stdout:3/795: mknod d0/d7/d1f/d81/dc5/dc7/c103 0 2024-04-26T04:53:20.660 INFO:tasks.workunit.client.0.smithi077.stdout:3/796: fdatasync d0/d7/d1f/f4a 0 2024-04-26T04:53:20.661 INFO:tasks.workunit.client.0.smithi077.stdout:3/797: write d0/d7/d1f/d4d/dc6/f101 [89616,76348] 0 2024-04-26T04:53:20.661 INFO:tasks.workunit.client.0.smithi077.stdout:1/877: mkdir d9/d53/d8b/d93/da8/db8/d11d 0 2024-04-26T04:53:20.661 INFO:tasks.workunit.client.0.smithi077.stdout:3/798: fdatasync d0/d7/f54 0 2024-04-26T04:53:20.662 INFO:tasks.workunit.client.0.smithi077.stdout:1/878: chown d9/db/dd/d68/d81/fb5 9327613 1 2024-04-26T04:53:20.663 INFO:tasks.workunit.client.0.smithi077.stdout:1/879: stat d9/db/dd/lc6 0 2024-04-26T04:53:20.664 INFO:tasks.workunit.client.0.smithi077.stdout:1/880: chown d9/db/d2b/d3f/l47 28828901 1 2024-04-26T04:53:20.664 INFO:tasks.workunit.client.0.smithi077.stdout:3/799: creat d0/dae/db6/f104 x:0 0 0 2024-04-26T04:53:20.664 INFO:tasks.workunit.client.0.smithi077.stdout:1/881: readlink d9/d53/d6e/l80 0 2024-04-26T04:53:20.668 INFO:tasks.workunit.client.0.smithi077.stdout:3/800: rename d0/d9/f78 to d0/d7/d1f/d4d/dbe/f105 0 2024-04-26T04:53:20.673 INFO:tasks.workunit.client.0.smithi077.stdout:3/801: mknod d0/dae/ded/d2a/d85/dd3/c106 0 2024-04-26T04:53:20.676 INFO:tasks.workunit.client.0.smithi077.stdout:1/882: creat d9/db/dd/d34/d4a/f11e x:0 0 0 2024-04-26T04:53:20.678 INFO:tasks.workunit.client.0.smithi077.stdout:3/802: dwrite d0/f19 [0,4194304] 0 2024-04-26T04:53:20.682 INFO:tasks.workunit.client.0.smithi077.stdout:3/803: creat d0/d7/d1f/d81/dc5/f107 x:0 0 0 2024-04-26T04:53:20.693 INFO:tasks.workunit.client.0.smithi077.stdout:3/804: dread d0/dae/ded/d29/f71 [0,4194304] 0 2024-04-26T04:53:20.698 INFO:tasks.workunit.client.0.smithi077.stdout:1/883: write d9/db/dd/d68/d81/f85 [967537,99973] 0 2024-04-26T04:53:20.702 INFO:tasks.workunit.client.0.smithi077.stdout:1/884: read d9/db/dd/d68/f7b [3875908,22915] 0 2024-04-26T04:53:20.703 INFO:tasks.workunit.client.0.smithi077.stdout:3/805: truncate d0/dae/ded/d2a/f4e 4200769 0 2024-04-26T04:53:20.703 INFO:tasks.workunit.client.0.smithi077.stdout:3/806: readlink d0/d7/d9a/db3/l4f 0 2024-04-26T04:53:20.705 INFO:tasks.workunit.client.0.smithi077.stdout:1/885: truncate d9/db/dd/f3e 1669910 0 2024-04-26T04:53:20.706 INFO:tasks.workunit.client.0.smithi077.stdout:3/807: unlink d0/d7/d1f/d81/dc5/cd9 0 2024-04-26T04:53:20.707 INFO:tasks.workunit.client.0.smithi077.stdout:3/808: write d0/d7/d9a/db7/ff9 [460728,115411] 0 2024-04-26T04:53:20.708 INFO:tasks.workunit.client.0.smithi077.stdout:3/809: readlink d0/dae/ded/d2a/l2d 0 2024-04-26T04:53:20.710 INFO:tasks.workunit.client.0.smithi077.stdout:1/886: dwrite d9/f73 [4194304,4194304] 0 2024-04-26T04:53:20.712 INFO:tasks.workunit.client.0.smithi077.stdout:1/887: stat d9/db/dd/d68/d81/fb5 0 2024-04-26T04:53:20.715 INFO:tasks.workunit.client.0.smithi077.stdout:3/810: dwrite d0/d7/d9a/db3/f22 [0,4194304] 0 2024-04-26T04:53:20.730 INFO:tasks.workunit.client.0.smithi077.stdout:3/811: link d0/d7/d1f/l86 d0/d7/d1f/d4d/l108 0 2024-04-26T04:53:20.733 INFO:tasks.workunit.client.0.smithi077.stdout:3/812: rename d0/dae/ded/f35 to d0/dae/ded/d2a/d85/f109 0 2024-04-26T04:53:20.736 INFO:tasks.workunit.client.0.smithi077.stdout:3/813: unlink d0/dae/lfa 0 2024-04-26T04:53:20.738 INFO:tasks.workunit.client.0.smithi077.stdout:3/814: dread - d0/dae/ded/d29/faf zero size 2024-04-26T04:53:20.740 INFO:tasks.workunit.client.0.smithi077.stdout:3/815: dread - d0/dae/ded/d29/fc2 zero size 2024-04-26T04:53:20.742 INFO:tasks.workunit.client.0.smithi077.stdout:1/888: dwrite d9/db/dd/d9c/dab/fb7 [0,4194304] 0 2024-04-26T04:53:20.745 INFO:tasks.workunit.client.0.smithi077.stdout:1/889: read d9/db/fd1 [2353258,72930] 0 2024-04-26T04:53:20.746 INFO:tasks.workunit.client.0.smithi077.stdout:3/816: creat d0/dae/ded/d2a/d85/dd3/f10a x:0 0 0 2024-04-26T04:53:20.751 INFO:tasks.workunit.client.0.smithi077.stdout:1/890: write d9/d53/d6e/f40 [4467162,107305] 0 2024-04-26T04:53:20.752 INFO:tasks.workunit.client.0.smithi077.stdout:1/891: write d9/db/f39 [4353725,101618] 0 2024-04-26T04:53:20.755 INFO:tasks.workunit.client.0.smithi077.stdout:3/817: creat d0/dae/ded/d2a/f10b x:0 0 0 2024-04-26T04:53:20.758 INFO:tasks.workunit.client.0.smithi077.stdout:8/883: sync 2024-04-26T04:53:20.759 INFO:tasks.workunit.client.0.smithi077.stdout:3/818: creat d0/dae/ded/d2a/d85/f10c x:0 0 0 2024-04-26T04:53:20.762 INFO:tasks.workunit.client.0.smithi077.stdout:1/892: truncate d9/d90/d99/fd5 2731206 0 2024-04-26T04:53:20.762 INFO:tasks.workunit.client.0.smithi077.stdout:1/893: chown d9/d53/d8b/d93/ca9 0 1 2024-04-26T04:53:20.797 INFO:tasks.workunit.client.0.smithi077.stdout:8/884: mkdir d4/df/d122 0 2024-04-26T04:53:20.798 INFO:tasks.workunit.client.0.smithi077.stdout:3/819: unlink d0/d7/d1f/d81/cf3 0 2024-04-26T04:53:20.806 INFO:tasks.workunit.client.0.smithi077.stdout:1/894: creat d9/db/dd/d9c/dab/f11f x:0 0 0 2024-04-26T04:53:20.807 INFO:tasks.workunit.client.0.smithi077.stdout:3/820: creat d0/d7/d1f/d81/f10d x:0 0 0 2024-04-26T04:53:20.808 INFO:tasks.workunit.client.0.smithi077.stdout:3/821: chown d0/d7/fad 320 1 2024-04-26T04:53:20.809 INFO:tasks.workunit.client.0.smithi077.stdout:3/822: read d0/d7/d1f/d4d/dc6/f101 [95078,130436] 0 2024-04-26T04:53:20.809 INFO:tasks.workunit.client.0.smithi077.stdout:3/823: fsync d0/d7/d1f/fcf 0 2024-04-26T04:53:20.810 INFO:tasks.workunit.client.0.smithi077.stdout:1/895: unlink d9/d90/fe6 0 2024-04-26T04:53:20.810 INFO:tasks.workunit.client.0.smithi077.stdout:3/824: dread - d0/dae/db6/f104 zero size 2024-04-26T04:53:20.811 INFO:tasks.workunit.client.0.smithi077.stdout:1/896: chown d9/db/dd/d34/d63 3437 1 2024-04-26T04:53:20.811 INFO:tasks.workunit.client.0.smithi077.stdout:3/825: dread - d0/dae/db6/f104 zero size 2024-04-26T04:53:20.821 INFO:tasks.workunit.client.0.smithi077.stdout:3/826: dwrite d0/d7/d1f/d4d/dbe/f105 [0,4194304] 0 2024-04-26T04:53:20.825 INFO:tasks.workunit.client.0.smithi077.stdout:8/885: truncate d4/df/d22/d2d/d2b/d33/fa7 367875 0 2024-04-26T04:53:20.829 INFO:tasks.workunit.client.0.smithi077.stdout:1/897: write d9/d53/f86 [2534674,26757] 0 2024-04-26T04:53:20.829 INFO:tasks.workunit.client.0.smithi077.stdout:3/827: dread d0/d7/d9a/db3/f44 [0,4194304] 0 2024-04-26T04:53:20.829 INFO:tasks.workunit.client.0.smithi077.stdout:8/886: fsync d4/df/dce/d5b/f11a 0 2024-04-26T04:53:20.831 INFO:tasks.workunit.client.0.smithi077.stdout:3/828: write d0/d7/d1f/d81/dc5/f107 [820277,124712] 0 2024-04-26T04:53:20.833 INFO:tasks.workunit.client.0.smithi077.stdout:3/829: write d0/d7/d1f/d4d/dbe/f105 [322714,123511] 0 2024-04-26T04:53:20.834 INFO:tasks.workunit.client.0.smithi077.stdout:3/830: fdatasync d0/f1 0 2024-04-26T04:53:20.837 INFO:tasks.workunit.client.0.smithi077.stdout:3/831: mknod d0/dae/db6/c10e 0 2024-04-26T04:53:20.839 INFO:tasks.workunit.client.0.smithi077.stdout:8/887: write d4/d87/f9b [1994836,90475] 0 2024-04-26T04:53:20.839 INFO:tasks.workunit.client.0.smithi077.stdout:1/898: getdents d9/d53/d8b/d93/da8/da7 0 2024-04-26T04:53:20.844 INFO:tasks.workunit.client.0.smithi077.stdout:8/888: dwrite d4/f5 [0,4194304] 0 2024-04-26T04:53:20.845 INFO:tasks.workunit.client.0.smithi077.stdout:1/899: symlink d9/db/dd/d9c/dab/d117/l120 0 2024-04-26T04:53:20.847 INFO:tasks.workunit.client.0.smithi077.stdout:1/900: rmdir d9/d53/d6e 39 2024-04-26T04:53:20.848 INFO:tasks.workunit.client.0.smithi077.stdout:3/832: write d0/d7/d1f/d4d/f56 [1349279,119217] 0 2024-04-26T04:53:20.852 INFO:tasks.workunit.client.0.smithi077.stdout:1/901: rename f6 to d9/d90/de0/de4/f121 0 2024-04-26T04:53:20.856 INFO:tasks.workunit.client.0.smithi077.stdout:3/833: link d0/d7/fd5 d0/d7/d9a/db7/f10f 0 2024-04-26T04:53:20.857 INFO:tasks.workunit.client.0.smithi077.stdout:3/834: readlink d0/d7/d9a/le1 0 2024-04-26T04:53:20.862 INFO:tasks.workunit.client.0.smithi077.stdout:3/835: dwrite d0/f1 [4194304,4194304] 0 2024-04-26T04:53:20.864 INFO:tasks.workunit.client.0.smithi077.stdout:3/836: symlink d0/l110 0 2024-04-26T04:53:20.873 INFO:tasks.workunit.client.0.smithi077.stdout:8/889: dread d4/df/dce/d5b/f7f [0,4194304] 0 2024-04-26T04:53:20.874 INFO:tasks.workunit.client.0.smithi077.stdout:8/890: fdatasync d4/f5 0 2024-04-26T04:53:20.881 INFO:tasks.workunit.client.0.smithi077.stdout:8/891: getdents d4/df/d22/d2d/d104 0 2024-04-26T04:53:20.882 INFO:tasks.workunit.client.0.smithi077.stdout:8/892: dread - d4/d17/ddc/f10d zero size 2024-04-26T04:53:20.887 INFO:tasks.workunit.client.0.smithi077.stdout:1/902: dread d9/db/dd/d34/f38 [0,4194304] 0 2024-04-26T04:53:20.887 INFO:tasks.workunit.client.0.smithi077.stdout:3/837: link d0/d7/d1f/d4d/f56 d0/d7/d9a/db7/ddb/f111 0 2024-04-26T04:53:20.889 INFO:tasks.workunit.client.0.smithi077.stdout:3/838: chown d0/dae/ded/d29/l66 0 1 2024-04-26T04:53:20.890 INFO:tasks.workunit.client.0.smithi077.stdout:3/839: stat d0/d7/d1f/d76/f7e 0 2024-04-26T04:53:20.891 INFO:tasks.workunit.client.0.smithi077.stdout:1/903: creat d9/d53/d8b/d93/da8/dd7/ddc/f122 x:0 0 0 2024-04-26T04:53:20.892 INFO:tasks.workunit.client.0.smithi077.stdout:3/840: dread d0/dae/ded/d29/f71 [0,4194304] 0 2024-04-26T04:53:20.893 INFO:tasks.workunit.client.0.smithi077.stdout:1/904: creat d9/d53/d8b/f123 x:0 0 0 2024-04-26T04:53:20.894 INFO:tasks.workunit.client.0.smithi077.stdout:1/905: chown d9/d90/de0/de4/f121 0 1 2024-04-26T04:53:20.895 INFO:tasks.workunit.client.0.smithi077.stdout:1/906: fsync d9/db/faf 0 2024-04-26T04:53:20.898 INFO:tasks.workunit.client.0.smithi077.stdout:1/907: write d9/db/dd/d34/d63/f77 [3221291,18606] 0 2024-04-26T04:53:20.900 INFO:tasks.workunit.client.0.smithi077.stdout:3/841: rename d0/d7/d1f/d76/f8d to d0/d7/d1f/d81/dc5/f112 0 2024-04-26T04:53:20.900 INFO:tasks.workunit.client.0.smithi077.stdout:8/893: dread d4/d69/fa2 [4194304,4194304] 0 2024-04-26T04:53:20.906 INFO:tasks.workunit.client.0.smithi077.stdout:1/908: creat d9/db/d2b/f124 x:0 0 0 2024-04-26T04:53:20.910 INFO:tasks.workunit.client.0.smithi077.stdout:3/842: unlink d0/d7/d1f/l86 0 2024-04-26T04:53:20.910 INFO:tasks.workunit.client.0.smithi077.stdout:3/843: read - d0/dae/ded/d29/db8/f100 zero size 2024-04-26T04:53:20.919 INFO:tasks.workunit.client.0.smithi077.stdout:3/844: dwrite d0/fb [8388608,4194304] 0 2024-04-26T04:53:20.922 INFO:tasks.workunit.client.0.smithi077.stdout:1/909: rename d9/db/dd/d68/d81/fb5 to d9/db/f125 0 2024-04-26T04:53:20.929 INFO:tasks.workunit.client.0.smithi077.stdout:8/894: write d4/df/dc6/fd3 [168872,33587] 0 2024-04-26T04:53:20.931 INFO:tasks.workunit.client.0.smithi077.stdout:1/910: creat d9/d53/d8b/d10c/f126 x:0 0 0 2024-04-26T04:53:20.935 INFO:tasks.workunit.client.0.smithi077.stdout:8/895: rmdir d4/df/dce 39 2024-04-26T04:53:20.937 INFO:tasks.workunit.client.0.smithi077.stdout:3/845: write d0/dae/ded/f3b [4118702,56580] 0 2024-04-26T04:53:20.938 INFO:tasks.workunit.client.0.smithi077.stdout:3/846: chown d0/dae/ded/d2a/f40 401393365 1 2024-04-26T04:53:20.947 INFO:tasks.workunit.client.0.smithi077.stdout:1/911: getdents d9/d53/d8b/d93/da8 0 2024-04-26T04:53:20.951 INFO:tasks.workunit.client.0.smithi077.stdout:1/912: dread d9/db/dd/d34/d63/f104 [0,4194304] 0 2024-04-26T04:53:20.951 INFO:tasks.workunit.client.0.smithi077.stdout:3/847: creat d0/d7/d1f/d4d/f113 x:0 0 0 2024-04-26T04:53:20.953 INFO:tasks.workunit.client.0.smithi077.stdout:8/896: creat d4/df/d22/f123 x:0 0 0 2024-04-26T04:53:20.954 INFO:tasks.workunit.client.0.smithi077.stdout:8/897: fdatasync d4/df/d22/d2d/d2b/d33/dd8/d117/d120/f11e 0 2024-04-26T04:53:20.954 INFO:tasks.workunit.client.0.smithi077.stdout:3/848: write d0/f1b [2348348,23155] 0 2024-04-26T04:53:20.958 INFO:tasks.workunit.client.0.smithi077.stdout:1/913: unlink d9/d53/d6e/c48 0 2024-04-26T04:53:20.959 INFO:tasks.workunit.client.0.smithi077.stdout:8/898: fdatasync d4/d17/d90/faf 0 2024-04-26T04:53:20.965 INFO:tasks.workunit.client.0.smithi077.stdout:8/899: dread d4/df/dce/d5b/f7f [0,4194304] 0 2024-04-26T04:53:20.965 INFO:tasks.workunit.client.0.smithi077.stdout:1/914: rmdir d9/d53/d6e 39 2024-04-26T04:53:20.969 INFO:tasks.workunit.client.0.smithi077.stdout:1/915: rmdir d9/d53/dba 39 2024-04-26T04:53:20.972 INFO:tasks.workunit.client.0.smithi077.stdout:8/900: rmdir d4/df/d18/d56 39 2024-04-26T04:53:20.974 INFO:tasks.workunit.client.0.smithi077.stdout:1/916: unlink d9/d53/dc1/cce 0 2024-04-26T04:53:20.980 INFO:tasks.workunit.client.0.smithi077.stdout:1/917: dread d9/d53/d8b/d93/da8/db8/fc9 [0,4194304] 0 2024-04-26T04:53:20.981 INFO:tasks.workunit.client.0.smithi077.stdout:1/918: chown d9/d53/f86 430 1 2024-04-26T04:53:20.983 INFO:tasks.workunit.client.0.smithi077.stdout:8/901: getdents d4/df/d22/d80/d86/de3 0 2024-04-26T04:53:20.985 INFO:tasks.workunit.client.0.smithi077.stdout:1/919: stat d9/d53/d6e/f30 0 2024-04-26T04:53:20.987 INFO:tasks.workunit.client.0.smithi077.stdout:8/902: unlink d4/df/dce/d5b/cea 0 2024-04-26T04:53:20.988 INFO:tasks.workunit.client.0.smithi077.stdout:1/920: dread d9/d53/d8b/d93/da8/db8/fc5 [0,4194304] 0 2024-04-26T04:53:20.990 INFO:tasks.workunit.client.0.smithi077.stdout:1/921: mkdir d9/db/dd/d9c/d127 0 2024-04-26T04:53:20.993 INFO:tasks.workunit.client.0.smithi077.stdout:1/922: mknod d9/d53/d8b/d93/da8/db8/c128 0 2024-04-26T04:53:20.995 INFO:tasks.workunit.client.0.smithi077.stdout:8/903: dwrite d4/df/dc6/fd3 [0,4194304] 0 2024-04-26T04:53:20.997 INFO:tasks.workunit.client.0.smithi077.stdout:8/904: write d4/df/dce/d5b/f11a [625099,89020] 0 2024-04-26T04:53:20.998 INFO:tasks.workunit.client.0.smithi077.stdout:8/905: write d4/df/dce/f70 [2214015,116356] 0 2024-04-26T04:53:20.999 INFO:tasks.workunit.client.0.smithi077.stdout:1/923: dread d9/db/dd/f74 [0,4194304] 0 2024-04-26T04:53:21.000 INFO:tasks.workunit.client.0.smithi077.stdout:8/906: write d4/df/d18/f26 [428895,101099] 0 2024-04-26T04:53:21.000 INFO:tasks.workunit.client.0.smithi077.stdout:1/924: write d9/d90/fad [2556679,1971] 0 2024-04-26T04:53:21.002 INFO:tasks.workunit.client.0.smithi077.stdout:8/907: truncate d4/df/d22/d80/fae 4686651 0 2024-04-26T04:53:21.004 INFO:tasks.workunit.client.0.smithi077.stdout:1/925: mknod d9/db/dd/d9c/dab/d117/c129 0 2024-04-26T04:53:21.005 INFO:tasks.workunit.client.0.smithi077.stdout:8/908: symlink d4/df/d22/d80/d86/d103/l124 0 2024-04-26T04:53:21.017 INFO:tasks.workunit.client.0.smithi077.stdout:8/909: write d4/fd [1068675,75773] 0 2024-04-26T04:53:21.022 INFO:tasks.workunit.client.0.smithi077.stdout:1/926: dwrite d9/d90/f83 [0,4194304] 0 2024-04-26T04:53:21.024 INFO:tasks.workunit.client.0.smithi077.stdout:8/910: dread d4/d17/fc5 [0,4194304] 0 2024-04-26T04:53:21.026 INFO:tasks.workunit.client.0.smithi077.stdout:8/911: write d4/df/d22/d2d/d2b/d33/dd8/d117/f11d [960540,80184] 0 2024-04-26T04:53:21.027 INFO:tasks.workunit.client.0.smithi077.stdout:1/927: write d9/db/dd/d34/d4a/f11e [216229,93425] 0 2024-04-26T04:53:21.044 INFO:tasks.workunit.client.0.smithi077.stdout:1/928: dwrite d9/f24 [0,4194304] 0 2024-04-26T04:53:21.079 INFO:tasks.workunit.client.0.smithi077.stdout:1/929: chown d9/db/dd/d9c/dab/de8/c112 714 1 2024-04-26T04:53:21.084 INFO:tasks.workunit.client.0.smithi077.stdout:1/930: link d9/db/dd/c16 d9/db/dd/d34/d4a/c12a 0 2024-04-26T04:53:21.085 INFO:tasks.workunit.client.0.smithi077.stdout:1/931: write d9/d53/d8b/d10c/f126 [663404,73952] 0 2024-04-26T04:53:21.088 INFO:tasks.workunit.client.0.smithi077.stdout:1/932: creat d9/db/dd/d9c/dab/f12b x:0 0 0 2024-04-26T04:53:21.438 INFO:tasks.workunit.client.0.smithi077.stdout:7/867: sync 2024-04-26T04:53:21.465 INFO:tasks.workunit.client.0.smithi077.stdout:7/868: truncate da/f87 1875074 0 2024-04-26T04:53:21.471 INFO:tasks.workunit.client.0.smithi077.stdout:7/869: dwrite da/d6b/d63/d64/fbc [0,4194304] 0 2024-04-26T04:53:21.479 INFO:tasks.workunit.client.0.smithi077.stdout:7/870: link da/d13/d22/f99 da/d6b/d63/f125 0 2024-04-26T04:53:21.481 INFO:tasks.workunit.client.0.smithi077.stdout:7/871: creat da/d15/d1a/d39/f126 x:0 0 0 2024-04-26T04:53:21.483 INFO:tasks.workunit.client.0.smithi077.stdout:7/872: write da/d13/d22/d7d/d80/d113/f114 [78200,136] 0 2024-04-26T04:53:21.484 INFO:tasks.workunit.client.0.smithi077.stdout:7/873: write da/d13/d118/f11d [970001,50474] 0 2024-04-26T04:53:21.487 INFO:tasks.workunit.client.0.smithi077.stdout:7/874: readlink da/d13/d22/d5d/d9f/da7/de0/le7 0 2024-04-26T04:53:21.491 INFO:tasks.workunit.client.0.smithi077.stdout:7/875: read da/d6b/d63/f125 [4831547,24760] 0 2024-04-26T04:53:21.510 INFO:tasks.workunit.client.0.smithi077.stdout:7/876: dwrite da/d13/d22/d7d/f85 [0,4194304] 0 2024-04-26T04:53:21.517 INFO:tasks.workunit.client.0.smithi077.stdout:7/877: truncate da/d6b/d63/f125 7034671 0 2024-04-26T04:53:21.519 INFO:tasks.workunit.client.0.smithi077.stdout:7/878: fdatasync da/d13/d22/d5d/d9f/ddd/f121 0 2024-04-26T04:53:21.526 INFO:tasks.workunit.client.0.smithi077.stdout:7/879: getdents da/d6b/d63/d64 0 2024-04-26T04:53:21.534 INFO:tasks.workunit.client.0.smithi077.stdout:7/880: write da/d13/d22/d34/f51 [3051256,91734] 0 2024-04-26T04:53:21.536 INFO:tasks.workunit.client.0.smithi077.stdout:7/881: symlink da/d15/d1a/d111/l127 0 2024-04-26T04:53:21.538 INFO:tasks.workunit.client.0.smithi077.stdout:7/882: symlink da/d13/d22/d5d/d9f/da7/de0/l128 0 2024-04-26T04:53:21.541 INFO:tasks.workunit.client.0.smithi077.stdout:7/883: creat da/f129 x:0 0 0 2024-04-26T04:53:21.548 INFO:tasks.workunit.client.0.smithi077.stdout:7/884: dwrite da/d15/f94 [0,4194304] 0 2024-04-26T04:53:21.571 INFO:tasks.workunit.client.0.smithi077.stdout:5/834: sync 2024-04-26T04:53:21.571 INFO:tasks.workunit.client.0.smithi077.stdout:2/971: sync 2024-04-26T04:53:21.574 INFO:tasks.workunit.client.0.smithi077.stdout:2/972: mkdir d9/dc/d5f/d78/d14f 0 2024-04-26T04:53:21.580 INFO:tasks.workunit.client.0.smithi077.stdout:5/835: creat d1/d42/d62/d6b/deb/f115 x:0 0 0 2024-04-26T04:53:21.586 INFO:tasks.workunit.client.0.smithi077.stdout:2/973: rename d9/dc/d1d/d34/d39/d63/f104 to d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/dd0/f150 0 2024-04-26T04:53:21.598 INFO:tasks.workunit.client.0.smithi077.stdout:2/974: creat d9/dc/d5f/d8e/da6/dbd/d112/f151 x:0 0 0 2024-04-26T04:53:21.598 INFO:tasks.workunit.client.0.smithi077.stdout:5/836: link d1/d12/l52 d1/d42/da4/d25/l116 0 2024-04-26T04:53:21.601 INFO:tasks.workunit.client.0.smithi077.stdout:2/975: creat d9/d15/d91/f152 x:0 0 0 2024-04-26T04:53:21.602 INFO:tasks.workunit.client.0.smithi077.stdout:2/976: chown d9/f2c 107891852 1 2024-04-26T04:53:21.603 INFO:tasks.workunit.client.0.smithi077.stdout:2/977: chown d9/dc/d1d/d34/l14e 59 1 2024-04-26T04:53:21.604 INFO:tasks.workunit.client.0.smithi077.stdout:2/978: write d9/dc/d1d/f13d [345952,103914] 0 2024-04-26T04:53:21.611 INFO:tasks.workunit.client.0.smithi077.stdout:2/979: creat d9/dc/d1d/d1e/d77/d11b/d14b/f153 x:0 0 0 2024-04-26T04:53:21.612 INFO:tasks.workunit.client.0.smithi077.stdout:5/837: dwrite d1/d12/d87/fc3 [0,4194304] 0 2024-04-26T04:53:21.618 INFO:tasks.workunit.client.0.smithi077.stdout:2/980: truncate d9/dc/d1d/d34/d39/d63/f9c 4365161 0 2024-04-26T04:53:21.622 INFO:tasks.workunit.client.0.smithi077.stdout:2/981: mknod d9/dc/d5f/d8e/db9/c154 0 2024-04-26T04:53:21.626 INFO:tasks.workunit.client.0.smithi077.stdout:5/838: dwrite d1/d42/da4/f17 [0,4194304] 0 2024-04-26T04:53:21.626 INFO:tasks.workunit.client.0.smithi077.stdout:2/982: creat d9/dc/d1d/d122/f155 x:0 0 0 2024-04-26T04:53:21.629 INFO:tasks.workunit.client.0.smithi077.stdout:2/983: truncate d9/dc/d1d/d1e/d77/d11b/d14b/f153 308900 0 2024-04-26T04:53:21.632 INFO:tasks.workunit.client.0.smithi077.stdout:5/839: rename d1/d30/d33/d66/l79 to d1/d42/d62/dc8/dd5/df8/dfd/l117 0 2024-04-26T04:53:21.634 INFO:tasks.workunit.client.0.smithi077.stdout:2/984: mknod d9/dc/d1d/d34/d39/d63/df6/d12b/c156 0 2024-04-26T04:53:21.635 INFO:tasks.workunit.client.0.smithi077.stdout:5/840: symlink d1/d8d/l118 0 2024-04-26T04:53:21.636 INFO:tasks.workunit.client.0.smithi077.stdout:5/841: chown d1/d42/dac/fcf 12768 1 2024-04-26T04:53:21.637 INFO:tasks.workunit.client.0.smithi077.stdout:2/985: symlink d9/dc/d5f/d78/l157 0 2024-04-26T04:53:21.642 INFO:tasks.workunit.client.0.smithi077.stdout:2/986: creat d9/dc/d1d/d34/d39/f158 x:0 0 0 2024-04-26T04:53:21.644 INFO:tasks.workunit.client.0.smithi077.stdout:2/987: write d9/dc/d5f/d8e/db9/f10a [935270,113765] 0 2024-04-26T04:53:21.648 INFO:tasks.workunit.client.0.smithi077.stdout:5/842: dread d1/d42/d62/d6b/f85 [0,4194304] 0 2024-04-26T04:53:21.652 INFO:tasks.workunit.client.0.smithi077.stdout:5/843: dread d1/d30/d33/fe4 [0,4194304] 0 2024-04-26T04:53:21.653 INFO:tasks.workunit.client.0.smithi077.stdout:5/844: fdatasync d1/d42/da4/d25/f104 0 2024-04-26T04:53:21.657 INFO:tasks.workunit.client.0.smithi077.stdout:2/988: write d9/dc/d1d/d1e/f21 [8345962,86941] 0 2024-04-26T04:53:21.660 INFO:tasks.workunit.client.0.smithi077.stdout:5/845: getdents d1/d12/d45 0 2024-04-26T04:53:21.662 INFO:tasks.workunit.client.0.smithi077.stdout:2/989: symlink d9/dc/d1d/d34/d39/d63/d79/d5b/db8/l159 0 2024-04-26T04:53:21.662 INFO:tasks.workunit.client.0.smithi077.stdout:2/990: fdatasync d9/d15/f9f 0 2024-04-26T04:53:21.663 INFO:tasks.workunit.client.0.smithi077.stdout:5/846: creat d1/d42/d62/d6b/deb/f119 x:0 0 0 2024-04-26T04:53:21.664 INFO:tasks.workunit.client.0.smithi077.stdout:2/991: read d9/dc/d1d/d34/d39/d63/d79/f3f [31480,80209] 0 2024-04-26T04:53:21.675 INFO:tasks.workunit.client.0.smithi077.stdout:5/847: write d1/d30/d33/f76 [659886,121912] 0 2024-04-26T04:53:21.678 INFO:tasks.workunit.client.0.smithi077.stdout:2/992: getdents d9/dc/d1d/d122 0 2024-04-26T04:53:21.682 INFO:tasks.workunit.client.0.smithi077.stdout:2/993: unlink d9/dc/d1d/d1e/d77/df9/f110 0 2024-04-26T04:53:21.683 INFO:tasks.workunit.client.0.smithi077.stdout:2/994: fdatasync d9/dc/d1d/d34/ffc 0 2024-04-26T04:53:21.687 INFO:tasks.workunit.client.0.smithi077.stdout:2/995: mknod d9/dc/d1d/d34/d39/d63/df6/c15a 0 2024-04-26T04:53:21.688 INFO:tasks.workunit.client.0.smithi077.stdout:5/848: dwrite d1/d8/f49 [0,4194304] 0 2024-04-26T04:53:21.692 INFO:tasks.workunit.client.0.smithi077.stdout:5/849: rmdir d1/d42/d62/d6b/de8 39 2024-04-26T04:53:21.692 INFO:tasks.workunit.client.0.smithi077.stdout:5/850: fdatasync d1/d12/d28/ffc 0 2024-04-26T04:53:21.693 INFO:tasks.workunit.client.0.smithi077.stdout:5/851: write d1/d12/f1f [1386863,79051] 0 2024-04-26T04:53:21.700 INFO:tasks.workunit.client.0.smithi077.stdout:5/852: fsync d1/d42/da4/d25/d31/f71 0 2024-04-26T04:53:21.705 INFO:tasks.workunit.client.0.smithi077.stdout:5/853: rename d1/d8d/da7/f10c to d1/d8d/f11a 0 2024-04-26T04:53:21.706 INFO:tasks.workunit.client.0.smithi077.stdout:2/996: write d9/dc/d1d/d34/df8/dd1/dc3/dc6/dc9/f103 [131311,23471] 0 2024-04-26T04:53:21.707 INFO:tasks.workunit.client.0.smithi077.stdout:5/854: mknod d1/d42/da4/d25/d31/c11b 0 2024-04-26T04:53:21.707 INFO:tasks.workunit.client.0.smithi077.stdout:2/997: stat d9/dc/d1d/d1e/d77/d11b/d14b/f153 0 2024-04-26T04:53:21.710 INFO:tasks.workunit.client.0.smithi077.stdout:5/855: truncate d1/d12/d87/fe5 304028 0 2024-04-26T04:53:21.711 INFO:tasks.workunit.client.0.smithi077.stdout:2/998: mknod d9/dc/d1d/d34/d39/d63/d79/d5b/c15b 0 2024-04-26T04:53:21.712 INFO:tasks.workunit.client.0.smithi077.stdout:5/856: truncate d1/d42/d62/dc8/dd5/f102 278034 0 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:8/912: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:9/780: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:0/829: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:4/917: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:6/903: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:3/849: sync 2024-04-26T04:53:21.714 INFO:tasks.workunit.client.0.smithi077.stdout:1/933: sync 2024-04-26T04:53:21.716 INFO:tasks.workunit.client.0.smithi077.stdout:2/999: mkdir d9/d15/d91/d106/d140/d15c 0 2024-04-26T04:53:21.716 INFO:tasks.workunit.client.0.smithi077.stdout:1/934: write d9/f43 [820186,57876] 0 2024-04-26T04:53:21.722 INFO:tasks.workunit.client.0.smithi077.stdout:6/904: creat d8/d1c/f142 x:0 0 0 2024-04-26T04:53:21.725 INFO:tasks.workunit.client.0.smithi077.stdout:4/918: fdatasync d4/d4e/ff4 0 2024-04-26T04:53:21.726 INFO:tasks.workunit.client.0.smithi077.stdout:0/830: dread - d0/fdb zero size 2024-04-26T04:53:21.733 INFO:tasks.workunit.client.0.smithi077.stdout:3/850: dread d0/feb [0,4194304] 0 2024-04-26T04:53:21.734 INFO:tasks.workunit.client.0.smithi077.stdout:6/905: chown d8/d52/d53/cdb 123 1 2024-04-26T04:53:21.736 INFO:tasks.workunit.client.0.smithi077.stdout:8/913: dwrite d4/df/d22/d2d/f1b [0,4194304] 0 2024-04-26T04:53:21.736 INFO:tasks.workunit.client.0.smithi077.stdout:4/919: creat d4/d17/d18/d43/d53/d64/f137 x:0 0 0 2024-04-26T04:53:21.738 INFO:tasks.workunit.client.0.smithi077.stdout:9/781: rename d9/dd/c2a to d9/dd/d3d/d7c/c100 0 2024-04-26T04:53:21.743 INFO:tasks.workunit.client.0.smithi077.stdout:5/857: creat d1/d42/da4/d25/d31/d3c/f11c x:0 0 0 2024-04-26T04:53:21.748 INFO:tasks.workunit.client.0.smithi077.stdout:4/920: dread - d4/d17/d18/d43/d53/fe3 zero size 2024-04-26T04:53:21.750 INFO:tasks.workunit.client.0.smithi077.stdout:9/782: fdatasync d9/d43/f56 0 2024-04-26T04:53:21.751 INFO:tasks.workunit.client.0.smithi077.stdout:3/851: write d0/dae/ded/f21 [2058905,55945] 0 2024-04-26T04:53:21.752 INFO:tasks.workunit.client.0.smithi077.stdout:3/852: dread - d0/d7/d1f/d81/fda zero size 2024-04-26T04:53:21.753 INFO:tasks.workunit.client.0.smithi077.stdout:8/914: rename d4/d87/l112 to d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/dde/l125 0 2024-04-26T04:53:21.757 INFO:tasks.workunit.client.0.smithi077.stdout:0/831: link d0/d2c/d109/dd7/f107 d0/d47/d64/d97/dba/dc4/f11b 0 2024-04-26T04:53:21.759 INFO:tasks.workunit.client.0.smithi077.stdout:5/858: chown d1/d42/da4/d25/d31/f71 7 1 2024-04-26T04:53:21.761 INFO:tasks.workunit.client.0.smithi077.stdout:4/921: mkdir d4/d17/d18/d43/d51/db4/d15/d1f/d25/d138 0 2024-04-26T04:53:21.763 INFO:tasks.workunit.client.0.smithi077.stdout:4/922: readlink d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/l11e 0 2024-04-26T04:53:21.766 INFO:tasks.workunit.client.0.smithi077.stdout:3/853: unlink d0/d7/d1f/d81/dc5/cf5 0 2024-04-26T04:53:21.766 INFO:tasks.workunit.client.0.smithi077.stdout:6/906: write d8/fe [1371971,70223] 0 2024-04-26T04:53:21.768 INFO:tasks.workunit.client.0.smithi077.stdout:1/935: getdents d9 0 2024-04-26T04:53:21.770 INFO:tasks.workunit.client.0.smithi077.stdout:0/832: read - d0/d47/ff0 zero size 2024-04-26T04:53:21.771 INFO:tasks.workunit.client.0.smithi077.stdout:0/833: fdatasync d0/d19/d69/f84 0 2024-04-26T04:53:21.772 INFO:tasks.workunit.client.0.smithi077.stdout:7/885: sync 2024-04-26T04:53:21.775 INFO:tasks.workunit.client.0.smithi077.stdout:1/936: dwrite d9/d53/d6e/f30 [0,4194304] 0 2024-04-26T04:53:21.776 INFO:tasks.workunit.client.0.smithi077.stdout:9/783: unlink d9/d45/f71 0 2024-04-26T04:53:21.779 INFO:tasks.workunit.client.0.smithi077.stdout:9/784: read d9/dd/d1b/d2b/f83 [290960,98268] 0 2024-04-26T04:53:21.782 INFO:tasks.workunit.client.0.smithi077.stdout:1/937: dread d9/db/dd/d68/d81/f85 [0,4194304] 0 2024-04-26T04:53:21.783 INFO:tasks.workunit.client.0.smithi077.stdout:5/859: symlink d1/d30/l11d 0 2024-04-26T04:53:21.786 INFO:tasks.workunit.client.0.smithi077.stdout:9/785: dwrite d9/f62 [0,4194304] 0 2024-04-26T04:53:21.787 INFO:tasks.workunit.client.0.smithi077.stdout:9/786: chown d9/dd/d1b/d2b/f38 705389 1 2024-04-26T04:53:21.790 INFO:tasks.workunit.client.0.smithi077.stdout:7/886: rename da/d13/d22/f46 to da/d13/d22/d34/f12a 0 2024-04-26T04:53:21.791 INFO:tasks.workunit.client.0.smithi077.stdout:7/887: dread - da/d6b/d63/ff5 zero size 2024-04-26T04:53:21.793 INFO:tasks.workunit.client.0.smithi077.stdout:7/888: truncate da/d15/dc6/fff 920592 0 2024-04-26T04:53:21.794 INFO:tasks.workunit.client.0.smithi077.stdout:8/915: creat d4/f126 x:0 0 0 2024-04-26T04:53:21.795 INFO:tasks.workunit.client.0.smithi077.stdout:8/916: write d4/df/d22/d2d/f1b [1171354,64211] 0 2024-04-26T04:53:21.803 INFO:tasks.workunit.client.0.smithi077.stdout:4/923: creat d4/d17/d18/f139 x:0 0 0 2024-04-26T04:53:21.805 INFO:tasks.workunit.client.0.smithi077.stdout:5/860: truncate d1/d30/d33/fe4 613067 0 2024-04-26T04:53:21.806 INFO:tasks.workunit.client.0.smithi077.stdout:6/907: dwrite d8/d52/d53/da7/f113 [0,4194304] 0 2024-04-26T04:53:21.808 INFO:tasks.workunit.client.0.smithi077.stdout:7/889: rmdir da/d13/d22/d5d/dbd/df2 39 2024-04-26T04:53:21.808 INFO:tasks.workunit.client.0.smithi077.stdout:0/834: dwrite d0/d47/d93/d94/fcd [0,4194304] 0 2024-04-26T04:53:21.812 INFO:tasks.workunit.client.0.smithi077.stdout:6/908: chown d8/d52/d53/d5e/da6/d133/d139/f130 25078 1 2024-04-26T04:53:21.814 INFO:tasks.workunit.client.0.smithi077.stdout:8/917: symlink d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/l127 0 2024-04-26T04:53:21.817 INFO:tasks.workunit.client.0.smithi077.stdout:9/787: unlink d9/dd/le 0 2024-04-26T04:53:21.822 INFO:tasks.workunit.client.0.smithi077.stdout:4/924: dread d4/d17/d18/d43/d51/db4/fa6 [0,4194304] 0 2024-04-26T04:53:21.825 INFO:tasks.workunit.client.0.smithi077.stdout:7/890: write da/d15/d42/f77 [1361234,71263] 0 2024-04-26T04:53:21.826 INFO:tasks.workunit.client.0.smithi077.stdout:3/854: getdents d0/dae/ded/d2a/d85/dd3 0 2024-04-26T04:53:21.830 INFO:tasks.workunit.client.0.smithi077.stdout:6/909: mknod d8/d52/d53/d5e/da6/de6/c143 0 2024-04-26T04:53:21.834 INFO:tasks.workunit.client.0.smithi077.stdout:5/861: dread d1/d12/f50 [0,4194304] 0 2024-04-26T04:53:21.838 INFO:tasks.workunit.client.0.smithi077.stdout:4/925: read d4/d17/d18/d43/d51/db4/d15/f50 [3870296,72750] 0 2024-04-26T04:53:21.839 INFO:tasks.workunit.client.0.smithi077.stdout:4/926: write d4/d17/d18/d43/d51/db4/d15/ffa [1136834,109737] 0 2024-04-26T04:53:21.844 INFO:tasks.workunit.client.0.smithi077.stdout:3/855: fsync d0/d7/d9a/db7/ddb/f111 0 2024-04-26T04:53:21.845 INFO:tasks.workunit.client.0.smithi077.stdout:6/910: truncate d8/d2a/d86/f106 4339537 0 2024-04-26T04:53:21.848 INFO:tasks.workunit.client.0.smithi077.stdout:8/918: dwrite d4/df/d22/d2d/d2b/f54 [0,4194304] 0 2024-04-26T04:53:21.853 INFO:tasks.workunit.client.0.smithi077.stdout:9/788: dwrite d9/dd/d1b/d2b/d32/f39 [0,4194304] 0 2024-04-26T04:53:21.855 INFO:tasks.workunit.client.0.smithi077.stdout:9/789: write d9/d3a/fea [564833,90389] 0 2024-04-26T04:53:21.856 INFO:tasks.workunit.client.0.smithi077.stdout:5/862: chown d1/d42/d62/d6b/de8/ce9 355566687 1 2024-04-26T04:53:21.858 INFO:tasks.workunit.client.0.smithi077.stdout:5/863: write d1/d30/f53 [3778809,77553] 0 2024-04-26T04:53:21.859 INFO:tasks.workunit.client.0.smithi077.stdout:4/927: creat d4/d17/d18/d43/d51/db4/d15/d1f/d35/f13a x:0 0 0 2024-04-26T04:53:21.859 INFO:tasks.workunit.client.0.smithi077.stdout:5/864: truncate d1/d42/da4/f17 4807639 0 2024-04-26T04:53:21.860 INFO:tasks.workunit.client.0.smithi077.stdout:7/891: symlink da/d13/d22/d5d/dbd/df2/l12b 0 2024-04-26T04:53:21.864 INFO:tasks.workunit.client.0.smithi077.stdout:8/919: rename d4/df/d22/d80/d86/de3/f116 to d4/df/dce/d5b/f128 0 2024-04-26T04:53:21.865 INFO:tasks.workunit.client.0.smithi077.stdout:9/790: unlink d9/dd/d1b/d4e/c6c 0 2024-04-26T04:53:21.869 INFO:tasks.workunit.client.0.smithi077.stdout:8/920: unlink d4/df/dce/l49 0 2024-04-26T04:53:21.870 INFO:tasks.workunit.client.0.smithi077.stdout:9/791: mkdir d9/d35/d99/d101 0 2024-04-26T04:53:21.871 INFO:tasks.workunit.client.0.smithi077.stdout:4/928: mknod d4/c13b 0 2024-04-26T04:53:21.872 INFO:tasks.workunit.client.0.smithi077.stdout:5/865: mknod d1/c11e 0 2024-04-26T04:53:21.873 INFO:tasks.workunit.client.0.smithi077.stdout:5/866: chown d1/d42/da4 0 1 2024-04-26T04:53:21.874 INFO:tasks.workunit.client.0.smithi077.stdout:7/892: mknod da/d15/d1a/d39/df3/c12c 0 2024-04-26T04:53:21.874 INFO:tasks.workunit.client.0.smithi077.stdout:5/867: chown d1/d12/f1f 642573 1 2024-04-26T04:53:21.876 INFO:tasks.workunit.client.0.smithi077.stdout:3/856: getdents d0 0 2024-04-26T04:53:21.877 INFO:tasks.workunit.client.0.smithi077.stdout:8/921: rename d4/c20 to d4/d69/d73/c129 0 2024-04-26T04:53:21.878 INFO:tasks.workunit.client.0.smithi077.stdout:9/792: dwrite d9/d45/dcd/ffe [0,4194304] 0 2024-04-26T04:53:21.879 INFO:tasks.workunit.client.0.smithi077.stdout:3/857: write d0/d7/d9a/db3/f22 [3321829,10874] 0 2024-04-26T04:53:21.881 INFO:tasks.workunit.client.0.smithi077.stdout:4/929: fsync d4/d17/d18/d43/d53/fe3 0 2024-04-26T04:53:21.883 INFO:tasks.workunit.client.0.smithi077.stdout:6/911: write d8/d2a/ddd/fe3 [616060,12693] 0 2024-04-26T04:53:21.884 INFO:tasks.workunit.client.0.smithi077.stdout:8/922: rename d4/df/d22/d2d to d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/df2/d12a 22 2024-04-26T04:53:21.887 INFO:tasks.workunit.client.0.smithi077.stdout:6/912: dread - d8/d52/d53/d5e/da6/de6/fef zero size 2024-04-26T04:53:21.892 INFO:tasks.workunit.client.0.smithi077.stdout:4/930: dread d4/d17/d18/d43/d51/db4/d15/ffa [0,4194304] 0 2024-04-26T04:53:21.892 INFO:tasks.workunit.client.0.smithi077.stdout:9/793: chown d9/dd/d1b/d2b/d32/lc5 2289638 1 2024-04-26T04:53:21.893 INFO:tasks.workunit.client.0.smithi077.stdout:9/794: readlink d9/dd/def/lf1 0 2024-04-26T04:53:21.901 INFO:tasks.workunit.client.0.smithi077.stdout:8/923: creat d4/d17/d90/dfe/d114/f12b x:0 0 0 2024-04-26T04:53:21.902 INFO:tasks.workunit.client.0.smithi077.stdout:5/868: write d1/f5 [3965781,20413] 0 2024-04-26T04:53:21.907 INFO:tasks.workunit.client.0.smithi077.stdout:4/931: creat d4/d17/d18/d43/d51/db4/d15/d1f/d35/f13c x:0 0 0 2024-04-26T04:53:21.909 INFO:tasks.workunit.client.0.smithi077.stdout:9/795: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/f102 x:0 0 0 2024-04-26T04:53:21.910 INFO:tasks.workunit.client.0.smithi077.stdout:7/893: dwrite da/d15/dc6/fda [4194304,4194304] 0 2024-04-26T04:53:21.910 INFO:tasks.workunit.client.0.smithi077.stdout:8/924: dwrite d4/df/d22/d2d/f1e [0,4194304] 0 2024-04-26T04:53:21.912 INFO:tasks.workunit.client.0.smithi077.stdout:3/858: symlink d0/dae/ded/d29/db8/dbf/l114 0 2024-04-26T04:53:21.913 INFO:tasks.workunit.client.0.smithi077.stdout:9/796: write d9/d7d/d94/ff6 [4170299,5981] 0 2024-04-26T04:53:21.913 INFO:tasks.workunit.client.0.smithi077.stdout:7/894: chown da/d13/d22/d7d/d80/lde 252343967 1 2024-04-26T04:53:21.922 INFO:tasks.workunit.client.0.smithi077.stdout:7/895: fsync da/d13/d22/f2e 0 2024-04-26T04:53:21.922 INFO:tasks.workunit.client.0.smithi077.stdout:9/797: truncate d9/dd/d1b/d2b/f83 919335 0 2024-04-26T04:53:21.931 INFO:tasks.workunit.client.0.smithi077.stdout:9/798: creat d9/d3a/d79/d95/dad/de7/f103 x:0 0 0 2024-04-26T04:53:21.933 INFO:tasks.workunit.client.0.smithi077.stdout:5/869: rename d1/d8/l2f to d1/d30/l11f 0 2024-04-26T04:53:21.933 INFO:tasks.workunit.client.0.smithi077.stdout:6/913: dwrite d8/d52/d53/fc7 [0,4194304] 0 2024-04-26T04:53:21.936 INFO:tasks.workunit.client.0.smithi077.stdout:4/932: creat d4/f13d x:0 0 0 2024-04-26T04:53:21.936 INFO:tasks.workunit.client.0.smithi077.stdout:9/799: mkdir d9/dd/d1b/d4e/d104 0 2024-04-26T04:53:21.937 INFO:tasks.workunit.client.0.smithi077.stdout:8/925: dread d4/d87/fbe [0,4194304] 0 2024-04-26T04:53:21.938 INFO:tasks.workunit.client.0.smithi077.stdout:3/859: rename d0/d9/f8b to d0/d7/d9a/db7/f115 0 2024-04-26T04:53:21.940 INFO:tasks.workunit.client.0.smithi077.stdout:3/860: read - d0/d7/d1f/d81/fde zero size 2024-04-26T04:53:21.942 INFO:tasks.workunit.client.0.smithi077.stdout:5/870: fdatasync d1/d30/d33/fe4 0 2024-04-26T04:53:21.945 INFO:tasks.workunit.client.0.smithi077.stdout:7/896: rename da/d13/d22/d5d/d9f/fc0 to da/d13/d22/d5d/d9f/da7/f12d 0 2024-04-26T04:53:21.947 INFO:tasks.workunit.client.0.smithi077.stdout:9/800: symlink d9/dd/d1b/l105 0 2024-04-26T04:53:21.948 INFO:tasks.workunit.client.0.smithi077.stdout:9/801: write d9/f62 [4354093,61310] 0 2024-04-26T04:53:21.949 INFO:tasks.workunit.client.0.smithi077.stdout:8/926: creat d4/df/d22/d80/d86/de3/f12c x:0 0 0 2024-04-26T04:53:21.949 INFO:tasks.workunit.client.0.smithi077.stdout:3/861: dwrite d0/d7/d1f/d81/fda [0,4194304] 0 2024-04-26T04:53:21.951 INFO:tasks.workunit.client.0.smithi077.stdout:3/862: chown d0/dae/ded/d2a 0 1 2024-04-26T04:53:21.953 INFO:tasks.workunit.client.0.smithi077.stdout:3/863: chown d0/dae/ded/d29/db8/f100 819071 1 2024-04-26T04:53:21.957 INFO:tasks.workunit.client.0.smithi077.stdout:5/871: truncate d1/d8/fc 2598529 0 2024-04-26T04:53:21.963 INFO:tasks.workunit.client.0.smithi077.stdout:4/933: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/d76/f87 [0,4194304] 0 2024-04-26T04:53:21.963 INFO:tasks.workunit.client.0.smithi077.stdout:9/802: rmdir d9/dd/d1b/d2b/d32/d36/d7f 39 2024-04-26T04:53:21.967 INFO:tasks.workunit.client.0.smithi077.stdout:3/864: creat d0/d7/d1f/d81/dc5/dd1/f116 x:0 0 0 2024-04-26T04:53:21.981 INFO:tasks.workunit.client.0.smithi077.stdout:7/897: creat da/d13/d22/d5d/dbd/f12e x:0 0 0 2024-04-26T04:53:21.983 INFO:tasks.workunit.client.0.smithi077.stdout:4/934: getdents d4/d17/d18/d43/d51/db4/d15/d1f/d35/d115 0 2024-04-26T04:53:21.984 INFO:tasks.workunit.client.0.smithi077.stdout:3/865: write d0/dae/ded/d2a/d85/fa4 [515923,87938] 0 2024-04-26T04:53:21.989 INFO:tasks.workunit.client.0.smithi077.stdout:9/803: symlink d9/dd/d1b/d2b/d32/d36/d7f/dae/dfa/l106 0 2024-04-26T04:53:21.991 INFO:tasks.workunit.client.0.smithi077.stdout:3/866: dwrite d0/d7/d1f/d81/dc5/dd1/f102 [0,4194304] 0 2024-04-26T04:53:21.993 INFO:tasks.workunit.client.0.smithi077.stdout:8/927: getdents d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3 0 2024-04-26T04:53:21.996 INFO:tasks.workunit.client.0.smithi077.stdout:5/872: getdents d1/d8/d7e 0 2024-04-26T04:53:21.997 INFO:tasks.workunit.client.0.smithi077.stdout:5/873: chown d1/d42/d62/dc8/dd5/df8/dfd/c106 5433282 1 2024-04-26T04:53:21.998 INFO:tasks.workunit.client.0.smithi077.stdout:7/898: rmdir da/d13/d22/d5d/dbd 39 2024-04-26T04:53:21.998 INFO:tasks.workunit.client.0.smithi077.stdout:5/874: readlink d1/d12/d28/d38/l91 0 2024-04-26T04:53:22.003 INFO:tasks.workunit.client.0.smithi077.stdout:3/867: creat d0/d7/d9a/db7/ddb/f117 x:0 0 0 2024-04-26T04:53:22.005 INFO:tasks.workunit.client.0.smithi077.stdout:3/868: write d0/d7/d1f/fcf [753084,46379] 0 2024-04-26T04:53:22.006 INFO:tasks.workunit.client.0.smithi077.stdout:8/928: mknod d4/df/d22/d80/d86/de3/c12d 0 2024-04-26T04:53:22.009 INFO:tasks.workunit.client.0.smithi077.stdout:5/875: rename d1/d42/d62/dc8/dd5/l107 to d1/d8f/l120 0 2024-04-26T04:53:22.010 INFO:tasks.workunit.client.0.smithi077.stdout:9/804: mkdir d9/dd/df2/d107 0 2024-04-26T04:53:22.012 INFO:tasks.workunit.client.0.smithi077.stdout:3/869: symlink d0/dae/ded/d2a/d85/dd3/l118 0 2024-04-26T04:53:22.015 INFO:tasks.workunit.client.0.smithi077.stdout:9/805: creat d9/d3a/d79/d95/dad/de7/f108 x:0 0 0 2024-04-26T04:53:22.018 INFO:tasks.workunit.client.0.smithi077.stdout:3/870: rename d0/dae/ded/d29/dc8/ld4 to d0/dae/l119 0 2024-04-26T04:53:22.020 INFO:tasks.workunit.client.0.smithi077.stdout:7/899: link da/d15/faa da/d13/d22/d5d/f12f 0 2024-04-26T04:53:22.021 INFO:tasks.workunit.client.0.smithi077.stdout:8/929: dread d4/fd4 [0,4194304] 0 2024-04-26T04:53:22.022 INFO:tasks.workunit.client.0.smithi077.stdout:8/930: chown d4/df/d22/d2d/f94 616 1 2024-04-26T04:53:22.023 INFO:tasks.workunit.client.0.smithi077.stdout:3/871: symlink d0/dae/ded/d2a/d85/dd3/l11a 0 2024-04-26T04:53:22.025 INFO:tasks.workunit.client.0.smithi077.stdout:9/806: rename d9/d45/dc4/cdd to d9/d7d/dd1/c109 0 2024-04-26T04:53:22.028 INFO:tasks.workunit.client.0.smithi077.stdout:3/872: symlink d0/d7/d9a/db7/ddb/l11b 0 2024-04-26T04:53:22.031 INFO:tasks.workunit.client.0.smithi077.stdout:8/931: rename d4/df/d22/d2d/d2b/d33/dd8/d117/d120/db8/l102 to d4/df/d122/l12e 0 2024-04-26T04:53:22.032 INFO:tasks.workunit.client.0.smithi077.stdout:9/807: chown d9/d3a/d79/d95/fb8 32 1 2024-04-26T04:53:22.037 INFO:tasks.workunit.client.0.smithi077.stdout:3/873: unlink d0/dae/ded/l73 0 2024-04-26T04:53:22.040 INFO:tasks.workunit.client.0.smithi077.stdout:3/874: truncate d0/dae/ded/d2a/f40 824965 0 2024-04-26T04:53:22.042 INFO:tasks.workunit.client.0.smithi077.stdout:3/875: write d0/dae/fe2 [345496,11383] 0 2024-04-26T04:53:22.079 INFO:tasks.workunit.client.0.smithi077.stdout:8/932: dwrite d4/df/d22/f9e [0,4194304] 0 2024-04-26T04:53:22.103 INFO:tasks.workunit.client.0.smithi077.stdout:7/900: dwrite da/d13/d22/d5d/dbd/df2/f108 [0,4194304] 0 2024-04-26T04:53:22.154 INFO:tasks.workunit.client.0.smithi077.stdout:8/933: creat d4/d17/d90/dfe/f12f x:0 0 0 2024-04-26T04:53:22.156 INFO:tasks.workunit.client.0.smithi077.stdout:8/934: truncate d4/d17/d90/dfe/d114/f12b 573760 0 2024-04-26T04:53:22.157 INFO:tasks.workunit.client.0.smithi077.stdout:7/901: rmdir da/d13/d22 39 2024-04-26T04:53:22.167 INFO:tasks.workunit.client.0.smithi077.stdout:3/876: getdents d0/d7/d1f/d4d 0 2024-04-26T04:53:22.167 INFO:tasks.workunit.client.0.smithi077.stdout:8/935: dread d4/fd [0,4194304] 0 2024-04-26T04:53:22.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:21 smithi077 ceph-mon[26311]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 58 MiB/s rd, 144 MiB/s wr, 330 op/s 2024-04-26T04:53:22.200 INFO:tasks.workunit.client.0.smithi077.stdout:8/936: creat d4/df/d22/d2d/d2b/d33/dbc/f130 x:0 0 0 2024-04-26T04:53:22.205 INFO:tasks.workunit.client.0.smithi077.stdout:3/877: dwrite d0/dae/ded/d2a/faa [4194304,4194304] 0 2024-04-26T04:53:22.209 INFO:tasks.workunit.client.0.smithi077.stdout:7/902: dwrite da/d13/d22/d5d/f66 [0,4194304] 0 2024-04-26T04:53:22.216 INFO:tasks.workunit.client.0.smithi077.stdout:7/903: symlink da/d13/d22/d5d/d8d/l130 0 2024-04-26T04:53:22.220 INFO:tasks.workunit.client.0.smithi077.stdout:8/937: dread d4/d69/d73/fef [0,4194304] 0 2024-04-26T04:53:22.223 INFO:tasks.workunit.client.0.smithi077.stdout:3/878: dwrite d0/d7/d1f/d4d/dc6/f101 [0,4194304] 0 2024-04-26T04:53:22.227 INFO:tasks.workunit.client.0.smithi077.stdout:8/938: symlink d4/df/d22/d2d/d2b/d33/l131 0 2024-04-26T04:53:22.228 INFO:tasks.workunit.client.0.smithi077.stdout:8/939: chown d4/df/d22/c108 630617 1 2024-04-26T04:53:22.228 INFO:tasks.workunit.client.0.smithi077.stdout:3/879: mkdir d0/d7/d1f/d81/dc5/dc7/d11c 0 2024-04-26T04:53:22.232 INFO:tasks.workunit.client.0.smithi077.stdout:3/880: truncate d0/d7/fd5 3229796 0 2024-04-26T04:53:22.239 INFO:tasks.workunit.client.0.smithi077.stdout:3/881: getdents d0/d7/d1f/d81/dc5 0 2024-04-26T04:53:22.245 INFO:tasks.workunit.client.0.smithi077.stdout:3/882: dread d0/dae/ded/d2a/d85/fa4 [0,4194304] 0 2024-04-26T04:53:22.245 INFO:tasks.workunit.client.0.smithi077.stdout:8/940: dwrite d4/d17/d90/fab [0,4194304] 0 2024-04-26T04:53:22.253 INFO:tasks.workunit.client.0.smithi077.stdout:8/941: truncate d4/d17/d90/fd1 257068 0 2024-04-26T04:53:22.254 INFO:tasks.workunit.client.0.smithi077.stdout:3/883: unlink d0/dae/ded/c62 0 2024-04-26T04:53:22.255 INFO:tasks.workunit.client.0.smithi077.stdout:3/884: fdatasync d0/f38 0 2024-04-26T04:53:22.260 INFO:tasks.workunit.client.0.smithi077.stdout:8/942: rename d4/df/d18/d56/cb9 to d4/df/dce/c132 0 2024-04-26T04:53:22.261 INFO:tasks.workunit.client.0.smithi077.stdout:8/943: readlink d4/df/d22/d2d/d2b/d33/dd8/d117/d120/l7a 0 2024-04-26T04:53:22.264 INFO:tasks.workunit.client.0.smithi077.stdout:8/944: truncate d4/f5 1290055 0 2024-04-26T04:53:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:21 smithi078 ceph-mon[33746]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 58 MiB/s rd, 144 MiB/s wr, 330 op/s 2024-04-26T04:53:22.269 INFO:tasks.workunit.client.0.smithi077.stdout:8/945: rename d4/df/d22/d2d/d2b/d33/dd8/d117/d120/db8 to d4/df/d18/d56/d133 0 2024-04-26T04:53:22.275 INFO:tasks.workunit.client.0.smithi077.stdout:8/946: mknod d4/df/dce/c134 0 2024-04-26T04:53:22.295 INFO:tasks.workunit.client.0.smithi077.stdout:8/947: dwrite d4/df/dce/f52 [0,4194304] 0 2024-04-26T04:53:22.299 INFO:tasks.workunit.client.0.smithi077.stdout:8/948: rename d4/d17/d90 to d4/df/dce/d5b/d135 0 2024-04-26T04:53:22.301 INFO:tasks.workunit.client.0.smithi077.stdout:8/949: symlink d4/df/d122/l136 0 2024-04-26T04:53:22.307 INFO:tasks.workunit.client.0.smithi077.stdout:8/950: link d4/df/d18/d56/d133/c121 d4/df/dce/d5b/d135/dfe/c137 0 2024-04-26T04:53:22.313 INFO:tasks.workunit.client.0.smithi077.stdout:8/951: dwrite d4/fd4 [0,4194304] 0 2024-04-26T04:53:22.318 INFO:tasks.workunit.client.0.smithi077.stdout:8/952: rename d4/df/d22/d80/d86/l96 to d4/df/d18/d56/l138 0 2024-04-26T04:53:22.320 INFO:tasks.workunit.client.0.smithi077.stdout:8/953: dread - d4/df/dce/d5b/d8d/fee zero size 2024-04-26T04:53:22.325 INFO:tasks.workunit.client.0.smithi077.stdout:8/954: unlink d4/df/d22/d2d/l81 0 2024-04-26T04:53:22.330 INFO:tasks.workunit.client.0.smithi077.stdout:8/955: rename d4/lc to d4/df/dce/d5b/d135/l139 0 2024-04-26T04:53:22.338 INFO:tasks.workunit.client.0.smithi077.stdout:1/938: sync 2024-04-26T04:53:22.340 INFO:tasks.workunit.client.0.smithi077.stdout:1/939: unlink d9/db/d2b/d3f/f50 0 2024-04-26T04:53:22.347 INFO:tasks.workunit.client.0.smithi077.stdout:1/940: getdents d9/d53/d6e/dd3/d113 0 2024-04-26T04:53:22.352 INFO:tasks.workunit.client.0.smithi077.stdout:1/941: chown d9/db/dd/d68/f89 26 1 2024-04-26T04:53:22.355 INFO:tasks.workunit.client.0.smithi077.stdout:8/956: dread d4/df/d22/d80/d86/d103/f10a [0,4194304] 0 2024-04-26T04:53:22.357 INFO:tasks.workunit.client.0.smithi077.stdout:1/942: mkdir d9/d53/d8b/d10c/d12c 0 2024-04-26T04:53:22.363 INFO:tasks.workunit.client.0.smithi077.stdout:8/957: write d4/df/dce/d5b/d135/fdd [235956,74459] 0 2024-04-26T04:53:22.367 INFO:tasks.workunit.client.0.smithi077.stdout:1/943: creat d9/df1/f12d x:0 0 0 2024-04-26T04:53:22.367 INFO:tasks.workunit.client.0.smithi077.stdout:8/958: dread - d4/df/d18/d56/f8f zero size 2024-04-26T04:53:22.369 INFO:tasks.workunit.client.0.smithi077.stdout:1/944: creat d9/db/dff/f12e x:0 0 0 2024-04-26T04:53:22.373 INFO:tasks.workunit.client.0.smithi077.stdout:1/945: rename d9/d53/db9 to d9/d53/d8b/d12f 0 2024-04-26T04:53:22.373 INFO:tasks.workunit.client.0.smithi077.stdout:8/959: mkdir d4/df/d22/d13a 0 2024-04-26T04:53:22.374 INFO:tasks.workunit.client.0.smithi077.stdout:1/946: truncate d9/db/d2b/d75/f97 220400 0 2024-04-26T04:53:22.375 INFO:tasks.workunit.client.0.smithi077.stdout:8/960: write d4/df/dce/d5b/d135/dfe/f12f [161972,57598] 0 2024-04-26T04:53:22.379 INFO:tasks.workunit.client.0.smithi077.stdout:8/961: creat d4/df/d122/f13b x:0 0 0 2024-04-26T04:53:22.381 INFO:tasks.workunit.client.0.smithi077.stdout:1/947: dwrite d9/db/d2b/d3f/fc3 [0,4194304] 0 2024-04-26T04:53:22.382 INFO:tasks.workunit.client.0.smithi077.stdout:8/962: rmdir d4/df/dce/d5b 39 2024-04-26T04:53:22.383 INFO:tasks.workunit.client.0.smithi077.stdout:8/963: write d4/f126 [868420,35070] 0 2024-04-26T04:53:22.386 INFO:tasks.workunit.client.0.smithi077.stdout:1/948: fdatasync d9/d53/d6e/d7c/fcf 0 2024-04-26T04:53:22.387 INFO:tasks.workunit.client.0.smithi077.stdout:1/949: readlink d9/db/dd/lfb 0 2024-04-26T04:53:22.390 INFO:tasks.workunit.client.0.smithi077.stdout:8/964: creat d4/d69/d73/f13c x:0 0 0 2024-04-26T04:53:22.396 INFO:tasks.workunit.client.0.smithi077.stdout:8/965: fsync d4/df/d22/d2d/d2b/d33/dbc/f100 0 2024-04-26T04:53:22.408 INFO:tasks.workunit.client.0.smithi077.stdout:8/966: dread d4/df/fd6 [0,4194304] 0 2024-04-26T04:53:22.410 INFO:tasks.workunit.client.0.smithi077.stdout:1/950: getdents d9/db/d2b 0 2024-04-26T04:53:22.424 INFO:tasks.workunit.client.0.smithi077.stdout:1/951: mkdir d9/d53/d6e/d7c/d130 0 2024-04-26T04:53:22.430 INFO:tasks.workunit.client.0.smithi077.stdout:1/952: creat d9/d53/d8b/d93/da8/da7/f131 x:0 0 0 2024-04-26T04:53:22.437 INFO:tasks.workunit.client.0.smithi077.stdout:1/953: dwrite d9/d53/d6e/f40 [0,4194304] 0 2024-04-26T04:53:22.441 INFO:tasks.workunit.client.0.smithi077.stdout:1/954: read d9/d90/fbf [1808958,74197] 0 2024-04-26T04:53:22.442 INFO:tasks.workunit.client.0.smithi077.stdout:8/967: dwrite d4/df/dce/d5b/d8d/fd7 [0,4194304] 0 2024-04-26T04:53:22.447 INFO:tasks.workunit.client.0.smithi077.stdout:1/955: dread - d9/db/dd/d68/d81/f95 zero size 2024-04-26T04:53:22.476 INFO:tasks.workunit.client.0.smithi077.stdout:8/968: dread d4/df/d22/d2d/fc1 [0,4194304] 0 2024-04-26T04:53:22.480 INFO:tasks.workunit.client.0.smithi077.stdout:8/969: creat d4/df/dce/d5b/d135/dfe/f13d x:0 0 0 2024-04-26T04:53:22.484 INFO:tasks.workunit.client.0.smithi077.stdout:8/970: mkdir d4/df/d22/d13e 0 2024-04-26T04:53:22.486 INFO:tasks.workunit.client.0.smithi077.stdout:1/956: write d9/d53/d6e/d7c/f82 [110090,112466] 0 2024-04-26T04:53:22.488 INFO:tasks.workunit.client.0.smithi077.stdout:1/957: write d9/d90/fad [241402,33098] 0 2024-04-26T04:53:22.488 INFO:tasks.workunit.client.0.smithi077.stdout:8/971: symlink d4/df/dce/d5b/d135/l13f 0 2024-04-26T04:53:22.492 INFO:tasks.workunit.client.0.smithi077.stdout:1/958: chown d9/db/dd/d34/l35 61 1 2024-04-26T04:53:22.493 INFO:tasks.workunit.client.0.smithi077.stdout:1/959: chown d9/db/dd/d9c 348837 1 2024-04-26T04:53:22.502 INFO:tasks.workunit.client.0.smithi077.stdout:8/972: getdents d4/df/d22/d2d/d104 0 2024-04-26T04:53:22.516 INFO:tasks.workunit.client.0.smithi077.stdout:1/960: dread d9/f43 [0,4194304] 0 2024-04-26T04:53:22.518 INFO:tasks.workunit.client.0.smithi077.stdout:1/961: chown d9/d53/d6e/d7c/f82 38612 1 2024-04-26T04:53:22.520 INFO:tasks.workunit.client.0.smithi077.stdout:8/973: symlink d4/l140 0 2024-04-26T04:53:22.535 INFO:tasks.workunit.client.0.smithi077.stdout:1/962: link d9/d53/d8b/d93/c64 d9/df1/c132 0 2024-04-26T04:53:22.541 INFO:tasks.workunit.client.0.smithi077.stdout:1/963: dwrite d9/db/f39 [0,4194304] 0 2024-04-26T04:53:22.542 INFO:tasks.workunit.client.0.smithi077.stdout:1/964: stat d9/d90/de0/c11c 0 2024-04-26T04:53:22.550 INFO:tasks.workunit.client.0.smithi077.stdout:1/965: mknod d9/db/dd/d34/d4a/dbb/d115/c133 0 2024-04-26T04:53:22.554 INFO:tasks.workunit.client.0.smithi077.stdout:1/966: creat d9/d90/de0/d107/f134 x:0 0 0 2024-04-26T04:53:22.556 INFO:tasks.workunit.client.0.smithi077.stdout:8/974: write d4/df/d22/d2d/d2b/d33/dd8/ffb [186555,8261] 0 2024-04-26T04:53:22.560 INFO:tasks.workunit.client.0.smithi077.stdout:8/975: truncate d4/d17/f2c 1154343 0 2024-04-26T04:53:22.561 INFO:tasks.workunit.client.0.smithi077.stdout:8/976: write d4/df/dc6/fd3 [2967031,32562] 0 2024-04-26T04:53:22.562 INFO:tasks.workunit.client.0.smithi077.stdout:8/977: fsync d4/df/dce/d5b/f11a 0 2024-04-26T04:53:22.565 INFO:tasks.workunit.client.0.smithi077.stdout:1/967: write d9/f44 [980608,112667] 0 2024-04-26T04:53:22.566 INFO:tasks.workunit.client.0.smithi077.stdout:1/968: dread - d9/d53/d8b/d93/da8/f102 zero size 2024-04-26T04:53:22.570 INFO:tasks.workunit.client.0.smithi077.stdout:8/978: unlink d4/d69/d73/fe5 0 2024-04-26T04:53:22.574 INFO:tasks.workunit.client.0.smithi077.stdout:1/969: dwrite d9/d53/d8b/d93/da8/f102 [0,4194304] 0 2024-04-26T04:53:22.579 INFO:tasks.workunit.client.0.smithi077.stdout:8/979: rename d4/fd4 to d4/df/d22/d2d/d2b/d33/dd8/f141 0 2024-04-26T04:53:22.583 INFO:tasks.workunit.client.0.smithi077.stdout:1/970: dwrite d9/d53/d6e/f40 [0,4194304] 0 2024-04-26T04:53:22.585 INFO:tasks.workunit.client.0.smithi077.stdout:1/971: chown d9/db/dd/d68/d81/f85 208 1 2024-04-26T04:53:22.586 INFO:tasks.workunit.client.0.smithi077.stdout:8/980: creat d4/df/dce/d5b/d135/dfe/d114/f142 x:0 0 0 2024-04-26T04:53:22.606 INFO:tasks.workunit.client.0.smithi077.stdout:8/981: dread d4/df/d22/f68 [0,4194304] 0 2024-04-26T04:53:22.608 INFO:tasks.workunit.client.0.smithi077.stdout:8/982: rmdir d4/df/dc6 39 2024-04-26T04:53:22.612 INFO:tasks.workunit.client.0.smithi077.stdout:8/983: unlink d4/df/d22/d80/fe4 0 2024-04-26T04:53:22.613 INFO:tasks.workunit.client.0.smithi077.stdout:8/984: write d4/d69/d73/f13c [208215,20514] 0 2024-04-26T04:53:22.614 INFO:tasks.workunit.client.0.smithi077.stdout:1/972: dread d9/d53/d6e/f87 [4194304,4194304] 0 2024-04-26T04:53:22.614 INFO:tasks.workunit.client.0.smithi077.stdout:1/973: stat d9/db/fd1 0 2024-04-26T04:53:22.617 INFO:tasks.workunit.client.0.smithi077.stdout:1/974: mknod d9/d53/d8b/d93/da8/da7/c135 0 2024-04-26T04:53:22.621 INFO:tasks.workunit.client.0.smithi077.stdout:8/985: rename d4/df/d22/d75 to d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/dde/d143 0 2024-04-26T04:53:22.622 INFO:tasks.workunit.client.0.smithi077.stdout:1/975: rmdir d9/db/dd/d9c/dab/d117 39 2024-04-26T04:53:22.625 INFO:tasks.workunit.client.0.smithi077.stdout:8/986: dwrite d4/df/d22/d2d/d2b/f4d [0,4194304] 0 2024-04-26T04:53:22.634 INFO:tasks.workunit.client.0.smithi077.stdout:1/976: mkdir d9/d90/d136 0 2024-04-26T04:53:22.649 INFO:tasks.workunit.client.0.smithi077.stdout:0/835: sync 2024-04-26T04:53:22.649 INFO:tasks.workunit.client.0.smithi077.stdout:8/987: dwrite d4/df/f8c [0,4194304] 0 2024-04-26T04:53:22.659 INFO:tasks.workunit.client.0.smithi077.stdout:8/988: creat d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/df2/f144 x:0 0 0 2024-04-26T04:53:22.663 INFO:tasks.workunit.client.0.smithi077.stdout:8/989: dread d4/df/d18/d56/f72 [0,4194304] 0 2024-04-26T04:53:22.667 INFO:tasks.workunit.client.0.smithi077.stdout:8/990: mknod d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/dde/c145 0 2024-04-26T04:53:22.672 INFO:tasks.workunit.client.0.smithi077.stdout:8/991: rename d4/df/d22/d2d/d2b/d33/dd8/d117/d120/da3/dc7/dde/f11f to d4/df/dc6/f146 0 2024-04-26T04:53:22.674 INFO:tasks.workunit.client.0.smithi077.stdout:8/992: rmdir d4/df/d22/d2d/d2b/d33/dbc 39 2024-04-26T04:53:22.675 INFO:tasks.workunit.client.0.smithi077.stdout:8/993: truncate d4/d17/fc5 716708 0 2024-04-26T04:53:22.677 INFO:tasks.workunit.client.0.smithi077.stdout:8/994: fdatasync d4/d87/fb2 0 2024-04-26T04:53:22.679 INFO:tasks.workunit.client.0.smithi077.stdout:8/995: read d4/df/dce/d5b/f7f [3777799,17196] 0 2024-04-26T04:53:22.680 INFO:tasks.workunit.client.0.smithi077.stdout:8/996: stat d4/df/d22/d2d/d2b/d33/l131 0 2024-04-26T04:53:22.687 INFO:tasks.workunit.client.0.smithi077.stdout:8/997: getdents d4/df/d22/d2d/d2b/d33 0 2024-04-26T04:53:22.690 INFO:tasks.workunit.client.0.smithi077.stdout:8/998: rename d4/df/d22/d2d/d2b/ca9 to d4/d69/d73/c147 0 2024-04-26T04:53:22.693 INFO:tasks.workunit.client.0.smithi077.stdout:8/999: fsync d4/df/dce/d5b/fa0 0 2024-04-26T04:53:22.791 INFO:tasks.workunit.client.0.smithi077.stdout:4/935: sync 2024-04-26T04:53:22.791 INFO:tasks.workunit.client.0.smithi077.stdout:6/914: sync 2024-04-26T04:53:22.792 INFO:tasks.workunit.client.0.smithi077.stdout:4/936: chown d4/d17/d18/d43/f74 303 1 2024-04-26T04:53:22.793 INFO:tasks.workunit.client.0.smithi077.stdout:4/937: chown d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/la8 31665201 1 2024-04-26T04:53:22.796 INFO:tasks.workunit.client.0.smithi077.stdout:4/938: symlink d4/d4e/dad/d8b/l13e 0 2024-04-26T04:53:22.797 INFO:tasks.workunit.client.0.smithi077.stdout:6/915: write d8/d52/d53/d5e/da6/faa [1019828,38984] 0 2024-04-26T04:53:22.803 INFO:tasks.workunit.client.0.smithi077.stdout:4/939: creat d4/d17/d18/d43/d63/f13f x:0 0 0 2024-04-26T04:53:22.806 INFO:tasks.workunit.client.0.smithi077.stdout:4/940: mkdir d4/d17/d18/d43/d6e/d6c/dae/dc5/d140 0 2024-04-26T04:53:22.808 INFO:tasks.workunit.client.0.smithi077.stdout:4/941: symlink d4/d17/d18/d43/d53/d64/l141 0 2024-04-26T04:53:22.810 INFO:tasks.workunit.client.0.smithi077.stdout:4/942: write d4/d17/d18/d43/d6e/f95 [2682743,61714] 0 2024-04-26T04:53:22.811 INFO:tasks.workunit.client.0.smithi077.stdout:6/916: dwrite d8/d52/d53/da7/fea [0,4194304] 0 2024-04-26T04:53:22.815 INFO:tasks.workunit.client.0.smithi077.stdout:4/943: dread d4/d17/d18/d43/d51/db4/d15/d1f/d76/f87 [0,4194304] 0 2024-04-26T04:53:22.820 INFO:tasks.workunit.client.0.smithi077.stdout:4/944: fdatasync d4/d4e/ff4 0 2024-04-26T04:53:22.820 INFO:tasks.workunit.client.0.smithi077.stdout:4/945: stat d4/d17/d18/d43/d51/db4/d15/d1f/dd8/ld9 0 2024-04-26T04:53:22.825 INFO:tasks.workunit.client.0.smithi077.stdout:4/946: unlink d4/d17/d18/d43/d51/db4/d15/f50 0 2024-04-26T04:53:22.825 INFO:tasks.workunit.client.0.smithi077.stdout:6/917: write d8/d8f/f10a [3985319,33247] 0 2024-04-26T04:53:22.830 INFO:tasks.workunit.client.0.smithi077.stdout:4/947: rename d4/d17/d18/d43/d51/db4/d15/d1f/d35/l9b to d4/d17/d18/d43/d6e/d6c/dda/l142 0 2024-04-26T04:53:22.831 INFO:tasks.workunit.client.0.smithi077.stdout:6/918: rmdir d8/d52/d53/d5e/d6c/d135 0 2024-04-26T04:53:22.832 INFO:tasks.workunit.client.0.smithi077.stdout:4/948: write d4/d17/d18/d43/d51/db4/d15/d1f/d35/f13a [54564,67325] 0 2024-04-26T04:53:22.836 INFO:tasks.workunit.client.0.smithi077.stdout:4/949: mknod d4/d17/d18/d43/d51/db4/d15/d1f/d25/c143 0 2024-04-26T04:53:22.839 INFO:tasks.workunit.client.0.smithi077.stdout:4/950: rename d4/d4e/dad/cef to d4/d17/d18/d43/d6e/d6c/dae/dc5/c144 0 2024-04-26T04:53:22.841 INFO:tasks.workunit.client.0.smithi077.stdout:4/951: mknod d4/d4e/dad/d8b/c145 0 2024-04-26T04:53:22.844 INFO:tasks.workunit.client.0.smithi077.stdout:4/952: unlink d4/d4e/dad/d8b/dba/cc8 0 2024-04-26T04:53:22.846 INFO:tasks.workunit.client.0.smithi077.stdout:4/953: fdatasync d4/d4e/dad/d8b/ddc/fe9 0 2024-04-26T04:53:22.848 INFO:tasks.workunit.client.0.smithi077.stdout:4/954: mknod d4/d4e/dad/d8b/ddc/dec/c146 0 2024-04-26T04:53:22.852 INFO:tasks.workunit.client.0.smithi077.stdout:6/919: dwrite d8/d52/d53/da7/dfa/f12c [0,4194304] 0 2024-04-26T04:53:22.853 INFO:tasks.workunit.client.0.smithi077.stdout:6/920: write d8/d2a/fec [2147835,86221] 0 2024-04-26T04:53:22.862 INFO:tasks.workunit.client.0.smithi077.stdout:6/921: write d8/d52/d53/f107 [3736192,126189] 0 2024-04-26T04:53:22.864 INFO:tasks.workunit.client.0.smithi077.stdout:6/922: chown d8/d1c/f140 31904 1 2024-04-26T04:53:22.866 INFO:tasks.workunit.client.0.smithi077.stdout:4/955: dwrite d4/d17/d18/d43/fb5 [0,4194304] 0 2024-04-26T04:53:22.871 INFO:tasks.workunit.client.0.smithi077.stdout:6/923: symlink d8/d52/d11c/l144 0 2024-04-26T04:53:22.877 INFO:tasks.workunit.client.0.smithi077.stdout:4/956: getdents d4/d17/d18/d43/d51/db4/d15 0 2024-04-26T04:53:22.877 INFO:tasks.workunit.client.0.smithi077.stdout:6/924: write d8/d2a/f2e [6447039,95617] 0 2024-04-26T04:53:22.884 INFO:tasks.workunit.client.0.smithi077.stdout:6/925: getdents d8/d52/d53 0 2024-04-26T04:53:22.885 INFO:tasks.workunit.client.0.smithi077.stdout:4/957: rename d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/l100 to d4/d17/d18/d43/d51/db4/d15/d1f/l147 0 2024-04-26T04:53:22.890 INFO:tasks.workunit.client.0.smithi077.stdout:6/926: truncate d8/d55/f9b 3972792 0 2024-04-26T04:53:22.891 INFO:tasks.workunit.client.0.smithi077.stdout:6/927: mknod d8/d1c/d108/c145 0 2024-04-26T04:53:22.893 INFO:tasks.workunit.client.0.smithi077.stdout:6/928: fsync d8/d1c/db5/ffb 0 2024-04-26T04:53:22.898 INFO:tasks.workunit.client.0.smithi077.stdout:4/958: dwrite d4/d4e/dad/ffd [0,4194304] 0 2024-04-26T04:53:22.898 INFO:tasks.workunit.client.0.smithi077.stdout:4/959: stat d4/f117 0 2024-04-26T04:53:22.900 INFO:tasks.workunit.client.0.smithi077.stdout:6/929: rename d8/d52/la2 to d8/d52/d53/d5e/l146 0 2024-04-26T04:53:22.902 INFO:tasks.workunit.client.0.smithi077.stdout:6/930: read d8/f48 [2271641,18008] 0 2024-04-26T04:53:22.902 INFO:tasks.workunit.client.0.smithi077.stdout:4/960: rmdir d4/d4e/dad/d8b/ddc/dec 39 2024-04-26T04:53:22.906 INFO:tasks.workunit.client.0.smithi077.stdout:4/961: truncate d4/d17/d18/d43/d53/d56/f2d 7738389 0 2024-04-26T04:53:22.909 INFO:tasks.workunit.client.0.smithi077.stdout:6/931: symlink d8/d55/dc8/l147 0 2024-04-26T04:53:22.910 INFO:tasks.workunit.client.0.smithi077.stdout:6/932: write d8/d52/da0/f137 [3120743,82915] 0 2024-04-26T04:53:22.911 INFO:tasks.workunit.client.0.smithi077.stdout:4/962: read d4/d4e/dad/d8b/dba/fbb [107139,14896] 0 2024-04-26T04:53:22.911 INFO:tasks.workunit.client.0.smithi077.stdout:4/963: readlink d4/l6 0 2024-04-26T04:53:22.917 INFO:tasks.workunit.client.0.smithi077.stdout:4/964: fsync d4/d17/d7f/feb 0 2024-04-26T04:53:22.920 INFO:tasks.workunit.client.0.smithi077.stdout:4/965: rmdir d4/d4e/dad/d8b/ddc 39 2024-04-26T04:53:22.927 INFO:tasks.workunit.client.0.smithi077.stdout:6/933: dwrite d8/d52/f4d [4194304,4194304] 0 2024-04-26T04:53:22.931 INFO:tasks.workunit.client.0.smithi077.stdout:6/934: creat d8/d52/d53/d5e/da6/f148 x:0 0 0 2024-04-26T04:53:22.932 INFO:tasks.workunit.client.0.smithi077.stdout:4/966: dwrite d4/f8 [0,4194304] 0 2024-04-26T04:53:22.935 INFO:tasks.workunit.client.0.smithi077.stdout:6/935: mknod d8/d55/c149 0 2024-04-26T04:53:22.936 INFO:tasks.workunit.client.0.smithi077.stdout:4/967: creat d4/d4e/dad/f148 x:0 0 0 2024-04-26T04:53:22.949 INFO:tasks.workunit.client.0.smithi077.stdout:4/968: truncate d4/d17/d18/d43/fb5 3385857 0 2024-04-26T04:53:22.950 INFO:tasks.workunit.client.0.smithi077.stdout:4/969: fdatasync d4/d4e/dad/df8/f10b 0 2024-04-26T04:53:22.951 INFO:tasks.workunit.client.0.smithi077.stdout:4/970: truncate d4/d17/d18/d43/d63/f13f 2060 0 2024-04-26T04:53:22.951 INFO:tasks.workunit.client.0.smithi077.stdout:4/971: read - d4/f117 zero size 2024-04-26T04:53:22.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:22 smithi077 ceph-mon[26311]: pgmap v31: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 73 MiB/s rd, 176 MiB/s wr, 452 op/s 2024-04-26T04:53:23.001 INFO:tasks.workunit.client.0.smithi077.stdout:7/904: sync 2024-04-26T04:53:23.002 INFO:tasks.workunit.client.0.smithi077.stdout:9/808: sync 2024-04-26T04:53:23.002 INFO:tasks.workunit.client.0.smithi077.stdout:5/876: sync 2024-04-26T04:53:23.002 INFO:tasks.workunit.client.0.smithi077.stdout:3/885: sync 2024-04-26T04:53:23.002 INFO:tasks.workunit.client.0.smithi077.stdout:3/886: readlink d0/d7/d9a/db3/ld7 0 2024-04-26T04:53:23.003 INFO:tasks.workunit.client.0.smithi077.stdout:7/905: stat da/d13/d22/d7d/d80/d113/d119/l11c 0 2024-04-26T04:53:23.004 INFO:tasks.workunit.client.0.smithi077.stdout:3/887: dread - d0/d7/d1f/d81/dc5/ff8 zero size 2024-04-26T04:53:23.008 INFO:tasks.workunit.client.0.smithi077.stdout:4/972: write d4/d17/d18/d43/d51/db4/d15/ff3 [263922,67477] 0 2024-04-26T04:53:23.010 INFO:tasks.workunit.client.0.smithi077.stdout:4/973: chown d4/d17/d18/d43/d6e/d6c/dae/dc5/f106 4713881 1 2024-04-26T04:53:23.012 INFO:tasks.workunit.client.0.smithi077.stdout:9/809: mkdir d9/d45/d10a 0 2024-04-26T04:53:23.013 INFO:tasks.workunit.client.0.smithi077.stdout:9/810: write d9/d7d/d94/ff6 [1891852,81930] 0 2024-04-26T04:53:23.015 INFO:tasks.workunit.client.0.smithi077.stdout:7/906: truncate da/d15/d1a/f79 886357 0 2024-04-26T04:53:23.018 INFO:tasks.workunit.client.0.smithi077.stdout:3/888: write d0/f9d [273857,11927] 0 2024-04-26T04:53:23.029 INFO:tasks.workunit.client.0.smithi077.stdout:7/907: creat da/d15/d1a/d111/f131 x:0 0 0 2024-04-26T04:53:23.034 INFO:tasks.workunit.client.0.smithi077.stdout:7/908: mkdir da/d6b/d132 0 2024-04-26T04:53:23.035 INFO:tasks.workunit.client.0.smithi077.stdout:4/974: creat d4/d17/d18/d43/d63/f149 x:0 0 0 2024-04-26T04:53:23.039 INFO:tasks.workunit.client.0.smithi077.stdout:3/889: rename d0/dae/ded/d29/dc8 to d0/d7/d11d 0 2024-04-26T04:53:23.041 INFO:tasks.workunit.client.0.smithi077.stdout:3/890: truncate d0/d7/d1f/d81/fde 13038 0 2024-04-26T04:53:23.042 INFO:tasks.workunit.client.0.smithi077.stdout:4/975: symlink d4/d17/d18/d43/df7/l14a 0 2024-04-26T04:53:23.044 INFO:tasks.workunit.client.0.smithi077.stdout:7/909: rename da/d13/d22/cc9 to da/d6b/c133 0 2024-04-26T04:53:23.046 INFO:tasks.workunit.client.0.smithi077.stdout:3/891: chown d0/dae/l119 3 1 2024-04-26T04:53:23.047 INFO:tasks.workunit.client.0.smithi077.stdout:7/910: truncate da/d13/d22/d34/f103 827453 0 2024-04-26T04:53:23.047 INFO:tasks.workunit.client.0.smithi077.stdout:3/892: readlink d0/dae/ded/d29/db8/dbf/l114 0 2024-04-26T04:53:23.049 INFO:tasks.workunit.client.0.smithi077.stdout:4/976: mknod d4/d4e/dad/d8b/ddc/c14b 0 2024-04-26T04:53:23.051 INFO:tasks.workunit.client.0.smithi077.stdout:3/893: read d0/d9/ff4 [2763853,81356] 0 2024-04-26T04:53:23.052 INFO:tasks.workunit.client.0.smithi077.stdout:3/894: chown d0/dae/ded/d29/db8/dbf/l114 636 1 2024-04-26T04:53:23.058 INFO:tasks.workunit.client.0.smithi077.stdout:3/895: rmdir d0/d7/d1f 39 2024-04-26T04:53:23.063 INFO:tasks.workunit.client.0.smithi077.stdout:7/911: dread da/d15/dc6/fda [0,4194304] 0 2024-04-26T04:53:23.065 INFO:tasks.workunit.client.0.smithi077.stdout:4/977: getdents d4 0 2024-04-26T04:53:23.065 INFO:tasks.workunit.client.0.smithi077.stdout:7/912: truncate da/d13/feb 641614 0 2024-04-26T04:53:23.066 INFO:tasks.workunit.client.0.smithi077.stdout:3/896: mkdir d0/d7/d1f/d4d/dbe/d11e 0 2024-04-26T04:53:23.066 INFO:tasks.workunit.client.0.smithi077.stdout:4/978: dread - d4/d4e/dad/d8b/dba/f11d zero size 2024-04-26T04:53:23.067 INFO:tasks.workunit.client.0.smithi077.stdout:4/979: write d4/d17/d18/f139 [856173,23231] 0 2024-04-26T04:53:23.068 INFO:tasks.workunit.client.0.smithi077.stdout:7/913: dread - da/d15/d1a/d39/d96/fbf zero size 2024-04-26T04:53:23.072 INFO:tasks.workunit.client.0.smithi077.stdout:3/897: dread d0/d7/d1f/f60 [0,4194304] 0 2024-04-26T04:53:23.074 INFO:tasks.workunit.client.0.smithi077.stdout:4/980: dwrite d4/d4e/dad/ffd [0,4194304] 0 2024-04-26T04:53:23.084 INFO:tasks.workunit.client.0.smithi077.stdout:7/914: getdents da/d13/d22/d5d/d9f/ddd 0 2024-04-26T04:53:23.087 INFO:tasks.workunit.client.0.smithi077.stdout:4/981: dread d4/d17/d18/d43/fb5 [0,4194304] 0 2024-04-26T04:53:23.249 INFO:tasks.workunit.client.0.smithi077.stdout:4/982: rename d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/c9f to d4/d4e/dad/d8b/dba/de5/c14c 0 2024-04-26T04:53:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:22 smithi078 ceph-mon[33746]: pgmap v31: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 73 MiB/s rd, 176 MiB/s wr, 452 op/s 2024-04-26T04:53:23.269 INFO:tasks.workunit.client.0.smithi077.stdout:7/915: write da/d6b/d63/d64/fd9 [809201,91119] 0 2024-04-26T04:53:23.270 INFO:tasks.workunit.client.0.smithi077.stdout:4/983: write d4/d17/d18/d43/d51/db4/fa6 [729461,18838] 0 2024-04-26T04:53:23.273 INFO:tasks.workunit.client.0.smithi077.stdout:7/916: mknod da/d13/d22/d5d/d9f/ddd/c134 0 2024-04-26T04:53:23.274 INFO:tasks.workunit.client.0.smithi077.stdout:7/917: chown da/d13/d22/d5d/d8d/lc7 444877807 1 2024-04-26T04:53:23.274 INFO:tasks.workunit.client.0.smithi077.stdout:4/984: chown d4/d17/d18/d43/df7/l107 322 1 2024-04-26T04:53:23.277 INFO:tasks.workunit.client.0.smithi077.stdout:7/918: creat da/d13/d22/d7d/d80/d113/f135 x:0 0 0 2024-04-26T04:53:23.281 INFO:tasks.workunit.client.0.smithi077.stdout:7/919: truncate da/d13/d22/f2e 2180326 0 2024-04-26T04:53:23.281 INFO:tasks.workunit.client.0.smithi077.stdout:4/985: dwrite d4/d17/d18/d43/d51/db4/d15/d1f/d25/db7/de0/f8e [0,4194304] 0 2024-04-26T04:53:23.282 INFO:tasks.workunit.client.0.smithi077.stdout:4/986: dread - d4/d4e/dad/d8b/ddc/f12e zero size 2024-04-26T04:53:23.288 INFO:tasks.workunit.client.0.smithi077.stdout:7/920: creat da/d13/d22/f136 x:0 0 0 2024-04-26T04:53:23.292 INFO:tasks.workunit.client.0.smithi077.stdout:7/921: mkdir da/d13/d22/d5d/d9f/ddd/d137 0 2024-04-26T04:53:23.298 INFO:tasks.workunit.client.0.smithi077.stdout:7/922: mknod da/d13/c138 0 2024-04-26T04:53:23.299 INFO:tasks.workunit.client.0.smithi077.stdout:4/987: dwrite d4/d17/d18/d43/d51/db4/f2b [0,4194304] 0 2024-04-26T04:53:23.299 INFO:tasks.workunit.client.0.smithi077.stdout:7/923: chown da/d6b/d63/d64/f83 19052968 1 2024-04-26T04:53:23.300 INFO:tasks.workunit.client.0.smithi077.stdout:4/988: dread - d4/d4e/dad/d8b/ddc/f12e zero size 2024-04-26T04:53:23.301 INFO:tasks.workunit.client.0.smithi077.stdout:7/924: dread da/d6b/d63/d64/db0/fee [0,4194304] 0 2024-04-26T04:53:23.308 INFO:tasks.workunit.client.0.smithi077.stdout:7/925: dwrite da/d13/d22/f136 [0,4194304] 0 2024-04-26T04:53:23.338 INFO:tasks.workunit.client.0.smithi077.stdout:4/989: dwrite d4/d17/d18/d43/d51/db4/d15/f6d [0,4194304] 0 2024-04-26T04:53:23.350 INFO:tasks.workunit.client.0.smithi077.stdout:4/990: link d4/d4e/fd4 d4/d17/d18/d43/d63/d80/f14d 0 2024-04-26T04:53:23.351 INFO:tasks.workunit.client.0.smithi077.stdout:1/977: sync 2024-04-26T04:53:23.352 INFO:tasks.workunit.client.0.smithi077.stdout:1/978: write d9/d53/f86 [449407,48037] 0 2024-04-26T04:53:23.355 INFO:tasks.workunit.client.0.smithi077.stdout:1/979: symlink d9/d90/de0/d107/l137 0 2024-04-26T04:53:23.356 INFO:tasks.workunit.client.0.smithi077.stdout:1/980: fsync d9/db/d2b/f124 0 2024-04-26T04:53:23.358 INFO:tasks.workunit.client.0.smithi077.stdout:4/991: getdents d4/d4e/dad/df8 0 2024-04-26T04:53:23.359 INFO:tasks.workunit.client.0.smithi077.stdout:4/992: truncate d4/d4e/dad/d8b/dba/f11d 225472 0 2024-04-26T04:53:23.360 INFO:tasks.workunit.client.0.smithi077.stdout:1/981: rename d9/d53/d6e/d7c/f82 to d9/d53/d6e/d7c/d130/f138 0 2024-04-26T04:53:23.360 INFO:tasks.workunit.client.0.smithi077.stdout:4/993: chown d4/d4e/dad/d8b/dba/f132 1 1 2024-04-26T04:53:23.360 INFO:tasks.workunit.client.0.smithi077.stdout:1/982: chown d9/db 2174 1 2024-04-26T04:53:23.362 INFO:tasks.workunit.client.0.smithi077.stdout:4/994: read d4/d17/d18/d43/d53/f79 [1147023,124643] 0 2024-04-26T04:53:23.362 INFO:tasks.workunit.client.0.smithi077.stdout:4/995: chown d4/d17/d7f/feb 17163174 1 2024-04-26T04:53:23.363 INFO:tasks.workunit.client.0.smithi077.stdout:4/996: stat d4/d17/d18/d43/d53/d64/l70 0 2024-04-26T04:53:23.366 INFO:tasks.workunit.client.0.smithi077.stdout:1/983: write d9/db/dd/d68/f7b [3956037,6102] 0 2024-04-26T04:53:23.368 INFO:tasks.workunit.client.0.smithi077.stdout:4/997: rename d4/d17/d18/d43/df7 to d4/d17/d18/d43/d51/db4/d15/d1f/d35/d115/d14e 0 2024-04-26T04:53:23.369 INFO:tasks.workunit.client.0.smithi077.stdout:4/998: chown d4/d17/d18/fe2 3 1 2024-04-26T04:53:23.370 INFO:tasks.workunit.client.0.smithi077.stdout:1/984: mknod d9/d53/d6e/dd3/c139 0 2024-04-26T04:53:23.375 INFO:tasks.workunit.client.0.smithi077.stdout:1/985: getdents d9/d53/dc1 0 2024-04-26T04:53:23.379 INFO:tasks.workunit.client.0.smithi077.stdout:1/986: unlink d9/db/dd/d9c/c116 0 2024-04-26T04:53:23.380 INFO:tasks.workunit.client.0.smithi077.stdout:1/987: write d9/fec [763547,111610] 0 2024-04-26T04:53:23.380 INFO:tasks.workunit.client.0.smithi077.stdout:4/999: dwrite d4/d17/d18/d43/d63/d80/fbd [0,4194304] 0 2024-04-26T04:53:23.392 INFO:tasks.workunit.client.0.smithi077.stdout:1/988: dread d9/d90/d99/fd5 [0,4194304] 0 2024-04-26T04:53:23.394 INFO:tasks.workunit.client.0.smithi077.stdout:1/989: stat d9/db/dd/d9c/dab/de8/df3/cfa 0 2024-04-26T04:53:23.396 INFO:tasks.workunit.client.0.smithi077.stdout:1/990: symlink d9/d90/d136/l13a 0 2024-04-26T04:53:23.398 INFO:tasks.workunit.client.0.smithi077.stdout:1/991: rmdir d9/db/dd/d9c/dab 39 2024-04-26T04:53:23.399 INFO:tasks.workunit.client.0.smithi077.stdout:1/992: rename d9/d53/d8b to d9/d53/d8b/d93/da8/da7/d13b 22 2024-04-26T04:53:23.404 INFO:tasks.workunit.client.0.smithi077.stdout:1/993: dwrite d9/db/dd/d34/d63/f77 [0,4194304] 0 2024-04-26T04:53:23.419 INFO:tasks.workunit.client.0.smithi077.stdout:1/994: dwrite d9/db/d2b/d3f/f52 [0,4194304] 0 2024-04-26T04:53:23.430 INFO:tasks.workunit.client.0.smithi077.stdout:1/995: mknod d9/db/dd/d9c/dab/de8/df3/c13c 0 2024-04-26T04:53:23.431 INFO:tasks.workunit.client.0.smithi077.stdout:1/996: read d9/d53/d6e/d7c/fe1 [380682,47906] 0 2024-04-26T04:53:23.434 INFO:tasks.workunit.client.0.smithi077.stdout:1/997: truncate d9/db/fd1 70037 0 2024-04-26T04:53:23.438 INFO:tasks.workunit.client.0.smithi077.stdout:1/998: creat d9/d53/d8b/d93/da8/dd7/ddc/d105/f13d x:0 0 0 2024-04-26T04:53:23.443 INFO:tasks.workunit.client.0.smithi077.stdout:1/999: creat d9/db/f13e x:0 0 0 2024-04-26T04:53:23.534 INFO:tasks.workunit.client.0.smithi077.stdout:0/836: sync 2024-04-26T04:53:23.536 INFO:tasks.workunit.client.0.smithi077.stdout:0/837: truncate d0/d19/d1e/d59/fde 3017275 0 2024-04-26T04:53:23.542 INFO:tasks.workunit.client.0.smithi077.stdout:0/838: creat d0/d19/f11c x:0 0 0 2024-04-26T04:53:23.545 INFO:tasks.workunit.client.0.smithi077.stdout:0/839: symlink d0/d19/d1e/d59/d96/l11d 0 2024-04-26T04:53:23.555 INFO:tasks.workunit.client.0.smithi077.stdout:3/898: sync 2024-04-26T04:53:23.555 INFO:tasks.workunit.client.0.smithi077.stdout:5/877: sync 2024-04-26T04:53:23.555 INFO:tasks.workunit.client.0.smithi077.stdout:6/936: sync 2024-04-26T04:53:23.555 INFO:tasks.workunit.client.0.smithi077.stdout:7/926: sync 2024-04-26T04:53:23.555 INFO:tasks.workunit.client.0.smithi077.stdout:9/811: sync 2024-04-26T04:53:23.556 INFO:tasks.workunit.client.0.smithi077.stdout:6/937: chown d8/d52/da0/f137 134 1 2024-04-26T04:53:23.557 INFO:tasks.workunit.client.0.smithi077.stdout:5/878: fsync d1/d42/d62/dc8/dd5/f102 0 2024-04-26T04:53:23.559 INFO:tasks.workunit.client.0.smithi077.stdout:0/840: write d0/d19/f32 [5179925,28588] 0 2024-04-26T04:53:23.561 INFO:tasks.workunit.client.0.smithi077.stdout:3/899: mkdir d0/d7/d1f/d4d/dc6/d11f 0 2024-04-26T04:53:23.561 INFO:tasks.workunit.client.0.smithi077.stdout:9/812: dread d9/dd/d3d/d7c/dc8/fda [0,4194304] 0 2024-04-26T04:53:23.563 INFO:tasks.workunit.client.0.smithi077.stdout:9/813: chown d9/d3a/d79/d95/dad/de7/f108 3 1 2024-04-26T04:53:23.564 INFO:tasks.workunit.client.0.smithi077.stdout:7/927: mkdir da/d15/d139 0 2024-04-26T04:53:23.570 INFO:tasks.workunit.client.0.smithi077.stdout:3/900: unlink d0/dae/ded/d2a/d85/dd3/l118 0 2024-04-26T04:53:23.571 INFO:tasks.workunit.client.0.smithi077.stdout:3/901: write d0/dae/fe2 [192714,62249] 0 2024-04-26T04:53:23.572 INFO:tasks.workunit.client.0.smithi077.stdout:9/814: readlink d9/dd/l20 0 2024-04-26T04:53:23.576 INFO:tasks.workunit.client.0.smithi077.stdout:0/841: symlink d0/d19/d1e/d105/l11e 0 2024-04-26T04:53:23.578 INFO:tasks.workunit.client.0.smithi077.stdout:5/879: write d1/d8d/da7/fb4 [5037465,13508] 0 2024-04-26T04:53:23.578 INFO:tasks.workunit.client.0.smithi077.stdout:6/938: dread d8/d52/f87 [0,4194304] 0 2024-04-26T04:53:23.581 INFO:tasks.workunit.client.0.smithi077.stdout:9/815: fsync d9/d35/d99/fa3 0 2024-04-26T04:53:23.585 INFO:tasks.workunit.client.0.smithi077.stdout:7/928: dread da/f87 [0,4194304] 0 2024-04-26T04:53:23.586 INFO:tasks.workunit.client.0.smithi077.stdout:6/939: dwrite d8/d52/da0/f137 [0,4194304] 0 2024-04-26T04:53:23.587 INFO:tasks.workunit.client.0.smithi077.stdout:5/880: dread - d1/d42/d62/d6b/de8/f10b zero size 2024-04-26T04:53:23.587 INFO:tasks.workunit.client.0.smithi077.stdout:6/940: readlink d8/d1c/l31 0 2024-04-26T04:53:23.597 INFO:tasks.workunit.client.0.smithi077.stdout:9/816: symlink d9/dd/df2/d107/l10b 0 2024-04-26T04:53:23.598 INFO:tasks.workunit.client.0.smithi077.stdout:7/929: mkdir da/d13/d22/d7d/dfd/d13a 0 2024-04-26T04:53:23.599 INFO:tasks.workunit.client.0.smithi077.stdout:3/902: dwrite d0/dae/ded/d2a/f40 [0,4194304] 0 2024-04-26T04:53:23.599 INFO:tasks.workunit.client.0.smithi077.stdout:0/842: dread d0/d19/d1e/d2f/d52/f62 [0,4194304] 0 2024-04-26T04:53:23.601 INFO:tasks.workunit.client.0.smithi077.stdout:0/843: chown d0/d47/d64/d97/dba/dc4 16703 1 2024-04-26T04:53:23.601 INFO:tasks.workunit.client.0.smithi077.stdout:7/930: chown da/d6b/d63/d64/fbc 504947 1 2024-04-26T04:53:23.601 INFO:tasks.workunit.client.0.smithi077.stdout:9/817: mkdir d9/d43/d10c 0 2024-04-26T04:53:23.604 INFO:tasks.workunit.client.0.smithi077.stdout:0/844: read d0/d47/d10a/ff3 [63330,67572] 0 2024-04-26T04:53:23.606 INFO:tasks.workunit.client.0.smithi077.stdout:5/881: rename d1/d12 to d1/d42/d62/d121 0 2024-04-26T04:53:23.608 INFO:tasks.workunit.client.0.smithi077.stdout:3/903: fdatasync d0/d7/d1f/d4d/f5f 0 2024-04-26T04:53:23.608 INFO:tasks.workunit.client.0.smithi077.stdout:7/931: mknod da/d13/d22/d7d/c13b 0 2024-04-26T04:53:23.609 INFO:tasks.workunit.client.0.smithi077.stdout:7/932: stat da/d6b/d63/f81 0 2024-04-26T04:53:23.611 INFO:tasks.workunit.client.0.smithi077.stdout:9/818: rename d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/f102 to d9/dd/d1b/d2b/d32/d36/d7f/dae/f10d 0 2024-04-26T04:53:23.615 INFO:tasks.workunit.client.0.smithi077.stdout:3/904: creat d0/dae/ded/d29/db8/dec/f120 x:0 0 0 2024-04-26T04:53:23.618 INFO:tasks.workunit.client.0.smithi077.stdout:7/933: mkdir da/d13/d22/d5d/d9f/da7/d13c 0 2024-04-26T04:53:23.622 INFO:tasks.workunit.client.0.smithi077.stdout:7/934: dread - da/d13/d22/d5d/d9f/fd5 zero size 2024-04-26T04:53:23.624 INFO:tasks.workunit.client.0.smithi077.stdout:6/941: dwrite d8/f22 [0,4194304] 0 2024-04-26T04:53:23.624 INFO:tasks.workunit.client.0.smithi077.stdout:3/905: mknod d0/d7/c121 0 2024-04-26T04:53:23.625 INFO:tasks.workunit.client.0.smithi077.stdout:7/935: truncate da/d13/d22/d5d/dbd/f12e 249243 0 2024-04-26T04:53:23.626 INFO:tasks.workunit.client.0.smithi077.stdout:7/936: stat da/d15/d1a/d39 0 2024-04-26T04:53:23.628 INFO:tasks.workunit.client.0.smithi077.stdout:0/845: write d0/d47/d68/fe5 [829295,12597] 0 2024-04-26T04:53:23.628 INFO:tasks.workunit.client.0.smithi077.stdout:5/882: dread d1/d42/d62/d121/d28/d38/fb5 [0,4194304] 0 2024-04-26T04:53:23.629 INFO:tasks.workunit.client.0.smithi077.stdout:5/883: readlink d1/d8d/ld8 0 2024-04-26T04:53:23.631 INFO:tasks.workunit.client.0.smithi077.stdout:3/906: creat d0/dae/ded/d2a/d85/dd3/f122 x:0 0 0 2024-04-26T04:53:23.631 INFO:tasks.workunit.client.0.smithi077.stdout:9/819: getdents d9/d45/dcd 0 2024-04-26T04:53:23.632 INFO:tasks.workunit.client.0.smithi077.stdout:9/820: stat d9/dd/d1b/d2b/ddb/l93 0 2024-04-26T04:53:23.633 INFO:tasks.workunit.client.0.smithi077.stdout:6/942: mkdir d8/d2a/d14a 0 2024-04-26T04:53:23.635 INFO:tasks.workunit.client.0.smithi077.stdout:0/846: symlink d0/d19/d8d/dee/l11f 0 2024-04-26T04:53:23.642 INFO:tasks.workunit.client.0.smithi077.stdout:9/821: symlink d9/d35/l10e 0 2024-04-26T04:53:23.643 INFO:tasks.workunit.client.0.smithi077.stdout:6/943: truncate d8/f48 1433489 0 2024-04-26T04:53:23.646 INFO:tasks.workunit.client.0.smithi077.stdout:0/847: mkdir d0/d19/d1e/d59/d96/d106/d120 0 2024-04-26T04:53:23.647 INFO:tasks.workunit.client.0.smithi077.stdout:5/884: getdents d1/d42/d62/d121/d45 0 2024-04-26T04:53:23.650 INFO:tasks.workunit.client.0.smithi077.stdout:3/907: dwrite d0/d7/d1f/d4d/f56 [0,4194304] 0 2024-04-26T04:53:23.651 INFO:tasks.workunit.client.0.smithi077.stdout:7/937: link da/d13/d22/d5d/c91 da/d13/d22/d7d/d80/d113/d119/c13d 0 2024-04-26T04:53:23.660 INFO:tasks.workunit.client.0.smithi077.stdout:5/885: creat d1/d42/d62/d121/d87/f122 x:0 0 0 2024-04-26T04:53:23.662 INFO:tasks.workunit.client.0.smithi077.stdout:7/938: creat da/d15/d1a/d39/f13e x:0 0 0 2024-04-26T04:53:23.664 INFO:tasks.workunit.client.0.smithi077.stdout:6/944: link d8/d1c/c9e d8/d52/d53/da7/df4/c14b 0 2024-04-26T04:53:23.665 INFO:tasks.workunit.client.0.smithi077.stdout:9/822: dwrite d9/dd/d1b/d2b/f2f [0,4194304] 0 2024-04-26T04:53:23.666 INFO:tasks.workunit.client.0.smithi077.stdout:7/939: write da/d13/d22/d7d/d80/d113/f135 [910465,12771] 0 2024-04-26T04:53:23.667 INFO:tasks.workunit.client.0.smithi077.stdout:0/848: creat d0/d19/f121 x:0 0 0 2024-04-26T04:53:23.671 INFO:tasks.workunit.client.0.smithi077.stdout:6/945: fdatasync d8/d52/d53/d5e/da6/d133/d139/f105 0 2024-04-26T04:53:23.674 INFO:tasks.workunit.client.0.smithi077.stdout:0/849: unlink d0/d19/d1e/d2f/d52/lda 0 2024-04-26T04:53:23.680 INFO:tasks.workunit.client.0.smithi077.stdout:9/823: mkdir d9/dd/dec/d10f 0 2024-04-26T04:53:23.680 INFO:tasks.workunit.client.0.smithi077.stdout:9/824: readlink d9/l1d 0 2024-04-26T04:53:23.680 INFO:tasks.workunit.client.0.smithi077.stdout:9/825: chown d9/d3a/d79/d95/feb 6532034 1 2024-04-26T04:53:23.681 INFO:tasks.workunit.client.0.smithi077.stdout:6/946: truncate d8/d52/d53/ff0 167077 0 2024-04-26T04:53:23.682 INFO:tasks.workunit.client.0.smithi077.stdout:6/947: chown d8/c12d 116 1 2024-04-26T04:53:23.683 INFO:tasks.workunit.client.0.smithi077.stdout:0/850: truncate d0/d47/f22 38418 0 2024-04-26T04:53:23.685 INFO:tasks.workunit.client.0.smithi077.stdout:5/886: dread d1/d42/da4/dd3/ffb [0,4194304] 0 2024-04-26T04:53:23.686 INFO:tasks.workunit.client.0.smithi077.stdout:7/940: mknod da/d13/d22/d5d/dbd/c13f 0 2024-04-26T04:53:23.687 INFO:tasks.workunit.client.0.smithi077.stdout:9/826: dwrite d9/f82 [0,4194304] 0 2024-04-26T04:53:23.687 INFO:tasks.workunit.client.0.smithi077.stdout:7/941: stat da/d15/d42/la9 0 2024-04-26T04:53:23.689 INFO:tasks.workunit.client.0.smithi077.stdout:3/908: getdents d0/dae/ded/d2a/d85 0 2024-04-26T04:53:23.691 INFO:tasks.workunit.client.0.smithi077.stdout:6/948: fdatasync d8/d52/da0/f9a 0 2024-04-26T04:53:23.694 INFO:tasks.workunit.client.0.smithi077.stdout:5/887: fsync d1/d8/d72/fa2 0 2024-04-26T04:53:23.702 INFO:tasks.workunit.client.0.smithi077.stdout:6/949: creat d8/d52/d11c/f14c x:0 0 0 2024-04-26T04:53:23.741 INFO:tasks.workunit.client.0.smithi077.stdout:6/950: mknod d8/d1c/d108/c14d 0 2024-04-26T04:53:23.743 INFO:tasks.workunit.client.0.smithi077.stdout:3/909: dread d0/f1b [0,4194304] 0 2024-04-26T04:53:23.745 INFO:tasks.workunit.client.0.smithi077.stdout:0/851: rename d0/d19/d69/c7e to d0/d28/c122 0 2024-04-26T04:53:23.746 INFO:tasks.workunit.client.0.smithi077.stdout:6/951: readlink d8/d1c/da4/d10f/d11d/l122 0 2024-04-26T04:53:23.747 INFO:tasks.workunit.client.0.smithi077.stdout:5/888: dwrite d1/d42/d62/d121/d28/d38/f8b [0,4194304] 0 2024-04-26T04:53:23.752 INFO:tasks.workunit.client.0.smithi077.stdout:7/942: dread da/d15/d42/f50 [0,4194304] 0 2024-04-26T04:53:23.756 INFO:tasks.workunit.client.0.smithi077.stdout:3/910: truncate d0/fb 4414084 0 2024-04-26T04:53:23.761 INFO:tasks.workunit.client.0.smithi077.stdout:6/952: getdents d8/d52/d53/d5e/d6c 0 2024-04-26T04:53:23.762 INFO:tasks.workunit.client.0.smithi077.stdout:3/911: symlink d0/d9/l123 0 2024-04-26T04:53:23.765 INFO:tasks.workunit.client.0.smithi077.stdout:6/953: rmdir d8/d52/d53/d5e/da6/de9 39 2024-04-26T04:53:23.768 INFO:tasks.workunit.client.0.smithi077.stdout:3/912: mknod d0/dae/ded/d29/db8/dbf/c124 0 2024-04-26T04:53:23.770 INFO:tasks.workunit.client.0.smithi077.stdout:0/852: dread d0/f1 [0,4194304] 0 2024-04-26T04:53:23.772 INFO:tasks.workunit.client.0.smithi077.stdout:7/943: write da/d15/faa [1997993,125467] 0 2024-04-26T04:53:23.774 INFO:tasks.workunit.client.0.smithi077.stdout:5/889: dread d1/d8/f55 [0,4194304] 0 2024-04-26T04:53:23.777 INFO:tasks.workunit.client.0.smithi077.stdout:3/913: link d0/d7/d1f/d4d/f56 d0/d7/d1f/d4d/dc6/d11f/f125 0 2024-04-26T04:53:23.778 INFO:tasks.workunit.client.0.smithi077.stdout:7/944: unlink da/d13/d22/d5d/d9f/da7/de0/l128 0 2024-04-26T04:53:23.779 INFO:tasks.workunit.client.0.smithi077.stdout:5/890: mknod d1/d8c/c123 0 2024-04-26T04:53:23.781 INFO:tasks.workunit.client.0.smithi077.stdout:6/954: truncate d8/f17 2333762 0 2024-04-26T04:53:23.782 INFO:tasks.workunit.client.0.smithi077.stdout:3/914: truncate d0/f94 189368 0 2024-04-26T04:53:23.782 INFO:tasks.workunit.client.0.smithi077.stdout:0/853: write d0/d19/d1e/d2f/d52/f76 [2145830,55564] 0 2024-04-26T04:53:23.783 INFO:tasks.workunit.client.0.smithi077.stdout:3/915: fsync d0/d7/f54 0 2024-04-26T04:53:23.785 INFO:tasks.workunit.client.0.smithi077.stdout:5/891: creat d1/d8/d7e/f124 x:0 0 0 2024-04-26T04:53:23.786 INFO:tasks.workunit.client.0.smithi077.stdout:5/892: fsync d1/d42/d62/dc8/dd5/f102 0 2024-04-26T04:53:23.786 INFO:tasks.workunit.client.0.smithi077.stdout:5/893: readlink d1/d8d/ld1 0 2024-04-26T04:53:23.787 INFO:tasks.workunit.client.0.smithi077.stdout:3/916: chown d0/d7/d9a/db3/f3f 1016330 1 2024-04-26T04:53:23.788 INFO:tasks.workunit.client.0.smithi077.stdout:0/854: write d0/d47/d64/d97/dba/dc4/f10c [1625027,93963] 0 2024-04-26T04:53:23.788 INFO:tasks.workunit.client.0.smithi077.stdout:7/945: creat da/f140 x:0 0 0 2024-04-26T04:53:23.789 INFO:tasks.workunit.client.0.smithi077.stdout:3/917: dread - d0/d7/d9a/db7/ddb/f117 zero size 2024-04-26T04:53:23.789 INFO:tasks.workunit.client.0.smithi077.stdout:0/855: chown d0/d47/d93/d94/faa 21373 1 2024-04-26T04:53:23.789 INFO:tasks.workunit.client.0.smithi077.stdout:7/946: stat da/d15/d1a/f65 0 2024-04-26T04:53:23.790 INFO:tasks.workunit.client.0.smithi077.stdout:3/918: fdatasync d0/d7/d9a/db3/f22 0 2024-04-26T04:53:23.791 INFO:tasks.workunit.client.0.smithi077.stdout:0/856: fdatasync d0/d19/d1e/d59/d96/f110 0 2024-04-26T04:53:23.792 INFO:tasks.workunit.client.0.smithi077.stdout:6/955: link d8/d52/d53/d5e/cfc d8/d1c/da4/d10f/d11d/c14e 0 2024-04-26T04:53:23.797 INFO:tasks.workunit.client.0.smithi077.stdout:3/919: mkdir d0/dae/ded/d29/db8/dbf/d126 0 2024-04-26T04:53:23.801 INFO:tasks.workunit.client.0.smithi077.stdout:0/857: rename d0/d47/d64/d97/cef to d0/d19/d1e/d82/c123 0 2024-04-26T04:53:23.802 INFO:tasks.workunit.client.0.smithi077.stdout:0/858: dread - d0/d47/ff0 zero size 2024-04-26T04:53:23.802 INFO:tasks.workunit.client.0.smithi077.stdout:6/956: dread - d8/d52/da0/ff7 zero size 2024-04-26T04:53:23.802 INFO:tasks.workunit.client.0.smithi077.stdout:6/957: readlink d8/d2a/lc9 0 2024-04-26T04:53:23.803 INFO:tasks.workunit.client.0.smithi077.stdout:0/859: truncate d0/d19/d69/fe0 1274110 0 2024-04-26T04:53:23.810 INFO:tasks.workunit.client.0.smithi077.stdout:5/894: dwrite d1/fe3 [0,4194304] 0 2024-04-26T04:53:23.813 INFO:tasks.workunit.client.0.smithi077.stdout:0/860: mkdir d0/d19/d1e/d59/d96/d106/d120/d124 0 2024-04-26T04:53:23.814 INFO:tasks.workunit.client.0.smithi077.stdout:5/895: chown d1/d30/d33/d66/lb3 1075764413 1 2024-04-26T04:53:23.815 INFO:tasks.workunit.client.0.smithi077.stdout:6/958: dwrite d8/d2a/ddd/f125 [0,4194304] 0 2024-04-26T04:53:23.817 INFO:tasks.workunit.client.0.smithi077.stdout:7/947: dwrite da/d15/d1a/f65 [0,4194304] 0 2024-04-26T04:53:23.827 INFO:tasks.workunit.client.0.smithi077.stdout:3/920: getdents d0/dae 0 2024-04-26T04:53:23.830 INFO:tasks.workunit.client.0.smithi077.stdout:5/896: symlink d1/d42/d62/d121/d45/l125 0 2024-04-26T04:53:23.834 INFO:tasks.workunit.client.0.smithi077.stdout:7/948: fsync da/ff 0 2024-04-26T04:53:23.836 INFO:tasks.workunit.client.0.smithi077.stdout:5/897: mkdir d1/d42/da4/dd3/d126 0 2024-04-26T04:53:23.837 INFO:tasks.workunit.client.0.smithi077.stdout:6/959: symlink d8/d52/d53/d5e/da6/l14f 0 2024-04-26T04:53:23.838 INFO:tasks.workunit.client.0.smithi077.stdout:0/861: dread d0/d47/d93/f9a [0,4194304] 0 2024-04-26T04:53:23.841 INFO:tasks.workunit.client.0.smithi077.stdout:6/960: truncate d8/d2a/f32 1842766 0 2024-04-26T04:53:23.843 INFO:tasks.workunit.client.0.smithi077.stdout:5/898: unlink d1/d42/d62/d121/d28/c40 0 2024-04-26T04:53:23.844 INFO:tasks.workunit.client.0.smithi077.stdout:0/862: symlink d0/d19/d1e/d59/d96/d106/d120/d124/l125 0 2024-04-26T04:53:23.845 INFO:tasks.workunit.client.0.smithi077.stdout:6/961: stat d8/d52/l119 0 2024-04-26T04:53:23.847 INFO:tasks.workunit.client.0.smithi077.stdout:7/949: creat da/d13/f141 x:0 0 0 2024-04-26T04:53:23.849 INFO:tasks.workunit.client.0.smithi077.stdout:5/899: chown d1/d42/d62/dc8/dd5/df8/dfd/l117 15 1 2024-04-26T04:53:23.850 INFO:tasks.workunit.client.0.smithi077.stdout:5/900: readlink d1/d8/lad 0 2024-04-26T04:53:23.851 INFO:tasks.workunit.client.0.smithi077.stdout:0/863: dread d0/d47/d93/d94/faa [0,4194304] 0 2024-04-26T04:53:23.853 INFO:tasks.workunit.client.0.smithi077.stdout:7/950: rename da/d6b/d63/d64/f67 to da/d13/d22/d7d/d80/d113/d119/f142 0 2024-04-26T04:53:23.857 INFO:tasks.workunit.client.0.smithi077.stdout:5/901: dwrite d1/d8d/da7/fb4 [0,4194304] 0 2024-04-26T04:53:23.858 INFO:tasks.workunit.client.0.smithi077.stdout:5/902: write d1/d42/d62/d121/d28/d38/fee [702364,43304] 0 2024-04-26T04:53:23.860 INFO:tasks.workunit.client.0.smithi077.stdout:0/864: creat d0/d47/d64/dfe/f126 x:0 0 0 2024-04-26T04:53:23.863 INFO:tasks.workunit.client.0.smithi077.stdout:7/951: rename da/d15/d1a/l6a to da/d15/d139/l143 0 2024-04-26T04:53:23.868 INFO:tasks.workunit.client.0.smithi077.stdout:7/952: readlink da/d13/l52 0 2024-04-26T04:53:23.873 INFO:tasks.workunit.client.0.smithi077.stdout:0/865: link d0/d47/d93/d94/fcd d0/d47/f127 0 2024-04-26T04:53:23.876 INFO:tasks.workunit.client.0.smithi077.stdout:6/962: dwrite d8/d52/f70 [0,4194304] 0 2024-04-26T04:53:23.883 INFO:tasks.workunit.client.0.smithi077.stdout:7/953: link da/d15/d1a/d111/f131 da/d13/d22/d7d/d80/d113/f144 0 2024-04-26T04:53:23.885 INFO:tasks.workunit.client.0.smithi077.stdout:0/866: truncate d0/d19/d1e/d2f/d52/f62 2435586 0 2024-04-26T04:53:23.888 INFO:tasks.workunit.client.0.smithi077.stdout:7/954: creat da/d13/d22/d5d/d8d/f145 x:0 0 0 2024-04-26T04:53:23.894 INFO:tasks.workunit.client.0.smithi077.stdout:7/955: dread da/d13/d118/f11d [0,4194304] 0 2024-04-26T04:53:23.898 INFO:tasks.workunit.client.0.smithi077.stdout:7/956: rmdir da/d13/d22/d7d/d80 39 2024-04-26T04:53:23.906 INFO:tasks.workunit.client.0.smithi077.stdout:6/963: dread d8/d1c/f28 [0,4194304] 0 2024-04-26T04:53:24.304 INFO:tasks.workunit.client.0.smithi077.stdout:9/827: sync 2024-04-26T04:53:24.308 INFO:tasks.workunit.client.0.smithi077.stdout:9/828: mknod d9/dd/d1b/d2b/d32/d36/d7f/c110 0 2024-04-26T04:53:24.309 INFO:tasks.workunit.client.0.smithi077.stdout:9/829: truncate d9/d3a/fea 1463928 0 2024-04-26T04:53:24.356 INFO:tasks.workunit.client.0.smithi077.stdout:9/830: creat d9/d35/f111 x:0 0 0 2024-04-26T04:53:24.361 INFO:tasks.workunit.client.0.smithi077.stdout:9/831: unlink d9/d3a/d79/d95/dad/de7/lfc 0 2024-04-26T04:53:24.362 INFO:tasks.workunit.client.0.smithi077.stdout:9/832: fsync d9/f26 0 2024-04-26T04:53:24.364 INFO:tasks.workunit.client.0.smithi077.stdout:9/833: mkdir d9/d43/d112 0 2024-04-26T04:53:24.370 INFO:tasks.workunit.client.0.smithi077.stdout:9/834: link d9/dd/d1b/d2b/d32/l98 d9/d45/dc4/df3/l113 0 2024-04-26T04:53:24.371 INFO:tasks.workunit.client.0.smithi077.stdout:9/835: creat d9/d43/f114 x:0 0 0 2024-04-26T04:53:24.375 INFO:tasks.workunit.client.0.smithi077.stdout:9/836: rename d9/d3a/d79/l49 to d9/dd/d3d/d7c/l115 0 2024-04-26T04:53:24.389 INFO:tasks.workunit.client.0.smithi077.stdout:9/837: dread d9/f2d [0,4194304] 0 2024-04-26T04:53:24.391 INFO:tasks.workunit.client.0.smithi077.stdout:9/838: read d9/dd/d1b/d2b/d32/d36/fa7 [1358806,118269] 0 2024-04-26T04:53:24.393 INFO:tasks.workunit.client.0.smithi077.stdout:9/839: readlink d9/d3a/d79/d95/l9d 0 2024-04-26T04:53:24.394 INFO:tasks.workunit.client.0.smithi077.stdout:9/840: mkdir d9/d45/dc4/d116 0 2024-04-26T04:53:24.398 INFO:tasks.workunit.client.0.smithi077.stdout:3/921: sync 2024-04-26T04:53:24.398 INFO:tasks.workunit.client.0.smithi077.stdout:5/903: sync 2024-04-26T04:53:24.401 INFO:tasks.workunit.client.0.smithi077.stdout:3/922: readlink d0/d9/l96 0 2024-04-26T04:53:24.405 INFO:tasks.workunit.client.0.smithi077.stdout:3/923: mkdir d0/d7/d1f/d81/dc5/dd1/d127 0 2024-04-26T04:53:24.409 INFO:tasks.workunit.client.0.smithi077.stdout:3/924: rename d0/dae/ded/d29/f6d to d0/d7/d1f/d81/dc5/dd1/d127/f128 0 2024-04-26T04:53:24.412 INFO:tasks.workunit.client.0.smithi077.stdout:3/925: mknod d0/dae/ded/d29/db8/c129 0 2024-04-26T04:53:24.412 INFO:tasks.workunit.client.0.smithi077.stdout:3/926: chown d0/d7/dce 1603 1 2024-04-26T04:53:24.496 INFO:tasks.workunit.client.0.smithi077.stdout:9/841: dwrite d9/d7d/fa1 [0,4194304] 0 2024-04-26T04:53:24.504 INFO:tasks.workunit.client.0.smithi077.stdout:9/842: fsync d9/d35/f4c 0 2024-04-26T04:53:24.507 INFO:tasks.workunit.client.0.smithi077.stdout:3/927: dwrite d0/d7/d1f/d4d/f56 [0,4194304] 0 2024-04-26T04:53:24.508 INFO:tasks.workunit.client.0.smithi077.stdout:3/928: dread - d0/d7/d1f/d4d/f113 zero size 2024-04-26T04:53:24.512 INFO:tasks.workunit.client.0.smithi077.stdout:9/843: truncate d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/fe1 168702 0 2024-04-26T04:53:24.516 INFO:tasks.workunit.client.0.smithi077.stdout:9/844: fdatasync d9/d3a/d79/f6b 0 2024-04-26T04:53:24.519 INFO:tasks.workunit.client.0.smithi077.stdout:9/845: symlink d9/d35/d99/l117 0 2024-04-26T04:53:24.521 INFO:tasks.workunit.client.0.smithi077.stdout:9/846: creat d9/dd/df2/f118 x:0 0 0 2024-04-26T04:53:24.522 INFO:tasks.workunit.client.0.smithi077.stdout:9/847: chown d9/d43/c70 11745 1 2024-04-26T04:53:24.528 INFO:tasks.workunit.client.0.smithi077.stdout:9/848: getdents d9/d35 0 2024-04-26T04:53:24.539 INFO:tasks.workunit.client.0.smithi077.stdout:9/849: dread d9/d7d/d94/ff6 [0,4194304] 0 2024-04-26T04:53:24.542 INFO:tasks.workunit.client.0.smithi077.stdout:9/850: mknod d9/d3a/d79/d95/c119 0 2024-04-26T04:53:24.547 INFO:tasks.workunit.client.0.smithi077.stdout:9/851: creat d9/d7d/f11a x:0 0 0 2024-04-26T04:53:24.549 INFO:tasks.workunit.client.0.smithi077.stdout:9/852: creat d9/dd/d1b/d2b/d32/d36/d7f/f11b x:0 0 0 2024-04-26T04:53:24.550 INFO:tasks.workunit.client.0.smithi077.stdout:9/853: dread - d9/d3a/d79/d95/dad/de7/f108 zero size 2024-04-26T04:53:24.550 INFO:tasks.workunit.client.0.smithi077.stdout:9/854: read - d9/d35/f111 zero size 2024-04-26T04:53:24.552 INFO:tasks.workunit.client.0.smithi077.stdout:9/855: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/d11c 0 2024-04-26T04:53:24.562 INFO:tasks.workunit.client.0.smithi077.stdout:9/856: dwrite d9/dd/fb3 [0,4194304] 0 2024-04-26T04:53:24.599 INFO:tasks.workunit.client.0.smithi077.stdout:0/867: sync 2024-04-26T04:53:24.599 INFO:tasks.workunit.client.0.smithi077.stdout:6/964: sync 2024-04-26T04:53:24.599 INFO:tasks.workunit.client.0.smithi077.stdout:7/957: sync 2024-04-26T04:53:24.602 INFO:tasks.workunit.client.0.smithi077.stdout:7/958: creat da/d15/dc6/f146 x:0 0 0 2024-04-26T04:53:24.603 INFO:tasks.workunit.client.0.smithi077.stdout:6/965: dread d8/d52/f87 [0,4194304] 0 2024-04-26T04:53:24.604 INFO:tasks.workunit.client.0.smithi077.stdout:7/959: fdatasync da/d6b/f93 0 2024-04-26T04:53:24.607 INFO:tasks.workunit.client.0.smithi077.stdout:6/966: symlink d8/d52/d53/d5e/da6/dd1/l150 0 2024-04-26T04:53:24.659 INFO:tasks.workunit.client.0.smithi077.stdout:7/960: symlink da/d6b/d63/d64/db0/l147 0 2024-04-26T04:53:24.660 INFO:tasks.workunit.client.0.smithi077.stdout:6/967: write f4 [3044735,96237] 0 2024-04-26T04:53:24.661 INFO:tasks.workunit.client.0.smithi077.stdout:7/961: readlink da/d13/d22/d5d/d9f/le9 0 2024-04-26T04:53:24.661 INFO:tasks.workunit.client.0.smithi077.stdout:0/868: dread d0/d47/f31 [0,4194304] 0 2024-04-26T04:53:24.661 INFO:tasks.workunit.client.0.smithi077.stdout:7/962: stat da/d13/f4b 0 2024-04-26T04:53:24.674 INFO:tasks.workunit.client.0.smithi077.stdout:6/968: write d8/d52/d53/d5e/da6/de9/f116 [872113,32153] 0 2024-04-26T04:53:24.675 INFO:tasks.workunit.client.0.smithi077.stdout:6/969: chown d8/f33 10 1 2024-04-26T04:53:24.676 INFO:tasks.workunit.client.0.smithi077.stdout:7/963: creat da/d13/d22/d5d/f148 x:0 0 0 2024-04-26T04:53:24.678 INFO:tasks.workunit.client.0.smithi077.stdout:0/869: dwrite d0/f3 [0,4194304] 0 2024-04-26T04:53:24.678 INFO:tasks.workunit.client.0.smithi077.stdout:6/970: creat d8/d52/d11c/f151 x:0 0 0 2024-04-26T04:53:24.678 INFO:tasks.workunit.client.0.smithi077.stdout:7/964: truncate da/d15/d42/f77 9356648 0 2024-04-26T04:53:24.687 INFO:tasks.workunit.client.0.smithi077.stdout:7/965: mknod da/d13/d22/d7d/d80/d113/c149 0 2024-04-26T04:53:24.692 INFO:tasks.workunit.client.0.smithi077.stdout:0/870: symlink d0/d19/d8d/df7/dfc/l128 0 2024-04-26T04:53:24.696 INFO:tasks.workunit.client.0.smithi077.stdout:6/971: dwrite d8/d52/da0/ff7 [0,4194304] 0 2024-04-26T04:53:24.703 INFO:tasks.workunit.client.0.smithi077.stdout:7/966: write da/d6b/d63/d64/fc1 [973467,12477] 0 2024-04-26T04:53:24.703 INFO:tasks.workunit.client.0.smithi077.stdout:6/972: dwrite d8/d52/d53/d5e/fb0 [0,4194304] 0 2024-04-26T04:53:24.710 INFO:tasks.workunit.client.0.smithi077.stdout:0/871: dwrite d0/d47/d93/d94/fcd [0,4194304] 0 2024-04-26T04:53:24.712 INFO:tasks.workunit.client.0.smithi077.stdout:6/973: getdents d8/d52/d53/d5e/da6/de9 0 2024-04-26T04:53:24.712 INFO:tasks.workunit.client.0.smithi077.stdout:0/872: write d0/d2c/f2e [3038237,12366] 0 2024-04-26T04:53:24.714 INFO:tasks.workunit.client.0.smithi077.stdout:0/873: write d0/d19/f11c [847690,115366] 0 2024-04-26T04:53:24.714 INFO:tasks.workunit.client.0.smithi077.stdout:6/974: chown d8/d2a/ddd/cfe 6674934 1 2024-04-26T04:53:24.719 INFO:tasks.workunit.client.0.smithi077.stdout:6/975: read d8/d2a/f2e [3958116,4842] 0 2024-04-26T04:53:24.719 INFO:tasks.workunit.client.0.smithi077.stdout:0/874: truncate d0/d47/f31 5258316 0 2024-04-26T04:53:24.721 INFO:tasks.workunit.client.0.smithi077.stdout:0/875: mkdir d0/d19/d8d/dee/d129 0 2024-04-26T04:53:24.722 INFO:tasks.workunit.client.0.smithi077.stdout:6/976: truncate d8/d2a/df2/fff 867018 0 2024-04-26T04:53:24.726 INFO:tasks.workunit.client.0.smithi077.stdout:6/977: rename d8/d52/d53/d5e/da6/lbd to d8/d52/d53/d5e/da6/ddf/l152 0 2024-04-26T04:53:24.726 INFO:tasks.workunit.client.0.smithi077.stdout:0/876: link d0/d47/d68/fa5 d0/d47/d93/f12a 0 2024-04-26T04:53:24.728 INFO:tasks.workunit.client.0.smithi077.stdout:6/978: creat d8/d1c/db5/f153 x:0 0 0 2024-04-26T04:53:24.729 INFO:tasks.workunit.client.0.smithi077.stdout:0/877: creat d0/d19/d1e/d59/d96/d106/d120/d124/f12b x:0 0 0 2024-04-26T04:53:24.730 INFO:tasks.workunit.client.0.smithi077.stdout:6/979: mkdir d8/d1c/d154 0 2024-04-26T04:53:24.735 INFO:tasks.workunit.client.0.smithi077.stdout:7/967: dread da/d6b/d63/d64/fbc [0,4194304] 0 2024-04-26T04:53:24.736 INFO:tasks.workunit.client.0.smithi077.stdout:0/878: dread d0/d47/d68/d9f/ff8 [0,4194304] 0 2024-04-26T04:53:24.742 INFO:tasks.workunit.client.0.smithi077.stdout:7/968: rmdir da/d6b/d132 0 2024-04-26T04:53:24.744 INFO:tasks.workunit.client.0.smithi077.stdout:6/980: dread d8/d52/d53/d5e/fae [0,4194304] 0 2024-04-26T04:53:24.748 INFO:tasks.workunit.client.0.smithi077.stdout:6/981: mknod d8/d52/d53/d5e/c155 0 2024-04-26T04:53:24.750 INFO:tasks.workunit.client.0.smithi077.stdout:6/982: chown d8/d52/l42 243 1 2024-04-26T04:53:24.753 INFO:tasks.workunit.client.0.smithi077.stdout:6/983: truncate d8/f48 2261644 0 2024-04-26T04:53:24.756 INFO:tasks.workunit.client.0.smithi077.stdout:6/984: rmdir d8/d52/d53/d5e/da6 39 2024-04-26T04:53:24.760 INFO:tasks.workunit.client.0.smithi077.stdout:7/969: write da/d6b/d63/f9e [941984,125814] 0 2024-04-26T04:53:24.762 INFO:tasks.workunit.client.0.smithi077.stdout:0/879: dwrite d0/d19/d1e/f9d [0,4194304] 0 2024-04-26T04:53:24.769 INFO:tasks.workunit.client.0.smithi077.stdout:0/880: rmdir d0/d28 39 2024-04-26T04:53:24.770 INFO:tasks.workunit.client.0.smithi077.stdout:0/881: fdatasync d0/d19/f121 0 2024-04-26T04:53:24.772 INFO:tasks.workunit.client.0.smithi077.stdout:7/970: creat da/d13/d22/d5d/d8d/d10c/f14a x:0 0 0 2024-04-26T04:53:24.773 INFO:tasks.workunit.client.0.smithi077.stdout:0/882: chown d0/d19/d1e/d2f/da8/d101 29208 1 2024-04-26T04:53:24.775 INFO:tasks.workunit.client.0.smithi077.stdout:6/985: dwrite d8/d52/d53/da7/dfa/f126 [0,4194304] 0 2024-04-26T04:53:24.778 INFO:tasks.workunit.client.0.smithi077.stdout:6/986: write d8/d2a/ddd/f125 [2689014,61176] 0 2024-04-26T04:53:24.779 INFO:tasks.workunit.client.0.smithi077.stdout:7/971: mknod da/c14b 0 2024-04-26T04:53:24.785 INFO:tasks.workunit.client.0.smithi077.stdout:6/987: dread - d8/d52/d53/da7/df4/f12f zero size 2024-04-26T04:53:24.785 INFO:tasks.workunit.client.0.smithi077.stdout:6/988: readlink d8/d52/da0/l8b 0 2024-04-26T04:53:24.789 INFO:tasks.workunit.client.0.smithi077.stdout:0/883: link d0/d47/d64/l4b d0/d19/d8d/df7/l12c 0 2024-04-26T04:53:24.790 INFO:tasks.workunit.client.0.smithi077.stdout:7/972: link da/d15/fe2 da/d13/d22/d5d/d8d/d10c/f14c 0 2024-04-26T04:53:24.791 INFO:tasks.workunit.client.0.smithi077.stdout:6/989: symlink d8/d52/d53/d5e/da6/de9/l156 0 2024-04-26T04:53:24.794 INFO:tasks.workunit.client.0.smithi077.stdout:6/990: rmdir d8/d52/d53/d5e/da6 39 2024-04-26T04:53:24.795 INFO:tasks.workunit.client.0.smithi077.stdout:6/991: chown d8/d1c/da4/fb6 2075634380 1 2024-04-26T04:53:24.797 INFO:tasks.workunit.client.0.smithi077.stdout:6/992: dread - d8/d52/d53/fcc zero size 2024-04-26T04:53:24.798 INFO:tasks.workunit.client.0.smithi077.stdout:7/973: dwrite da/d13/f19 [0,4194304] 0 2024-04-26T04:53:24.798 INFO:tasks.workunit.client.0.smithi077.stdout:7/974: dread - da/d6b/d63/ff5 zero size 2024-04-26T04:53:24.801 INFO:tasks.workunit.client.0.smithi077.stdout:6/993: rmdir d8/d2a 39 2024-04-26T04:53:24.803 INFO:tasks.workunit.client.0.smithi077.stdout:7/975: dread da/d13/d22/f136 [0,4194304] 0 2024-04-26T04:53:24.807 INFO:tasks.workunit.client.0.smithi077.stdout:7/976: readlink da/d15/d139/l143 0 2024-04-26T04:53:24.807 INFO:tasks.workunit.client.0.smithi077.stdout:7/977: chown da/d15/d1a/f10f 1939 1 2024-04-26T04:53:24.815 INFO:tasks.workunit.client.0.smithi077.stdout:0/884: dwrite d0/d47/fc8 [0,4194304] 0 2024-04-26T04:53:24.817 INFO:tasks.workunit.client.0.smithi077.stdout:0/885: truncate d0/d47/d64/dfe/f126 386931 0 2024-04-26T04:53:24.821 INFO:tasks.workunit.client.0.smithi077.stdout:0/886: creat d0/d47/d64/f12d x:0 0 0 2024-04-26T04:53:24.821 INFO:tasks.workunit.client.0.smithi077.stdout:6/994: truncate d8/d52/f4d 6659035 0 2024-04-26T04:53:24.822 INFO:tasks.workunit.client.0.smithi077.stdout:0/887: read d0/d47/d93/f9a [1934829,30825] 0 2024-04-26T04:53:24.824 INFO:tasks.workunit.client.0.smithi077.stdout:0/888: chown d0/d19/d1e/d2f/d52/f76 61 1 2024-04-26T04:53:24.825 INFO:tasks.workunit.client.0.smithi077.stdout:6/995: dread - d8/d2a/df2/f11a zero size 2024-04-26T04:53:24.826 INFO:tasks.workunit.client.0.smithi077.stdout:0/889: mknod d0/d47/d64/dfe/c12e 0 2024-04-26T04:53:24.829 INFO:tasks.workunit.client.0.smithi077.stdout:0/890: mknod d0/d19/d1e/d59/c12f 0 2024-04-26T04:53:24.832 INFO:tasks.workunit.client.0.smithi077.stdout:6/996: dwrite d8/d52/da0/dad/ff3 [0,4194304] 0 2024-04-26T04:53:24.833 INFO:tasks.workunit.client.0.smithi077.stdout:9/857: sync 2024-04-26T04:53:24.833 INFO:tasks.workunit.client.0.smithi077.stdout:3/929: sync 2024-04-26T04:53:24.833 INFO:tasks.workunit.client.0.smithi077.stdout:5/904: sync 2024-04-26T04:53:24.837 INFO:tasks.workunit.client.0.smithi077.stdout:7/978: dread da/d13/d22/d5d/d9f/fdf [0,4194304] 0 2024-04-26T04:53:24.840 INFO:tasks.workunit.client.0.smithi077.stdout:3/930: creat d0/d7/d1f/d4d/dc6/f12a x:0 0 0 2024-04-26T04:53:24.841 INFO:tasks.workunit.client.0.smithi077.stdout:3/931: fsync d0/dae/ded/d2a/f40 0 2024-04-26T04:53:24.841 INFO:tasks.workunit.client.0.smithi077.stdout:0/891: dread d0/d19/d1e/d59/d96/d106/fc3 [0,4194304] 0 2024-04-26T04:53:24.842 INFO:tasks.workunit.client.0.smithi077.stdout:3/932: dread - d0/dae/ded/d2a/d85/f10c zero size 2024-04-26T04:53:24.845 INFO:tasks.workunit.client.0.smithi077.stdout:5/905: mknod d1/d42/c127 0 2024-04-26T04:53:24.847 INFO:tasks.workunit.client.0.smithi077.stdout:7/979: mkdir da/d13/d22/d5d/dbd/df2/d14d 0 2024-04-26T04:53:24.849 INFO:tasks.workunit.client.0.smithi077.stdout:7/980: write da/d13/f19 [1521026,54730] 0 2024-04-26T04:53:24.852 INFO:tasks.workunit.client.0.smithi077.stdout:6/997: dwrite d8/d8f/f9f [0,4194304] 0 2024-04-26T04:53:24.859 INFO:tasks.workunit.client.0.smithi077.stdout:3/933: symlink d0/d7/d1f/d81/dc5/l12b 0 2024-04-26T04:53:24.861 INFO:tasks.workunit.client.0.smithi077.stdout:9/858: rename d9/dd/c84 to d9/dd/d1b/d2b/ddb/c11d 0 2024-04-26T04:53:24.861 INFO:tasks.workunit.client.0.smithi077.stdout:9/859: chown d9/d35/ca6 52 1 2024-04-26T04:53:24.864 INFO:tasks.workunit.client.0.smithi077.stdout:7/981: symlink da/d13/d22/d5d/d8d/l14e 0 2024-04-26T04:53:24.865 INFO:tasks.workunit.client.0.smithi077.stdout:7/982: fdatasync da/d6b/d63/d64/fd9 0 2024-04-26T04:53:24.865 INFO:tasks.workunit.client.0.smithi077.stdout:6/998: mkdir d8/d1c/da4/de7/d157 0 2024-04-26T04:53:24.866 INFO:tasks.workunit.client.0.smithi077.stdout:0/892: mkdir d0/d19/d8d/dee/d129/d130 0 2024-04-26T04:53:24.868 INFO:tasks.workunit.client.0.smithi077.stdout:9/860: unlink d9/cb9 0 2024-04-26T04:53:24.869 INFO:tasks.workunit.client.0.smithi077.stdout:9/861: chown d9/d45 653512075 1 2024-04-26T04:53:24.870 INFO:tasks.workunit.client.0.smithi077.stdout:9/862: write d9/d3a/fea [847175,60188] 0 2024-04-26T04:53:24.872 INFO:tasks.workunit.client.0.smithi077.stdout:7/983: symlink da/d13/d22/d5d/d9f/da7/de0/l14f 0 2024-04-26T04:53:24.875 INFO:tasks.workunit.client.0.smithi077.stdout:6/999: mknod d8/d52/d53/da7/dfa/c158 0 2024-04-26T04:53:24.878 INFO:tasks.workunit.client.0.smithi077.stdout:7/984: symlink da/d6b/l150 0 2024-04-26T04:53:24.879 INFO:tasks.workunit.client.0.smithi077.stdout:3/934: creat d0/d7/d1f/d81/dc5/dd1/d127/f12c x:0 0 0 2024-04-26T04:53:24.880 INFO:tasks.workunit.client.0.smithi077.stdout:3/935: chown d0/dae/ded/d29/faf 772611 1 2024-04-26T04:53:24.887 INFO:tasks.workunit.client.0.smithi077.stdout:9/863: creat d9/dd/d1b/d2b/d32/d36/f11e x:0 0 0 2024-04-26T04:53:24.887 INFO:tasks.workunit.client.0.smithi077.stdout:0/893: dread d0/d19/f35 [0,4194304] 0 2024-04-26T04:53:24.887 INFO:tasks.workunit.client.0.smithi077.stdout:0/894: write d0/d2c/d4e/f66 [1340002,38111] 0 2024-04-26T04:53:24.887 INFO:tasks.workunit.client.0.smithi077.stdout:3/936: unlink d0/d7/d1f/d4d/dbe/ff0 0 2024-04-26T04:53:24.888 INFO:tasks.workunit.client.0.smithi077.stdout:3/937: dread d0/d7/d1f/d4d/f61 [0,4194304] 0 2024-04-26T04:53:24.890 INFO:tasks.workunit.client.0.smithi077.stdout:9/864: rename d9/dd/d1b/d2b/d32/fce to d9/d35/d99/d101/f11f 0 2024-04-26T04:53:24.891 INFO:tasks.workunit.client.0.smithi077.stdout:9/865: chown d9/f8d 21466 1 2024-04-26T04:53:24.891 INFO:tasks.workunit.client.0.smithi077.stdout:0/895: symlink d0/d47/d64/dfe/l131 0 2024-04-26T04:53:24.892 INFO:tasks.workunit.client.0.smithi077.stdout:9/866: stat d9/d3a/d79/c74 0 2024-04-26T04:53:24.892 INFO:tasks.workunit.client.0.smithi077.stdout:9/867: chown d9/d35/lb4 3 1 2024-04-26T04:53:24.893 INFO:tasks.workunit.client.0.smithi077.stdout:7/985: write da/d15/d1a/f10f [115819,40647] 0 2024-04-26T04:53:24.902 INFO:tasks.workunit.client.0.smithi077.stdout:9/868: getdents d9/d45/dcd 0 2024-04-26T04:53:24.903 INFO:tasks.workunit.client.0.smithi077.stdout:9/869: write d9/d45/f7e [4938832,35669] 0 2024-04-26T04:53:24.904 INFO:tasks.workunit.client.0.smithi077.stdout:0/896: write d0/d47/d64/f80 [674480,33846] 0 2024-04-26T04:53:24.905 INFO:tasks.workunit.client.0.smithi077.stdout:3/938: creat d0/d7/d1f/f12d x:0 0 0 2024-04-26T04:53:24.907 INFO:tasks.workunit.client.0.smithi077.stdout:3/939: dread - d0/dae/ded/d29/faf zero size 2024-04-26T04:53:24.910 INFO:tasks.workunit.client.0.smithi077.stdout:9/870: rename d9/d45/db1 to d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/d11c/d120 0 2024-04-26T04:53:24.912 INFO:tasks.workunit.client.0.smithi077.stdout:7/986: dwrite da/f68 [4194304,4194304] 0 2024-04-26T04:53:24.912 INFO:tasks.workunit.client.0.smithi077.stdout:9/871: write d9/d3a/d79/d95/dad/de7/f103 [199415,92633] 0 2024-04-26T04:53:24.914 INFO:tasks.workunit.client.0.smithi077.stdout:7/987: stat da/d6b/d63/d64/db0/l147 0 2024-04-26T04:53:24.914 INFO:tasks.workunit.client.0.smithi077.stdout:7/988: chown da/d13/d22/d5d 165 1 2024-04-26T04:53:24.915 INFO:tasks.workunit.client.0.smithi077.stdout:7/989: chown da/d13/d22/d5d/d9f/ddd/d137 928463114 1 2024-04-26T04:53:24.918 INFO:tasks.workunit.client.0.smithi077.stdout:3/940: mknod d0/d7/d1f/d4d/dc6/c12e 0 2024-04-26T04:53:24.918 INFO:tasks.workunit.client.0.smithi077.stdout:0/897: write d0/d47/d64/d97/dba/dc4/f73 [956460,92284] 0 2024-04-26T04:53:24.919 INFO:tasks.workunit.client.0.smithi077.stdout:3/941: write d0/f38 [819893,45671] 0 2024-04-26T04:53:24.922 INFO:tasks.workunit.client.0.smithi077.stdout:0/898: symlink d0/d47/d68/l132 0 2024-04-26T04:53:24.922 INFO:tasks.workunit.client.0.smithi077.stdout:0/899: readlink d0/d47/d68/d9f/l114 0 2024-04-26T04:53:24.925 INFO:tasks.workunit.client.0.smithi077.stdout:3/942: fsync d0/d7/d1f/d4d/f63 0 2024-04-26T04:53:24.928 INFO:tasks.workunit.client.0.smithi077.stdout:0/900: mkdir d0/d2c/d4e/d133 0 2024-04-26T04:53:24.929 INFO:tasks.workunit.client.0.smithi077.stdout:7/990: creat da/d13/d22/d5d/dbd/f151 x:0 0 0 2024-04-26T04:53:24.930 INFO:tasks.workunit.client.0.smithi077.stdout:9/872: getdents d9/dd/d1b/d2b 0 2024-04-26T04:53:24.931 INFO:tasks.workunit.client.0.smithi077.stdout:3/943: dread - d0/d7/d1f/d4d/dbe/fdc zero size 2024-04-26T04:53:24.936 INFO:tasks.workunit.client.0.smithi077.stdout:9/873: rmdir d9/dd 39 2024-04-26T04:53:24.939 INFO:tasks.workunit.client.0.smithi077.stdout:3/944: dread - d0/dae/db6/fd0 zero size 2024-04-26T04:53:24.941 INFO:tasks.workunit.client.0.smithi077.stdout:0/901: write d0/d47/d64/d97/dba/dc4/fdf [211461,100255] 0 2024-04-26T04:53:24.941 INFO:tasks.workunit.client.0.smithi077.stdout:7/991: write da/d15/d1a/f20 [2208483,95028] 0 2024-04-26T04:53:24.943 INFO:tasks.workunit.client.0.smithi077.stdout:9/874: creat d9/d35/d99/d101/f121 x:0 0 0 2024-04-26T04:53:24.948 INFO:tasks.workunit.client.0.smithi077.stdout:0/902: write d0/d2c/d4e/f4c [1866202,14167] 0 2024-04-26T04:53:24.949 INFO:tasks.workunit.client.0.smithi077.stdout:9/875: mkdir d9/d43/d112/d122 0 2024-04-26T04:53:24.951 INFO:tasks.workunit.client.0.smithi077.stdout:7/992: creat da/d13/d22/d5d/dbd/f152 x:0 0 0 2024-04-26T04:53:24.953 INFO:tasks.workunit.client.0.smithi077.stdout:0/903: truncate d0/d19/d1e/d2f/d52/f10e 798013 0 2024-04-26T04:53:24.954 INFO:tasks.workunit.client.0.smithi077.stdout:0/904: stat d0/d47/d64/d97/df9/l112 0 2024-04-26T04:53:24.955 INFO:tasks.workunit.client.0.smithi077.stdout:0/905: fdatasync d0/d19/d1e/d59/f11a 0 2024-04-26T04:53:24.959 INFO:tasks.workunit.client.0.smithi077.stdout:0/906: dread d0/d19/f11c [0,4194304] 0 2024-04-26T04:53:24.959 INFO:tasks.workunit.client.0.smithi077.stdout:7/993: dread da/d13/d22/d5d/d9f/ddd/f120 [0,4194304] 0 2024-04-26T04:53:24.960 INFO:tasks.workunit.client.0.smithi077.stdout:0/907: write d0/d19/d69/f84 [1227098,792] 0 2024-04-26T04:53:24.964 INFO:tasks.workunit.client.0.smithi077.stdout:9/876: truncate d9/d35/d99/fa3 338077 0 2024-04-26T04:53:24.968 INFO:tasks.workunit.client.0.smithi077.stdout:9/877: symlink d9/d7d/df9/l123 0 2024-04-26T04:53:24.969 INFO:tasks.workunit.client.0.smithi077.stdout:7/994: dwrite da/f59 [0,4194304] 0 2024-04-26T04:53:24.974 INFO:tasks.workunit.client.0.smithi077.stdout:9/878: creat d9/d45/dc4/df3/f124 x:0 0 0 2024-04-26T04:53:24.976 INFO:tasks.workunit.client.0.smithi077.stdout:7/995: fdatasync da/d15/d1a/d39/d96/fbf 0 2024-04-26T04:53:24.978 INFO:tasks.workunit.client.0.smithi077.stdout:9/879: dread - d9/dd/d1b/d2b/d32/d36/d7f/dae/fc9 zero size 2024-04-26T04:53:24.982 INFO:tasks.workunit.client.0.smithi077.stdout:0/908: dread d0/fe [0,4194304] 0 2024-04-26T04:53:24.984 INFO:tasks.workunit.client.0.smithi077.stdout:7/996: symlink da/d13/d22/d7d/dfd/d13a/l153 0 2024-04-26T04:53:24.992 INFO:tasks.workunit.client.0.smithi077.stdout:0/909: write d0/d19/d1e/d81/fff [508422,39219] 0 2024-04-26T04:53:24.992 INFO:tasks.workunit.client.0.smithi077.stdout:7/997: symlink da/d13/d22/l154 0 2024-04-26T04:53:24.993 INFO:tasks.workunit.client.0.smithi077.stdout:9/880: write d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/fe1 [1098385,24465] 0 2024-04-26T04:53:24.994 INFO:tasks.workunit.client.0.smithi077.stdout:9/881: stat d9/dd/d1b/d2b/d32/d36/fa7 0 2024-04-26T04:53:24.996 INFO:tasks.workunit.client.0.smithi077.stdout:7/998: symlink da/d13/d22/d7d/d80/l155 0 2024-04-26T04:53:24.996 INFO:tasks.workunit.client.0.smithi077.stdout:9/882: mknod d9/d45/dcd/c125 0 2024-04-26T04:53:24.997 INFO:tasks.workunit.client.0.smithi077.stdout:9/883: dread - d9/dd/d3d/f59 zero size 2024-04-26T04:53:24.999 INFO:tasks.workunit.client.0.smithi077.stdout:9/884: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/d126 0 2024-04-26T04:53:25.001 INFO:tasks.workunit.client.0.smithi077.stdout:7/999: dread da/d13/f4b [0,4194304] 0 2024-04-26T04:53:25.012 INFO:tasks.workunit.client.0.smithi077.stdout:9/885: dwrite d9/dd/d1b/d2b/f38 [0,4194304] 0 2024-04-26T04:53:25.014 INFO:tasks.workunit.client.0.smithi077.stdout:9/886: stat d9/dd/d1b/d2b/d32/d36/d7f/fa8 0 2024-04-26T04:53:25.016 INFO:tasks.workunit.client.0.smithi077.stdout:9/887: dread - d9/d3a/d79/d95/dad/db2/fe8 zero size 2024-04-26T04:53:25.020 INFO:tasks.workunit.client.0.smithi077.stdout:9/888: rename d9/dd/d1b/d2b/ddb/l93 to d9/d45/l127 0 2024-04-26T04:53:25.022 INFO:tasks.workunit.client.0.smithi077.stdout:9/889: symlink d9/dd/d1b/d4e/l128 0 2024-04-26T04:53:25.036 INFO:tasks.workunit.client.0.smithi077.stdout:9/890: dwrite d9/dd/d3d/f9e [0,4194304] 0 2024-04-26T04:53:25.037 INFO:tasks.workunit.client.0.smithi077.stdout:9/891: read - d9/dd/d3d/f59 zero size 2024-04-26T04:53:25.045 INFO:tasks.workunit.client.0.smithi077.stdout:9/892: creat d9/dd/d1b/d2b/d32/f129 x:0 0 0 2024-04-26T04:53:25.378 INFO:tasks.workunit.client.0.smithi077.stdout:5/906: sync 2024-04-26T04:53:25.387 INFO:tasks.workunit.client.0.smithi077.stdout:5/907: dwrite d1/d42/d62/f89 [0,4194304] 0 2024-04-26T04:53:25.396 INFO:tasks.workunit.client.0.smithi077.stdout:5/908: rmdir d1/d8c/df9 0 2024-04-26T04:53:25.493 INFO:tasks.workunit.client.0.smithi077.stdout:3/945: sync 2024-04-26T04:53:25.493 INFO:tasks.workunit.client.0.smithi077.stdout:9/893: sync 2024-04-26T04:53:25.493 INFO:tasks.workunit.client.0.smithi077.stdout:0/910: sync 2024-04-26T04:53:25.499 INFO:tasks.workunit.client.0.smithi077.stdout:9/894: getdents d9/d7d/dd1 0 2024-04-26T04:53:25.500 INFO:tasks.workunit.client.0.smithi077.stdout:3/946: dwrite d0/dae/ded/d2a/d85/dd3/f122 [0,4194304] 0 2024-04-26T04:53:25.502 INFO:tasks.workunit.client.0.smithi077.stdout:0/911: write d0/d47/d64/d97/ffb [2183709,83074] 0 2024-04-26T04:53:25.506 INFO:tasks.workunit.client.0.smithi077.stdout:9/895: mkdir d9/dd/d12a 0 2024-04-26T04:53:25.509 INFO:tasks.workunit.client.0.smithi077.stdout:0/912: symlink d0/d19/d1e/d2f/da8/l134 0 2024-04-26T04:53:25.512 INFO:tasks.workunit.client.0.smithi077.stdout:0/913: unlink d0/d2c/f111 0 2024-04-26T04:53:25.516 INFO:tasks.workunit.client.0.smithi077.stdout:9/896: getdents d9/d45/dcd 0 2024-04-26T04:53:25.517 INFO:tasks.workunit.client.0.smithi077.stdout:0/914: rename d0/d47/d68/d9f/fc9 to d0/d19/d8d/dee/d129/d130/f135 0 2024-04-26T04:53:25.518 INFO:tasks.workunit.client.0.smithi077.stdout:9/897: dread - d9/fe5 zero size 2024-04-26T04:53:25.519 INFO:tasks.workunit.client.0.smithi077.stdout:9/898: chown d9/dd/d1b/d2b/c4f 0 1 2024-04-26T04:53:25.520 INFO:tasks.workunit.client.0.smithi077.stdout:0/915: symlink d0/d47/d93/dfa/l136 0 2024-04-26T04:53:25.522 INFO:tasks.workunit.client.0.smithi077.stdout:9/899: mkdir d9/dd/d1b/d4e/d104/d12b 0 2024-04-26T04:53:25.539 INFO:tasks.workunit.client.0.smithi077.stdout:0/916: write d0/d19/d1e/d59/fb9 [2493972,20707] 0 2024-04-26T04:53:25.540 INFO:tasks.workunit.client.0.smithi077.stdout:9/900: write d9/dd/d1b/d2b/f83 [1081748,87081] 0 2024-04-26T04:53:25.542 INFO:tasks.workunit.client.0.smithi077.stdout:0/917: mkdir d0/d47/d68/d9f/d137 0 2024-04-26T04:53:25.543 INFO:tasks.workunit.client.0.smithi077.stdout:9/901: creat d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/f12c x:0 0 0 2024-04-26T04:53:25.544 INFO:tasks.workunit.client.0.smithi077.stdout:0/918: truncate d0/d47/d64/d97/dba/dc4/f73 1727948 0 2024-04-26T04:53:25.546 INFO:tasks.workunit.client.0.smithi077.stdout:9/902: truncate d9/d3a/d79/d95/dad/de7/f103 1076681 0 2024-04-26T04:53:25.546 INFO:tasks.workunit.client.0.smithi077.stdout:0/919: truncate d0/d19/d1e/d2f/f3b 4760477 0 2024-04-26T04:53:25.549 INFO:tasks.workunit.client.0.smithi077.stdout:0/920: unlink d0/d19/d1e/d2f/f3b 0 2024-04-26T04:53:25.558 INFO:tasks.workunit.client.0.smithi077.stdout:9/903: write d9/dd/d1b/d2b/d32/d36/d7f/dae/ff5 [754723,53962] 0 2024-04-26T04:53:25.559 INFO:tasks.workunit.client.0.smithi077.stdout:0/921: truncate d0/d19/d1e/f9d 1210062 0 2024-04-26T04:53:25.562 INFO:tasks.workunit.client.0.smithi077.stdout:0/922: fsync d0/d19/d1e/f20 0 2024-04-26T04:53:25.563 INFO:tasks.workunit.client.0.smithi077.stdout:0/923: write d0/d2c/f88 [3578947,19811] 0 2024-04-26T04:53:25.566 INFO:tasks.workunit.client.0.smithi077.stdout:0/924: mkdir d0/d47/d64/d97/df9/d138 0 2024-04-26T04:53:25.576 INFO:tasks.workunit.client.0.smithi077.stdout:9/904: dread d9/dd/d1b/d2b/ddb/f9a [0,4194304] 0 2024-04-26T04:53:25.579 INFO:tasks.workunit.client.0.smithi077.stdout:0/925: write d0/d28/fd9 [680257,109240] 0 2024-04-26T04:53:25.584 INFO:tasks.workunit.client.0.smithi077.stdout:0/926: unlink d0/d47/d93/d94/fcd 0 2024-04-26T04:53:25.586 INFO:tasks.workunit.client.0.smithi077.stdout:9/905: write d9/d35/f69 [4289573,62104] 0 2024-04-26T04:53:25.589 INFO:tasks.workunit.client.0.smithi077.stdout:0/927: symlink d0/d19/d1e/d59/d96/d106/d120/l139 0 2024-04-26T04:53:25.592 INFO:tasks.workunit.client.0.smithi077.stdout:9/906: unlink d9/d7d/dd1/fd7 0 2024-04-26T04:53:25.595 INFO:tasks.workunit.client.0.smithi077.stdout:0/928: creat d0/d19/d1e/d59/d96/d106/d120/f13a x:0 0 0 2024-04-26T04:53:25.598 INFO:tasks.workunit.client.0.smithi077.stdout:9/907: rename d9/d35/c41 to d9/dd/d12a/c12d 0 2024-04-26T04:53:25.609 INFO:tasks.workunit.client.0.smithi077.stdout:0/929: rename d0/d19/f2d to d0/d19/d1e/d2f/da8/d101/f13b 0 2024-04-26T04:53:25.615 INFO:tasks.workunit.client.0.smithi077.stdout:9/908: dwrite d9/d35/d99/d101/f121 [0,4194304] 0 2024-04-26T04:53:25.623 INFO:tasks.workunit.client.0.smithi077.stdout:9/909: dwrite d9/dd/d1b/d2b/d32/d36/d7f/f11b [0,4194304] 0 2024-04-26T04:53:25.624 INFO:tasks.workunit.client.0.smithi077.stdout:0/930: link d0/d28/d40/f85 d0/d47/d64/f13c 0 2024-04-26T04:53:25.625 INFO:tasks.workunit.client.0.smithi077.stdout:9/910: write d9/dd/d3d/f5a [1485974,60057] 0 2024-04-26T04:53:25.625 INFO:tasks.workunit.client.0.smithi077.stdout:0/931: chown d0/d19/d8d/cd1 20 1 2024-04-26T04:53:25.636 INFO:tasks.workunit.client.0.smithi077.stdout:0/932: symlink d0/d47/d64/d97/dba/dc4/l13d 0 2024-04-26T04:53:25.636 INFO:tasks.workunit.client.0.smithi077.stdout:0/933: readlink d0/d19/d1e/l3d 0 2024-04-26T04:53:25.638 INFO:tasks.workunit.client.0.smithi077.stdout:9/911: truncate d9/d3a/d79/f68 315749 0 2024-04-26T04:53:25.644 INFO:tasks.workunit.client.0.smithi077.stdout:0/934: creat d0/d47/d93/d94/f13e x:0 0 0 2024-04-26T04:53:25.647 INFO:tasks.workunit.client.0.smithi077.stdout:0/935: fsync d0/d19/d69/fe8 0 2024-04-26T04:53:25.654 INFO:tasks.workunit.client.0.smithi077.stdout:0/936: creat d0/d19/d1e/d59/d96/f13f x:0 0 0 2024-04-26T04:53:25.654 INFO:tasks.workunit.client.0.smithi077.stdout:9/912: getdents d9/d3a/d79/d95/dad/de7 0 2024-04-26T04:53:25.655 INFO:tasks.workunit.client.0.smithi077.stdout:9/913: chown d9/dd/d1b/d2b/d32/d36/d7f/dae/dfa 89 1 2024-04-26T04:53:25.656 INFO:tasks.workunit.client.0.smithi077.stdout:0/937: mknod d0/d47/d64/d97/dba/dc4/c140 0 2024-04-26T04:53:25.657 INFO:tasks.workunit.client.0.smithi077.stdout:9/914: creat d9/dd/d1b/d4e/f12e x:0 0 0 2024-04-26T04:53:25.659 INFO:tasks.workunit.client.0.smithi077.stdout:9/915: write d9/d3a/d79/f33 [6371208,34137] 0 2024-04-26T04:53:25.659 INFO:tasks.workunit.client.0.smithi077.stdout:5/909: sync 2024-04-26T04:53:25.663 INFO:tasks.workunit.client.0.smithi077.stdout:5/910: dread d1/d42/d62/d121/d28/ffc [0,4194304] 0 2024-04-26T04:53:25.664 INFO:tasks.workunit.client.0.smithi077.stdout:0/938: rename d0/d19/d1e/d82/c123 to d0/d47/d64/d97/df9/d138/c141 0 2024-04-26T04:53:25.665 INFO:tasks.workunit.client.0.smithi077.stdout:0/939: fsync d0/d19/d1e/d59/fb9 0 2024-04-26T04:53:25.673 INFO:tasks.workunit.client.0.smithi077.stdout:5/911: write d1/d42/d62/d121/f19 [4074126,71898] 0 2024-04-26T04:53:25.675 INFO:tasks.workunit.client.0.smithi077.stdout:0/940: rename d0/d19/d1e/d2f/da8/fab to d0/d2c/d4e/d133/f142 0 2024-04-26T04:53:25.675 INFO:tasks.workunit.client.0.smithi077.stdout:9/916: dwrite d9/d3a/d79/f68 [0,4194304] 0 2024-04-26T04:53:25.677 INFO:tasks.workunit.client.0.smithi077.stdout:0/941: chown d0/d47/d93/l10f 3590308 1 2024-04-26T04:53:25.682 INFO:tasks.workunit.client.0.smithi077.stdout:9/917: dread d9/dd/d1b/d2b/ddb/f9a [0,4194304] 0 2024-04-26T04:53:25.683 INFO:tasks.workunit.client.0.smithi077.stdout:0/942: dwrite d0/d19/d1e/d2f/f44 [0,4194304] 0 2024-04-26T04:53:25.686 INFO:tasks.workunit.client.0.smithi077.stdout:5/912: write d1/d30/d35/f5a [3935773,60951] 0 2024-04-26T04:53:25.689 INFO:tasks.workunit.client.0.smithi077.stdout:3/947: sync 2024-04-26T04:53:25.692 INFO:tasks.workunit.client.0.smithi077.stdout:3/948: mkdir d0/d7/d1f/d4d/dbe/d12f 0 2024-04-26T04:53:25.694 INFO:tasks.workunit.client.0.smithi077.stdout:3/949: write d0/dae/ded/d29/db8/f100 [803638,119999] 0 2024-04-26T04:53:25.696 INFO:tasks.workunit.client.0.smithi077.stdout:0/943: write d0/d47/d64/d97/dba/dc4/f11b [894055,75913] 0 2024-04-26T04:53:25.698 INFO:tasks.workunit.client.0.smithi077.stdout:3/950: creat d0/d7/d1f/d4d/dc6/f130 x:0 0 0 2024-04-26T04:53:25.701 INFO:tasks.workunit.client.0.smithi077.stdout:0/944: fsync d0/d47/d93/d94/f9b 0 2024-04-26T04:53:25.706 INFO:tasks.workunit.client.0.smithi077.stdout:0/945: rmdir d0/d19/d1e/d59/d96 39 2024-04-26T04:53:25.707 INFO:tasks.workunit.client.0.smithi077.stdout:9/918: dread f2 [0,4194304] 0 2024-04-26T04:53:25.710 INFO:tasks.workunit.client.0.smithi077.stdout:9/919: mknod d9/dd/d3d/d7c/dc8/c12f 0 2024-04-26T04:53:25.720 INFO:tasks.workunit.client.0.smithi077.stdout:9/920: getdents d9/dd 0 2024-04-26T04:53:25.722 INFO:tasks.workunit.client.0.smithi077.stdout:3/951: dwrite d0/dae/ded/d29/f4b [0,4194304] 0 2024-04-26T04:53:25.726 INFO:tasks.workunit.client.0.smithi077.stdout:9/921: mkdir d9/d43/d10c/d130 0 2024-04-26T04:53:25.727 INFO:tasks.workunit.client.0.smithi077.stdout:3/952: rename d0/dae/ded/f21 to d0/dae/ded/d29/db8/dbf/f131 0 2024-04-26T04:53:25.731 INFO:tasks.workunit.client.0.smithi077.stdout:0/946: dwrite d0/d47/d68/fa5 [0,4194304] 0 2024-04-26T04:53:25.733 INFO:tasks.workunit.client.0.smithi077.stdout:3/953: creat d0/d7/d1f/d4d/dbe/d11e/f132 x:0 0 0 2024-04-26T04:53:25.735 INFO:tasks.workunit.client.0.smithi077.stdout:3/954: read d0/d7/f34 [4173843,73888] 0 2024-04-26T04:53:25.735 INFO:tasks.workunit.client.0.smithi077.stdout:0/947: mknod d0/d28/d40/c143 0 2024-04-26T04:53:25.735 INFO:tasks.workunit.client.0.smithi077.stdout:9/922: getdents d9/dd/df2 0 2024-04-26T04:53:25.736 INFO:tasks.workunit.client.0.smithi077.stdout:9/923: chown d9/d3a/d79/d95/c119 50 1 2024-04-26T04:53:25.740 INFO:tasks.workunit.client.0.smithi077.stdout:9/924: rmdir d9/dd/d1b 39 2024-04-26T04:53:25.742 INFO:tasks.workunit.client.0.smithi077.stdout:0/948: truncate d0/d47/f127 2800161 0 2024-04-26T04:53:25.744 INFO:tasks.workunit.client.0.smithi077.stdout:9/925: readlink d9/dd/d1b/d2b/d32/l98 0 2024-04-26T04:53:25.746 INFO:tasks.workunit.client.0.smithi077.stdout:9/926: getdents d9/d45/dc4/d116 0 2024-04-26T04:53:25.746 INFO:tasks.workunit.client.0.smithi077.stdout:9/927: dread - d9/dd/df2/f118 zero size 2024-04-26T04:53:25.748 INFO:tasks.workunit.client.0.smithi077.stdout:0/949: dread d0/d19/d69/fe8 [0,4194304] 0 2024-04-26T04:53:25.749 INFO:tasks.workunit.client.0.smithi077.stdout:9/928: getdents d9/d7d/df9 0 2024-04-26T04:53:25.751 INFO:tasks.workunit.client.0.smithi077.stdout:0/950: creat d0/d47/d64/f144 x:0 0 0 2024-04-26T04:53:25.752 INFO:tasks.workunit.client.0.smithi077.stdout:9/929: mknod d9/dd/d3d/d7c/dc8/c131 0 2024-04-26T04:53:25.754 INFO:tasks.workunit.client.0.smithi077.stdout:0/951: unlink d0/d19/f98 0 2024-04-26T04:53:25.755 INFO:tasks.workunit.client.0.smithi077.stdout:9/930: rename d9/f62 to d9/dd/d1b/d4e/d104/d12b/f132 0 2024-04-26T04:53:25.756 INFO:tasks.workunit.client.0.smithi077.stdout:9/931: dread - d9/dd/d3d/d7c/dc8/fcc zero size 2024-04-26T04:53:25.757 INFO:tasks.workunit.client.0.smithi077.stdout:9/932: readlink d9/l1d 0 2024-04-26T04:53:25.761 INFO:tasks.workunit.client.0.smithi077.stdout:3/955: write d0/dae/ded/d29/f6a [1132157,19601] 0 2024-04-26T04:53:25.764 INFO:tasks.workunit.client.0.smithi077.stdout:9/933: dwrite d9/dd/d1b/d2b/d32/f129 [0,4194304] 0 2024-04-26T04:53:25.768 INFO:tasks.workunit.client.0.smithi077.stdout:3/956: rmdir d0/dae/ded/d29/db8 39 2024-04-26T04:53:25.772 INFO:tasks.workunit.client.0.smithi077.stdout:0/952: write d0/d47/d10a/fd6 [723715,74739] 0 2024-04-26T04:53:25.776 INFO:tasks.workunit.client.0.smithi077.stdout:0/953: creat d0/d47/d93/d94/f145 x:0 0 0 2024-04-26T04:53:25.779 INFO:tasks.workunit.client.0.smithi077.stdout:9/934: link d9/dd/df2/d107/l10b d9/dd/d1b/l133 0 2024-04-26T04:53:25.779 INFO:tasks.workunit.client.0.smithi077.stdout:9/935: dread - d9/d7d/f11a zero size 2024-04-26T04:53:25.783 INFO:tasks.workunit.client.0.smithi077.stdout:9/936: symlink d9/dd/d1b/d4e/d104/d12b/l134 0 2024-04-26T04:53:25.785 INFO:tasks.workunit.client.0.smithi077.stdout:3/957: dread d0/d7/f34 [0,4194304] 0 2024-04-26T04:53:25.790 INFO:tasks.workunit.client.0.smithi077.stdout:0/954: write d0/d2c/d4e/f3e [4534948,130241] 0 2024-04-26T04:53:25.794 INFO:tasks.workunit.client.0.smithi077.stdout:9/937: write d9/dd/d1b/d2b/d32/d36/f6d [627403,73973] 0 2024-04-26T04:53:25.797 INFO:tasks.workunit.client.0.smithi077.stdout:3/958: getdents d0/d7/dce 0 2024-04-26T04:53:25.798 INFO:tasks.workunit.client.0.smithi077.stdout:0/955: symlink d0/d19/d1e/d59/l146 0 2024-04-26T04:53:25.799 INFO:tasks.workunit.client.0.smithi077.stdout:9/938: mkdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/d135 0 2024-04-26T04:53:25.799 INFO:tasks.workunit.client.0.smithi077.stdout:9/939: write d9/f82 [1607475,1878] 0 2024-04-26T04:53:25.801 INFO:tasks.workunit.client.0.smithi077.stdout:3/959: mknod d0/d7/dce/c133 0 2024-04-26T04:53:25.804 INFO:tasks.workunit.client.0.smithi077.stdout:9/940: dread d9/dd/d1b/fab [0,4194304] 0 2024-04-26T04:53:25.805 INFO:tasks.workunit.client.0.smithi077.stdout:0/956: symlink d0/d19/d1e/d81/l147 0 2024-04-26T04:53:25.806 INFO:tasks.workunit.client.0.smithi077.stdout:3/960: mkdir d0/dae/ded/d2a/d134 0 2024-04-26T04:53:25.809 INFO:tasks.workunit.client.0.smithi077.stdout:3/961: write d0/d7/fad [456703,6397] 0 2024-04-26T04:53:25.818 INFO:tasks.workunit.client.0.smithi077.stdout:9/941: truncate d9/dd/d3d/f5a 1820234 0 2024-04-26T04:53:25.822 INFO:tasks.workunit.client.0.smithi077.stdout:0/957: dwrite d0/d47/f8c [0,4194304] 0 2024-04-26T04:53:25.824 INFO:tasks.workunit.client.0.smithi077.stdout:0/958: fdatasync d0/d47/d64/d97/dba/dc4/f11b 0 2024-04-26T04:53:25.825 INFO:tasks.workunit.client.0.smithi077.stdout:9/942: rename d9/dd/def/lf1 to d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/d135/l136 0 2024-04-26T04:53:25.825 INFO:tasks.workunit.client.0.smithi077.stdout:3/962: dwrite d0/d7/d1f/d4d/f5f [0,4194304] 0 2024-04-26T04:53:25.833 INFO:tasks.workunit.client.0.smithi077.stdout:3/963: dread d0/f94 [0,4194304] 0 2024-04-26T04:53:25.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/959: creat d0/d19/d1e/d2f/f148 x:0 0 0 2024-04-26T04:53:25.837 INFO:tasks.workunit.client.0.smithi077.stdout:9/943: mkdir d9/d35/d137 0 2024-04-26T04:53:25.842 INFO:tasks.workunit.client.0.smithi077.stdout:3/964: dread d0/d7/d1f/fcf [0,4194304] 0 2024-04-26T04:53:25.843 INFO:tasks.workunit.client.0.smithi077.stdout:3/965: fsync d0/d7/d1f/d4d/f113 0 2024-04-26T04:53:25.843 INFO:tasks.workunit.client.0.smithi077.stdout:0/960: dread - d0/d19/d1e/d59/d96/d106/d120/d124/f12b zero size 2024-04-26T04:53:25.847 INFO:tasks.workunit.client.0.smithi077.stdout:3/966: dread d0/dae/ded/d29/f71 [0,4194304] 0 2024-04-26T04:53:25.847 INFO:tasks.workunit.client.0.smithi077.stdout:0/961: dread d0/d19/d1e/d81/fff [0,4194304] 0 2024-04-26T04:53:25.848 INFO:tasks.workunit.client.0.smithi077.stdout:3/967: write d0/d7/d1f/d4d/dc6/f101 [6275,99296] 0 2024-04-26T04:53:25.851 INFO:tasks.workunit.client.0.smithi077.stdout:3/968: truncate d0/d7/d1f/f51 897696 0 2024-04-26T04:53:25.852 INFO:tasks.workunit.client.0.smithi077.stdout:3/969: dread - d0/d7/dce/ff2 zero size 2024-04-26T04:53:25.856 INFO:tasks.workunit.client.0.smithi077.stdout:9/944: dwrite d9/d3a/f6e [0,4194304] 0 2024-04-26T04:53:25.859 INFO:tasks.workunit.client.0.smithi077.stdout:9/945: truncate d9/dd/d1b/d2b/d32/f129 4455612 0 2024-04-26T04:53:25.860 INFO:tasks.workunit.client.0.smithi077.stdout:9/946: rmdir d9/dd/d1b/d4e 39 2024-04-26T04:53:25.862 INFO:tasks.workunit.client.0.smithi077.stdout:9/947: write d9/d3a/d79/f68 [1791209,115398] 0 2024-04-26T04:53:25.864 INFO:tasks.workunit.client.0.smithi077.stdout:9/948: rmdir d9/d43/d10c 39 2024-04-26T04:53:25.867 INFO:tasks.workunit.client.0.smithi077.stdout:9/949: getdents d9/d43/d112/d122 0 2024-04-26T04:53:25.868 INFO:tasks.workunit.client.0.smithi077.stdout:9/950: stat d9/dd/d3d/d7c/c100 0 2024-04-26T04:53:25.873 INFO:tasks.workunit.client.0.smithi077.stdout:3/970: dread d0/d7/d1f/d81/dc5/f112 [0,4194304] 0 2024-04-26T04:53:25.877 INFO:tasks.workunit.client.0.smithi077.stdout:3/971: creat d0/d7/d1f/d81/dc5/f135 x:0 0 0 2024-04-26T04:53:25.880 INFO:tasks.workunit.client.0.smithi077.stdout:3/972: truncate d0/dae/ded/d29/db8/dec/ffd 38677 0 2024-04-26T04:53:25.881 INFO:tasks.workunit.client.0.smithi077.stdout:3/973: write d0/d7/d1f/d81/fda [4780678,36586] 0 2024-04-26T04:53:25.892 INFO:tasks.workunit.client.0.smithi077.stdout:3/974: dwrite d0/d7/d1f/d4d/f99 [0,4194304] 0 2024-04-26T04:53:25.896 INFO:tasks.workunit.client.0.smithi077.stdout:3/975: symlink d0/d7/d9a/l136 0 2024-04-26T04:53:25.897 INFO:tasks.workunit.client.0.smithi077.stdout:3/976: chown d0/dae/ded/d29/f6a 10504313 1 2024-04-26T04:53:25.906 INFO:tasks.workunit.client.0.smithi077.stdout:3/977: write d0/dae/fdf [742890,69776] 0 2024-04-26T04:53:25.909 INFO:tasks.workunit.client.0.smithi077.stdout:3/978: creat d0/d7/d1f/f137 x:0 0 0 2024-04-26T04:53:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:25 smithi077 ceph-mon[26311]: pgmap v32: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 55 MiB/s rd, 115 MiB/s wr, 360 op/s 2024-04-26T04:53:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:25 smithi078 ceph-mon[33746]: pgmap v32: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 55 MiB/s rd, 115 MiB/s wr, 360 op/s 2024-04-26T04:53:26.131 INFO:tasks.workunit.client.0.smithi077.stdout:5/913: sync 2024-04-26T04:53:26.195 INFO:tasks.workunit.client.0.smithi077.stdout:5/914: write d1/d8/fc [916983,47804] 0 2024-04-26T04:53:26.203 INFO:tasks.workunit.client.0.smithi077.stdout:5/915: dwrite d1/d42/d62/d121/d28/d38/fb5 [0,4194304] 0 2024-04-26T04:53:26.204 INFO:tasks.workunit.client.0.smithi077.stdout:5/916: chown d1/d30/d33/d66/caa 201606 1 2024-04-26T04:53:26.205 INFO:tasks.workunit.client.0.smithi077.stdout:5/917: chown d1/d42/da4/d25/c75 12346787 1 2024-04-26T04:53:26.207 INFO:tasks.workunit.client.0.smithi077.stdout:5/918: rename d1/d42/da4/d25/fd7 to d1/d8/d72/f128 0 2024-04-26T04:53:26.211 INFO:tasks.workunit.client.0.smithi077.stdout:5/919: link d1/d8/c6a d1/d42/d62/dc8/c129 0 2024-04-26T04:53:26.213 INFO:tasks.workunit.client.0.smithi077.stdout:5/920: creat d1/d42/da4/f12a x:0 0 0 2024-04-26T04:53:26.234 INFO:tasks.workunit.client.0.smithi077.stdout:5/921: dread d1/d30/f53 [0,4194304] 0 2024-04-26T04:53:26.244 INFO:tasks.workunit.client.0.smithi077.stdout:3/979: sync 2024-04-26T04:53:26.244 INFO:tasks.workunit.client.0.smithi077.stdout:9/951: sync 2024-04-26T04:53:26.244 INFO:tasks.workunit.client.0.smithi077.stdout:0/962: sync 2024-04-26T04:53:26.246 INFO:tasks.workunit.client.0.smithi077.stdout:5/922: dread d1/d42/d62/dc8/fdc [0,4194304] 0 2024-04-26T04:53:26.248 INFO:tasks.workunit.client.0.smithi077.stdout:3/980: rename d0/dae/ded/d29/fc2 to d0/dae/db6/f138 0 2024-04-26T04:53:26.249 INFO:tasks.workunit.client.0.smithi077.stdout:9/952: dread d9/dd/d3d/d7c/dc8/fda [0,4194304] 0 2024-04-26T04:53:26.249 INFO:tasks.workunit.client.0.smithi077.stdout:5/923: creat d1/d42/d62/d6b/de8/f12b x:0 0 0 2024-04-26T04:53:26.250 INFO:tasks.workunit.client.0.smithi077.stdout:3/981: fdatasync d0/d7/d1f/d4d/dc6/d11f/f125 0 2024-04-26T04:53:26.252 INFO:tasks.workunit.client.0.smithi077.stdout:0/963: rename d0/d19/d8d/dee/d129/d130 to d0/d47/d64/d97/df9/d138/d149 0 2024-04-26T04:53:26.254 INFO:tasks.workunit.client.0.smithi077.stdout:3/982: truncate d0/dae/ded/d29/faf 902323 0 2024-04-26T04:53:26.255 INFO:tasks.workunit.client.0.smithi077.stdout:3/983: fsync d0/d7/d9a/db3/f22 0 2024-04-26T04:53:26.257 INFO:tasks.workunit.client.0.smithi077.stdout:0/964: mknod d0/d47/d64/d97/df9/c14a 0 2024-04-26T04:53:26.258 INFO:tasks.workunit.client.0.smithi077.stdout:0/965: dread - d0/d47/d93/d94/f13e zero size 2024-04-26T04:53:26.258 INFO:tasks.workunit.client.0.smithi077.stdout:5/924: rename d1/d8d/f11a to d1/d42/d62/d6b/deb/f12c 0 2024-04-26T04:53:26.263 INFO:tasks.workunit.client.0.smithi077.stdout:5/925: rmdir d1/d42/d62/d121/d87 39 2024-04-26T04:53:26.265 INFO:tasks.workunit.client.0.smithi077.stdout:9/953: dread d9/dd/d1b/d2b/d32/f39 [0,4194304] 0 2024-04-26T04:53:26.266 INFO:tasks.workunit.client.0.smithi077.stdout:0/966: truncate d0/d47/f22 877741 0 2024-04-26T04:53:26.268 INFO:tasks.workunit.client.0.smithi077.stdout:5/926: creat d1/d42/d62/dc8/f12d x:0 0 0 2024-04-26T04:53:26.274 INFO:tasks.workunit.client.0.smithi077.stdout:0/967: mkdir d0/d47/d64/d97/dba/dc4/d14b 0 2024-04-26T04:53:26.279 INFO:tasks.workunit.client.0.smithi077.stdout:9/954: creat d9/dd/dec/d10f/f138 x:0 0 0 2024-04-26T04:53:26.279 INFO:tasks.workunit.client.0.smithi077.stdout:9/955: chown d9/dd/l24 861014 1 2024-04-26T04:53:26.283 INFO:tasks.workunit.client.0.smithi077.stdout:0/968: readlink d0/l57 0 2024-04-26T04:53:26.284 INFO:tasks.workunit.client.0.smithi077.stdout:9/956: rmdir d9/dd/d1b/d2b/d32/d36/d7f/dae/dd3/dd4 39 2024-04-26T04:53:26.286 INFO:tasks.workunit.client.0.smithi077.stdout:3/984: dwrite d0/d7/d1f/d81/dc5/f107 [0,4194304] 0 2024-04-26T04:53:26.287 INFO:tasks.workunit.client.0.smithi077.stdout:3/985: chown d0/d7/d9a/db7/ddb/f117 2 1 2024-04-26T04:53:26.291 INFO:tasks.workunit.client.0.smithi077.stdout:5/927: rename d1/d8/d72/lfe to d1/d30/l12e 0 2024-04-26T04:53:26.298 INFO:tasks.workunit.client.0.smithi077.stdout:9/957: mknod d9/d35/d99/c139 0 2024-04-26T04:53:26.300 INFO:tasks.workunit.client.0.smithi077.stdout:3/986: rmdir d0/dae/ded/d2a 39 2024-04-26T04:53:26.301 INFO:tasks.workunit.client.0.smithi077.stdout:3/987: chown d0/d7/d9a/db7/ddb/l11b 26356449 1 2024-04-26T04:53:26.306 INFO:tasks.workunit.client.0.smithi077.stdout:5/928: creat d1/d42/d62/d6b/deb/f12f x:0 0 0 2024-04-26T04:53:26.308 INFO:tasks.workunit.client.0.smithi077.stdout:9/958: read d9/dd/d3d/f3e [2954116,95530] 0 2024-04-26T04:53:26.308 INFO:tasks.workunit.client.0.smithi077.stdout:3/988: rmdir d0/d7/d1f/d4d/dbe 39 2024-04-26T04:53:26.312 INFO:tasks.workunit.client.0.smithi077.stdout:3/989: fdatasync d0/dae/db6/fd0 0 2024-04-26T04:53:26.313 INFO:tasks.workunit.client.0.smithi077.stdout:9/959: dread d9/dd/d1b/d2b/d32/f129 [0,4194304] 0 2024-04-26T04:53:26.317 INFO:tasks.workunit.client.0.smithi077.stdout:5/929: link d1/d42/d62/d121/c4d d1/d30/d33/d109/c130 0 2024-04-26T04:53:26.318 INFO:tasks.workunit.client.0.smithi077.stdout:9/960: chown d9/dd/fb3 792 1 2024-04-26T04:53:26.319 INFO:tasks.workunit.client.0.smithi077.stdout:5/930: fsync d1/d42/d62/d121/d28/d38/f8b 0 2024-04-26T04:53:26.319 INFO:tasks.workunit.client.0.smithi077.stdout:9/961: readlink d9/d45/dc4/df3/l113 0 2024-04-26T04:53:26.320 INFO:tasks.workunit.client.0.smithi077.stdout:9/962: chown d9/dd/d1b/d2b/ddb/l5c 439297104 1 2024-04-26T04:53:26.325 INFO:tasks.workunit.client.0.smithi077.stdout:9/963: creat d9/dd/df2/d107/f13a x:0 0 0 2024-04-26T04:53:26.326 INFO:tasks.workunit.client.0.smithi077.stdout:9/964: chown d9/dd/d1b/d2b/d32/f39 8896931 1 2024-04-26T04:53:26.328 INFO:tasks.workunit.client.0.smithi077.stdout:5/931: mkdir d1/d30/d35/de7/d131 0 2024-04-26T04:53:26.332 INFO:tasks.workunit.client.0.smithi077.stdout:9/965: symlink d9/d35/l13b 0 2024-04-26T04:53:26.332 INFO:tasks.workunit.client.0.smithi077.stdout:5/932: read d1/d42/d62/f6f [923622,84147] 0 2024-04-26T04:53:26.334 INFO:tasks.workunit.client.0.smithi077.stdout:9/966: chown d9/d3a/d79/d95/l9d 339250970 1 2024-04-26T04:53:26.335 INFO:tasks.workunit.client.0.smithi077.stdout:5/933: write d1/d42/da4/f12a [696658,116720] 0 2024-04-26T04:53:26.336 INFO:tasks.workunit.client.0.smithi077.stdout:3/990: dwrite d0/d9/f2b [0,4194304] 0 2024-04-26T04:53:26.340 INFO:tasks.workunit.client.0.smithi077.stdout:5/934: truncate d1/d42/d62/d121/d87/f122 94760 0 2024-04-26T04:53:26.340 INFO:tasks.workunit.client.0.smithi077.stdout:9/967: rename d9/dd/d1b/d2b/d32/d36/c77 to d9/dd/def/c13c 0 2024-04-26T04:53:26.344 INFO:tasks.workunit.client.0.smithi077.stdout:5/935: dread - d1/d42/d62/d6b/deb/f12c zero size 2024-04-26T04:53:26.348 INFO:tasks.workunit.client.0.smithi077.stdout:5/936: dread - d1/d42/d62/d121/fed zero size 2024-04-26T04:53:26.350 INFO:tasks.workunit.client.0.smithi077.stdout:3/991: getdents d0/dae/ded/d29/db8 0 2024-04-26T04:53:26.352 INFO:tasks.workunit.client.0.smithi077.stdout:3/992: mknod d0/d7/d9a/db7/c139 0 2024-04-26T04:53:26.352 INFO:tasks.workunit.client.0.smithi077.stdout:5/937: link d1/d8/lad d1/d42/dac/l132 0 2024-04-26T04:53:26.355 INFO:tasks.workunit.client.0.smithi077.stdout:3/993: truncate d0/dae/ded/d29/f31 9949974 0 2024-04-26T04:53:26.359 INFO:tasks.workunit.client.0.smithi077.stdout:3/994: dread - d0/d7/d1f/d4d/f113 zero size 2024-04-26T04:53:26.361 INFO:tasks.workunit.client.0.smithi077.stdout:9/968: write d9/dd/f9c [976126,23254] 0 2024-04-26T04:53:26.366 INFO:tasks.workunit.client.0.smithi077.stdout:3/995: dread d0/d7/d9a/db3/f44 [0,4194304] 0 2024-04-26T04:53:26.367 INFO:tasks.workunit.client.0.smithi077.stdout:9/969: unlink d9/d7d/fbf 0 2024-04-26T04:53:26.373 INFO:tasks.workunit.client.0.smithi077.stdout:3/996: dwrite d0/dae/fe2 [0,4194304] 0 2024-04-26T04:53:26.378 INFO:tasks.workunit.client.0.smithi077.stdout:5/938: dwrite d1/d42/da4/d25/d31/d3c/dec/f111 [0,4194304] 0 2024-04-26T04:53:26.378 INFO:tasks.workunit.client.0.smithi077.stdout:9/970: dwrite d9/d45/dcd/ffe [0,4194304] 0 2024-04-26T04:53:26.378 INFO:tasks.workunit.client.0.smithi077.stdout:9/971: chown d9/f8d 12388653 1 2024-04-26T04:53:26.384 INFO:tasks.workunit.client.0.smithi077.stdout:9/972: mknod d9/d3a/d79/d95/dad/c13d 0 2024-04-26T04:53:26.387 INFO:tasks.workunit.client.0.smithi077.stdout:3/997: dwrite d0/dae/ded/d29/db8/f100 [0,4194304] 0 2024-04-26T04:53:26.390 INFO:tasks.workunit.client.0.smithi077.stdout:3/998: dread - d0/d7/d1f/f137 zero size 2024-04-26T04:53:26.391 INFO:tasks.workunit.client.0.smithi077.stdout:3/999: chown d0/d7/d1f/c64 6634 1 2024-04-26T04:53:26.397 INFO:tasks.workunit.client.0.smithi077.stdout:9/973: rmdir d9/d7d 39 2024-04-26T04:53:26.399 INFO:tasks.workunit.client.0.smithi077.stdout:5/939: creat d1/d42/d62/d6b/f133 x:0 0 0 2024-04-26T04:53:26.402 INFO:tasks.workunit.client.0.smithi077.stdout:9/974: link d9/dd/df2/f118 d9/d3a/f13e 0 2024-04-26T04:53:26.404 INFO:tasks.workunit.client.0.smithi077.stdout:9/975: rmdir d9/d35 39 2024-04-26T04:53:26.407 INFO:tasks.workunit.client.0.smithi077.stdout:5/940: rename d1/d42/da4/d25/d31/d3c/l48 to d1/d30/l134 0 2024-04-26T04:53:26.407 INFO:tasks.workunit.client.0.smithi077.stdout:9/976: rmdir d9/dd/d3d/d7c/dc8 39 2024-04-26T04:53:26.408 INFO:tasks.workunit.client.0.smithi077.stdout:5/941: write d1/d42/d62/d121/d87/f122 [157334,71447] 0 2024-04-26T04:53:26.410 INFO:tasks.workunit.client.0.smithi077.stdout:5/942: fdatasync d1/d42/d62/d121/d28/ffc 0 2024-04-26T04:53:26.414 INFO:tasks.workunit.client.0.smithi077.stdout:9/977: dwrite d9/d7d/fa1 [0,4194304] 0 2024-04-26T04:53:26.417 INFO:tasks.workunit.client.0.smithi077.stdout:9/978: unlink d9/d45/dcd/ffe 0 2024-04-26T04:53:26.418 INFO:tasks.workunit.client.0.smithi077.stdout:9/979: chown d9/dd/d1b/d2b/l92 10837346 1 2024-04-26T04:53:26.422 INFO:tasks.workunit.client.0.smithi077.stdout:9/980: write d9/dd/d1b/d2b/d32/d36/d7f/dae/f10d [494559,128828] 0 2024-04-26T04:53:26.428 INFO:tasks.workunit.client.0.smithi077.stdout:9/981: rmdir d9/d35/d99/d101 39 2024-04-26T04:53:26.429 INFO:tasks.workunit.client.0.smithi077.stdout:9/982: chown d9/d3a/d79/d95/fb8 32553112 1 2024-04-26T04:53:26.434 INFO:tasks.workunit.client.0.smithi077.stdout:5/943: write d1/d42/d62/d121/d87/fe5 [826060,96272] 0 2024-04-26T04:53:26.443 INFO:tasks.workunit.client.0.smithi077.stdout:9/983: write d9/dd/d1b/d2b/d32/d36/fb6 [1901213,25266] 0 2024-04-26T04:53:26.456 INFO:tasks.workunit.client.0.smithi077.stdout:9/984: dread d9/dd/d1b/d2b/f2f [0,4194304] 0 2024-04-26T04:53:26.457 INFO:tasks.workunit.client.0.smithi077.stdout:9/985: chown d9/f82 9993 1 2024-04-26T04:53:26.459 INFO:tasks.workunit.client.0.smithi077.stdout:9/986: mkdir d9/d35/d99/d13f 0 2024-04-26T04:53:26.462 INFO:tasks.workunit.client.0.smithi077.stdout:9/987: stat d9/dd/df2/d107/l10b 0 2024-04-26T04:53:26.465 INFO:tasks.workunit.client.0.smithi077.stdout:9/988: symlink d9/dd/d1b/d2b/d32/d36/d7f/dae/d126/l140 0 2024-04-26T04:53:26.466 INFO:tasks.workunit.client.0.smithi077.stdout:9/989: symlink d9/d3a/d79/d95/dad/l141 0 2024-04-26T04:53:26.469 INFO:tasks.workunit.client.0.smithi077.stdout:9/990: rename d9/d45/cd8 to d9/d45/dc4/d116/c142 0 2024-04-26T04:53:26.472 INFO:tasks.workunit.client.0.smithi077.stdout:9/991: rename d9/dd/d1b/d4e/f12e to d9/d45/dc4/d116/f143 0 2024-04-26T04:53:26.473 INFO:tasks.workunit.client.0.smithi077.stdout:9/992: chown d9/dd/d3d/d7c 0 1 2024-04-26T04:53:26.476 INFO:tasks.workunit.client.0.smithi077.stdout:9/993: rename d9/d43/ld9 to d9/dd/d1b/l144 0 2024-04-26T04:53:26.509 INFO:tasks.workunit.client.0.smithi077.stdout:9/994: write d9/dd/d3d/d7c/dc8/fcc [601815,48432] 0 2024-04-26T04:53:26.510 INFO:tasks.workunit.client.0.smithi077.stdout:9/995: dread - d9/d43/f114 zero size 2024-04-26T04:53:26.537 INFO:tasks.workunit.client.0.smithi077.stdout:9/996: dread d9/f1c [4194304,4194304] 0 2024-04-26T04:53:26.616 INFO:tasks.workunit.client.0.smithi077.stdout:9/997: truncate d9/dd/d1b/d2b/d32/d36/d7f/dae/ff5 674552 0 2024-04-26T04:53:26.619 INFO:tasks.workunit.client.0.smithi077.stdout:9/998: mkdir d9/dd/d1b/d145 0 2024-04-26T04:53:26.628 INFO:tasks.workunit.client.0.smithi077.stdout:9/999: write d9/dd/d1b/d2b/f38 [771581,130205] 0 2024-04-26T04:53:26.718 INFO:tasks.workunit.client.0.smithi077.stdout:0/969: sync 2024-04-26T04:53:26.725 INFO:tasks.workunit.client.0.smithi077.stdout:0/970: write d0/d47/f22 [977655,93612] 0 2024-04-26T04:53:26.739 INFO:tasks.workunit.client.0.smithi077.stdout:0/971: dread d0/d19/d1e/f20 [0,4194304] 0 2024-04-26T04:53:26.740 INFO:tasks.workunit.client.0.smithi077.stdout:0/972: chown d0/d19/d1e/d59/d96/d106/d120/d124 1493 1 2024-04-26T04:53:26.742 INFO:tasks.workunit.client.0.smithi077.stdout:0/973: creat d0/d19/d69/f14c x:0 0 0 2024-04-26T04:53:26.744 INFO:tasks.workunit.client.0.smithi077.stdout:0/974: dread d0/d19/d1e/f9d [0,4194304] 0 2024-04-26T04:53:26.745 INFO:tasks.workunit.client.0.smithi077.stdout:0/975: readlink d0/l7 0 2024-04-26T04:53:26.747 INFO:tasks.workunit.client.0.smithi077.stdout:0/976: mkdir d0/d47/d93/d14d 0 2024-04-26T04:53:26.755 INFO:tasks.workunit.client.0.smithi077.stdout:0/977: write d0/d2c/d4e/d133/f142 [806901,99027] 0 2024-04-26T04:53:26.760 INFO:tasks.workunit.client.0.smithi077.stdout:0/978: creat d0/f14e x:0 0 0 2024-04-26T04:53:26.763 INFO:tasks.workunit.client.0.smithi077.stdout:5/944: sync 2024-04-26T04:53:26.769 INFO:tasks.workunit.client.0.smithi077.stdout:5/945: dread d1/d30/d35/ff7 [0,4194304] 0 2024-04-26T04:53:26.771 INFO:tasks.workunit.client.0.smithi077.stdout:0/979: dwrite d0/d19/f35 [0,4194304] 0 2024-04-26T04:53:26.774 INFO:tasks.workunit.client.0.smithi077.stdout:0/980: rmdir d0/d47/d64/d97/df9/d138 39 2024-04-26T04:53:26.779 INFO:tasks.workunit.client.0.smithi077.stdout:0/981: dread d0/d47/d64/d97/dba/fdd [0,4194304] 0 2024-04-26T04:53:26.782 INFO:tasks.workunit.client.0.smithi077.stdout:0/982: write d0/d19/d1e/d81/fff [1005677,98743] 0 2024-04-26T04:53:26.785 INFO:tasks.workunit.client.0.smithi077.stdout:5/946: dread d1/d42/d62/d121/f19 [0,4194304] 0 2024-04-26T04:53:26.787 INFO:tasks.workunit.client.0.smithi077.stdout:5/947: fsync d1/d42/d62/d121/f19 0 2024-04-26T04:53:26.789 INFO:tasks.workunit.client.0.smithi077.stdout:0/983: dwrite d0/d47/d64/d97/ffb [0,4194304] 0 2024-04-26T04:53:26.791 INFO:tasks.workunit.client.0.smithi077.stdout:5/948: mknod d1/d42/da4/dd3/d126/c135 0 2024-04-26T04:53:26.794 INFO:tasks.workunit.client.0.smithi077.stdout:0/984: truncate d0/d47/f50 4931698 0 2024-04-26T04:53:26.796 INFO:tasks.workunit.client.0.smithi077.stdout:0/985: truncate d0/d47/d64/d97/dba/dc4/f10c 2275975 0 2024-04-26T04:53:26.797 INFO:tasks.workunit.client.0.smithi077.stdout:0/986: stat d0/d19/d69/cae 0 2024-04-26T04:53:26.802 INFO:tasks.workunit.client.0.smithi077.stdout:0/987: rename d0/d19/d1e/d59/d96/d106/d120 to d0/d19/d1e/d59/d96/d106/d14f 0 2024-04-26T04:53:26.807 INFO:tasks.workunit.client.0.smithi077.stdout:5/949: write d1/d42/d62/f6f [1872027,90629] 0 2024-04-26T04:53:26.811 INFO:tasks.workunit.client.0.smithi077.stdout:0/988: write d0/fdb [610596,2511] 0 2024-04-26T04:53:26.814 INFO:tasks.workunit.client.0.smithi077.stdout:5/950: dwrite d1/d42/d62/d121/d28/d38/f8b [4194304,4194304] 0 2024-04-26T04:53:26.815 INFO:tasks.workunit.client.0.smithi077.stdout:0/989: creat d0/d19/d1e/f150 x:0 0 0 2024-04-26T04:53:26.820 INFO:tasks.workunit.client.0.smithi077.stdout:5/951: creat d1/d42/d62/d6b/de8/f136 x:0 0 0 2024-04-26T04:53:26.821 INFO:tasks.workunit.client.0.smithi077.stdout:0/990: creat d0/d47/d68/f151 x:0 0 0 2024-04-26T04:53:26.823 INFO:tasks.workunit.client.0.smithi077.stdout:5/952: creat d1/d30/d33/d66/f137 x:0 0 0 2024-04-26T04:53:26.827 INFO:tasks.workunit.client.0.smithi077.stdout:0/991: dwrite d0/d47/d68/f151 [0,4194304] 0 2024-04-26T04:53:26.831 INFO:tasks.workunit.client.0.smithi077.stdout:5/953: link d1/d30/d33/f76 d1/d42/d62/f138 0 2024-04-26T04:53:26.833 INFO:tasks.workunit.client.0.smithi077.stdout:0/992: mknod d0/d47/d64/d97/dba/c152 0 2024-04-26T04:53:26.834 INFO:tasks.workunit.client.0.smithi077.stdout:0/993: fdatasync d0/d47/d68/fa5 0 2024-04-26T04:53:26.837 INFO:tasks.workunit.client.0.smithi077.stdout:0/994: mkdir d0/d47/d68/d153 0 2024-04-26T04:53:26.841 INFO:tasks.workunit.client.0.smithi077.stdout:0/995: creat d0/d19/d8d/f154 x:0 0 0 2024-04-26T04:53:26.844 INFO:tasks.workunit.client.0.smithi077.stdout:0/996: mknod d0/d19/d1e/d59/d96/d106/c155 0 2024-04-26T04:53:26.849 INFO:tasks.workunit.client.0.smithi077.stdout:5/954: dwrite d1/d8/ff [0,4194304] 0 2024-04-26T04:53:26.852 INFO:tasks.workunit.client.0.smithi077.stdout:0/997: dread d0/d47/d93/d94/faa [0,4194304] 0 2024-04-26T04:53:26.854 INFO:tasks.workunit.client.0.smithi077.stdout:5/955: mknod d1/c139 0 2024-04-26T04:53:26.856 INFO:tasks.workunit.client.0.smithi077.stdout:5/956: creat d1/d8/d72/f13a x:0 0 0 2024-04-26T04:53:26.861 INFO:tasks.workunit.client.0.smithi077.stdout:5/957: unlink d1/d42/d62/d6b/cc7 0 2024-04-26T04:53:26.863 INFO:tasks.workunit.client.0.smithi077.stdout:5/958: mknod d1/d8f/c13b 0 2024-04-26T04:53:26.881 INFO:tasks.workunit.client.0.smithi077.stdout:5/959: truncate d1/d8/fc 765890 0 2024-04-26T04:53:26.884 INFO:tasks.workunit.client.0.smithi077.stdout:5/960: dread d1/d42/d62/dc8/fdc [0,4194304] 0 2024-04-26T04:53:26.885 INFO:tasks.workunit.client.0.smithi077.stdout:5/961: chown d1/d42/da4/dd3/c110 7 1 2024-04-26T04:53:26.886 INFO:tasks.workunit.client.0.smithi077.stdout:5/962: fdatasync d1/d30/d35/f5a 0 2024-04-26T04:53:26.903 INFO:tasks.workunit.client.0.smithi077.stdout:5/963: dread d1/d42/d62/d121/f60 [0,4194304] 0 2024-04-26T04:53:26.904 INFO:tasks.workunit.client.0.smithi077.stdout:5/964: read - d1/d42/da4/d25/d31/d3c/f11c zero size 2024-04-26T04:53:26.906 INFO:tasks.workunit.client.0.smithi077.stdout:5/965: symlink d1/d42/d62/dc8/dd5/ddb/l13c 0 2024-04-26T04:53:26.909 INFO:tasks.workunit.client.0.smithi077.stdout:5/966: truncate d1/d42/d62/d121/f2b 2101344 0 2024-04-26T04:53:26.912 INFO:tasks.workunit.client.0.smithi077.stdout:5/967: rename d1/d42/da4/c23 to d1/df6/c13d 0 2024-04-26T04:53:26.920 INFO:tasks.workunit.client.0.smithi077.stdout:5/968: write d1/d30/d35/ff7 [749798,25200] 0 2024-04-26T04:53:26.926 INFO:tasks.workunit.client.0.smithi077.stdout:5/969: write d1/d8c/ff4 [1695327,85399] 0 2024-04-26T04:53:26.928 INFO:tasks.workunit.client.0.smithi077.stdout:5/970: mknod d1/d42/da4/dd3/d126/c13e 0 2024-04-26T04:53:26.937 INFO:tasks.workunit.client.0.smithi077.stdout:5/971: dwrite d1/d42/f69 [0,4194304] 0 2024-04-26T04:53:26.940 INFO:tasks.workunit.client.0.smithi077.stdout:5/972: creat d1/d42/d62/d6b/de8/f13f x:0 0 0 2024-04-26T04:53:26.946 INFO:tasks.workunit.client.0.smithi077.stdout:5/973: getdents d1/d42/d62 0 2024-04-26T04:53:26.950 INFO:tasks.workunit.client.0.smithi077.stdout:5/974: readlink d1/d42/d62/d121/d45/l59 0 2024-04-26T04:53:26.952 INFO:tasks.workunit.client.0.smithi077.stdout:5/975: dread - d1/d42/da4/f64 zero size 2024-04-26T04:53:26.956 INFO:tasks.workunit.client.0.smithi077.stdout:5/976: getdents d1/d8 0 2024-04-26T04:53:26.961 INFO:tasks.workunit.client.0.smithi077.stdout:5/977: mkdir d1/d42/d140 0 2024-04-26T04:53:26.961 INFO:tasks.workunit.client.0.smithi077.stdout:5/978: readlink d1/d42/d62/d121/d28/ld2 0 2024-04-26T04:53:26.963 INFO:tasks.workunit.client.0.smithi077.stdout:5/979: fsync d1/d8/fc 0 2024-04-26T04:53:26.964 INFO:tasks.workunit.client.0.smithi077.stdout:5/980: creat d1/d8/d72/f141 x:0 0 0 2024-04-26T04:53:26.965 INFO:tasks.workunit.client.0.smithi077.stdout:5/981: chown d1/d30/d35/ff7 94340423 1 2024-04-26T04:53:26.966 INFO:tasks.workunit.client.0.smithi077.stdout:5/982: fsync d1/d42/da4/d25/d31/d3c/f11c 0 2024-04-26T04:53:26.966 INFO:tasks.workunit.client.0.smithi077.stdout:5/983: chown d1/c139 98065420 1 2024-04-26T04:53:26.972 INFO:tasks.workunit.client.0.smithi077.stdout:5/984: dwrite d1/d30/d33/d66/f137 [0,4194304] 0 2024-04-26T04:53:26.976 INFO:tasks.workunit.client.0.smithi077.stdout:5/985: symlink d1/d42/d140/l142 0 2024-04-26T04:53:27.293 INFO:tasks.workunit.client.0.smithi077.stdout:0/998: sync 2024-04-26T04:53:27.293 INFO:tasks.workunit.client.0.smithi077.stdout:5/986: sync 2024-04-26T04:53:27.295 INFO:tasks.workunit.client.0.smithi077.stdout:5/987: mkdir d1/d42/da4/d84/d143 0 2024-04-26T04:53:27.296 INFO:tasks.workunit.client.0.smithi077.stdout:0/999: mkdir d0/d47/d68/d117/d156 0 2024-04-26T04:53:27.300 INFO:tasks.workunit.client.0.smithi077.stdout:5/988: dwrite d1/d8/d72/f141 [0,4194304] 0 2024-04-26T04:53:27.315 INFO:tasks.workunit.client.0.smithi077.stdout:5/989: dread d1/d8/f55 [0,4194304] 0 2024-04-26T04:53:27.324 INFO:tasks.workunit.client.0.smithi077.stdout:5/990: getdents d1/d42/d62/d6b/de8 0 2024-04-26T04:53:27.336 INFO:tasks.workunit.client.0.smithi077.stdout:5/991: dread d1/d42/d62/d121/f2a [0,4194304] 0 2024-04-26T04:53:27.337 INFO:tasks.workunit.client.0.smithi077.stdout:5/992: write d1/fe3 [1249501,107799] 0 2024-04-26T04:53:27.339 INFO:tasks.workunit.client.0.smithi077.stdout:5/993: creat d1/d30/d35/de7/f144 x:0 0 0 2024-04-26T04:53:27.340 INFO:tasks.workunit.client.0.smithi077.stdout:5/994: dread - d1/d42/d62/d6b/deb/f12f zero size 2024-04-26T04:53:27.341 INFO:tasks.workunit.client.0.smithi077.stdout:5/995: creat d1/d42/da4/d25/d31/d3c/f145 x:0 0 0 2024-04-26T04:53:27.344 INFO:tasks.workunit.client.0.smithi077.stdout:5/996: rename d1/d42/d62/d121/d87/fc3 to d1/d8c/f146 0 2024-04-26T04:53:27.345 INFO:tasks.workunit.client.0.smithi077.stdout:5/997: write d1/d42/d62/d121/d87/fe5 [376629,33473] 0 2024-04-26T04:53:27.346 INFO:tasks.workunit.client.0.smithi077.stdout:5/998: fdatasync d1/d42/d62/d121/d28/ffc 0 2024-04-26T04:53:27.353 INFO:tasks.workunit.client.0.smithi077.stdout:5/999: write d1/d8f/fe1 [582852,25764] 0 2024-04-26T04:53:27.357 INFO:tasks.workunit.client.0.smithi077.stderr:+ rm -rf -- ./tmp.FvYFaNvwQz 2024-04-26T04:53:27.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:27 smithi077 ceph-mon[26311]: pgmap v33: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 49 MiB/s rd, 99 MiB/s wr, 288 op/s 2024-04-26T04:53:27.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:27 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:27.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:27 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:27.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:27 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:27 smithi078 ceph-mon[33746]: pgmap v33: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 49 MiB/s rd, 99 MiB/s wr, 288 op/s 2024-04-26T04:53:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:27 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:27 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:27 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:28.862 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:28 smithi077 ceph-mon[26311]: Reconfiguring grafana.smithi077 (dependencies changed)... 2024-04-26T04:53:28.862 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:28 smithi077 ceph-mon[26311]: Reconfiguring daemon grafana.smithi077 on smithi077 2024-04-26T04:53:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:28 smithi078 ceph-mon[33746]: Reconfiguring grafana.smithi077 (dependencies changed)... 2024-04-26T04:53:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:28 smithi078 ceph-mon[33746]: Reconfiguring daemon grafana.smithi077 on smithi077 2024-04-26T04:53:29.886 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:29 smithi077 ceph-mon[26311]: pgmap v34: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 64 MiB/s rd, 125 MiB/s wr, 402 op/s 2024-04-26T04:53:30.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:29 smithi078 ceph-mon[33746]: pgmap v34: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 64 MiB/s rd, 125 MiB/s wr, 402 op/s 2024-04-26T04:53:31.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:31 smithi077 ceph-mon[26311]: pgmap v35: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 34 MiB/s rd, 67 MiB/s wr, 251 op/s 2024-04-26T04:53:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:31 smithi078 ceph-mon[33746]: pgmap v35: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 34 MiB/s rd, 67 MiB/s wr, 251 op/s 2024-04-26T04:53:33.156 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T04:53:33.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-26T04:53:33.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:33 smithi077 ceph-mon[26311]: pgmap v36: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 34 MiB/s rd, 67 MiB/s wr, 325 op/s 2024-04-26T04:53:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:33 smithi078 ceph-mon[33746]: pgmap v36: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 34 MiB/s rd, 67 MiB/s wr, 325 op/s 2024-04-26T04:53:35.883 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:35 smithi077 ceph-mon[26311]: pgmap v37: 65 pgs: 65 active+clean; 2.2 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 20 MiB/s rd, 35 MiB/s wr, 209 op/s 2024-04-26T04:53:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:35 smithi078 ceph-mon[33746]: pgmap v37: 65 pgs: 65 active+clean; 2.2 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 20 MiB/s rd, 35 MiB/s wr, 209 op/s 2024-04-26T04:53:37.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:37 smithi077 ceph-mon[26311]: pgmap v38: 65 pgs: 65 active+clean; 2.2 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 14 MiB/s rd, 27 MiB/s wr, 194 op/s 2024-04-26T04:53:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:37 smithi078 ceph-mon[33746]: pgmap v38: 65 pgs: 65 active+clean; 2.2 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 14 MiB/s rd, 27 MiB/s wr, 194 op/s 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.397+0000 7fce63cd9700 1 -- 172.21.15.77:0/2319741390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 msgr2=0x7fce5c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.397+0000 7fce63cd9700 1 --2- 172.21.15.77:0/2319741390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c102330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce4c004940 tx=0x7fce4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.397+0000 7fce63cd9700 1 -- 172.21.15.77:0/2319741390 shutdown_connections 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.397+0000 7fce63cd9700 1 --2- 172.21.15.77:0/2319741390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c102330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.397+0000 7fce63cd9700 1 --2- 172.21.15.77:0/2319741390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.398+0000 7fce63cd9700 1 -- 172.21.15.77:0/2319741390 >> 172.21.15.77:0/2319741390 conn(0x7fce5c0f7420 msgr2=0x7fce5c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.398+0000 7fce63cd9700 1 -- 172.21.15.77:0/2319741390 shutdown_connections 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.398+0000 7fce63cd9700 1 -- 172.21.15.77:0/2319741390 wait complete. 2024-04-26T04:53:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.398+0000 7fce63cd9700 1 Processor -- start 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.398+0000 7fce63cd9700 1 -- start start 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce63cd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce63cd9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce63cd9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c10e4e0 con 0x7fce5c0fff50 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce63cd9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c10ed30 con 0x7fce5c066af0 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:43754/0 (socket says 172.21.15.77:43754) 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 -- 172.21.15.77:0/141786314 learned_addr learned my addr 172.21.15.77:0/141786314 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61274700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 -- 172.21.15.77:0/141786314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 msgr2=0x7fce5c10dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c10dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 -- 172.21.15.77:0/141786314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c005040 con 0x7fce5c066af0 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce61a75700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce58004ed0 tx=0x7fce5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:39.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.399+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce58007ea0 con 0x7fce5c066af0 2024-04-26T04:53:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.400+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce5800f040 con 0x7fce5c066af0 2024-04-26T04:53:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.400+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c10efb0 con 0x7fce5c066af0 2024-04-26T04:53:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.400+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce58013bb0 con 0x7fce5c066af0 2024-04-26T04:53:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.400+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c10f430 con 0x7fce5c066af0 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.401+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c05f5c0 con 0x7fce5c066af0 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.401+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fce58013d10 con 0x7fce5c066af0 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.402+0000 7fce527fc700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 0x7fce4806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.402+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fce5808e030 con 0x7fce5c066af0 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.402+0000 7fce61274700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 0x7fce4806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.402+0000 7fce61274700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 0x7fce4806dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00b0d0 tx=0x7fce4c009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:39.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.407+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fce5805b920 con 0x7fce5c066af0 2024-04-26T04:53:39.658 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:39 smithi077 ceph-mon[26311]: pgmap v39: 65 pgs: 65 active+clean; 1.3 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 15 MiB/s rd, 28 MiB/s wr, 263 op/s 2024-04-26T04:53:39.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.704+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce5c10f810 con 0x7fce4806b790 2024-04-26T04:53:39.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.705+0000 7fce527fc700 1 -- 172.21.15.77:0/141786314 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fce5c10f810 con 0x7fce4806b790 2024-04-26T04:53:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.707+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 msgr2=0x7fce4806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.707+0000 7fce63cd9700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 0x7fce4806dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00b0d0 tx=0x7fce4c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.707+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 msgr2=0x7fce5c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.707+0000 7fce63cd9700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce58004ed0 tx=0x7fce5800b040 comp rx=0 tx=0).stop 2024-04-26T04:53:39.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 shutdown_connections 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fce4806b790 0x7fce4806dc40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5c066af0 0x7fce5c10d9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 --2- 172.21.15.77:0/141786314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce5c0fff50 0x7fce5c10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 >> 172.21.15.77:0/141786314 conn(0x7fce5c0f7420 msgr2=0x7fce5c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 shutdown_connections 2024-04-26T04:53:39.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.708+0000 7fce63cd9700 1 -- 172.21.15.77:0/141786314 wait complete. 2024-04-26T04:53:39.722 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:53:39.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 -- 172.21.15.77:0/4170728478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234070890 msgr2=0x7f4234070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 --2- 172.21.15.77:0/4170728478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234070890 0x7f4234070d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f421c004940 tx=0x7f421c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 -- 172.21.15.77:0/4170728478 shutdown_connections 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 --2- 172.21.15.77:0/4170728478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234070890 0x7f4234070d00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 --2- 172.21.15.77:0/4170728478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234071e40 0x7f4234070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.874+0000 7f4239308700 1 -- 172.21.15.77:0/4170728478 >> 172.21.15.77:0/4170728478 conn(0x7f42340f8370 msgr2=0x7f42340fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:39.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.875+0000 7f4239308700 1 -- 172.21.15.77:0/4170728478 shutdown_connections 2024-04-26T04:53:39.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.875+0000 7f4239308700 1 -- 172.21.15.77:0/4170728478 wait complete. 2024-04-26T04:53:39.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.875+0000 7f4239308700 1 Processor -- start 2024-04-26T04:53:39.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4239308700 1 -- start start 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4239308700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 0x7f4234113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4239308700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 0x7f4234114390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4239308700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4234115100 con 0x7f4234070890 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4239308700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4234115270 con 0x7f4234071e40 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4233fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 0x7f4234113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f42337fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 0x7f4234114390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4233fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 0x7f4234113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55700/0 (socket says 172.21.15.77:55700) 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.876+0000 7f4233fff700 1 -- 172.21.15.77:0/253302787 learned_addr learned my addr 172.21.15.77:0/253302787 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f42337fe700 1 -- 172.21.15.77:0/253302787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 msgr2=0x7f4234113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f42337fe700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 0x7f4234113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f42337fe700 1 -- 172.21.15.77:0/253302787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c005040 con 0x7f4234071e40 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f42337fe700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 0x7f4234114390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f421c009fd0 tx=0x7f421c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c00e070 con 0x7f4234071e40 2024-04-26T04:53:39.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f421c01c400 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c00b650 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4234115490 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.877+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42341159f0 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.878+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423405f5c0 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.879+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f421c013af0 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.879+0000 7f4230ff9700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 0x7f422006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.880+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f421c08b8a0 con 0x7f4234071e40 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.880+0000 7f4233fff700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 0x7f422006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:39.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.880+0000 7f4233fff700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 0x7f422006dc90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4234113bc0 tx=0x7f422400b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:39.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:39.884+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f421c055590 con 0x7f4234071e40 2024-04-26T04:53:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:39 smithi078 ceph-mon[33746]: pgmap v39: 65 pgs: 65 active+clean; 1.3 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 15 MiB/s rd, 28 MiB/s wr, 263 op/s 2024-04-26T04:53:40.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.191+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4234115c60 con 0x7f422006b7e0 2024-04-26T04:53:40.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.192+0000 7f4230ff9700 1 -- 172.21.15.77:0/253302787 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f4234115c60 con 0x7f422006b7e0 2024-04-26T04:53:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 msgr2=0x7f422006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 0x7f422006dc90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4234113bc0 tx=0x7f422400b040 comp rx=0 tx=0).stop 2024-04-26T04:53:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 msgr2=0x7f4234114390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 0x7f4234114390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f421c009fd0 tx=0x7f421c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 shutdown_connections 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f422006b7e0 0x7f422006dc90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4234070890 0x7f4234113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 --2- 172.21.15.77:0/253302787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4234071e40 0x7f4234114390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.194+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 >> 172.21.15.77:0/253302787 conn(0x7f42340f8370 msgr2=0x7f42340f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.195+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 shutdown_connections 2024-04-26T04:53:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.195+0000 7f4239308700 1 -- 172.21.15.77:0/253302787 wait complete. 2024-04-26T04:53:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.318+0000 7ff61c552700 1 -- 172.21.15.77:0/397916933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fce70 msgr2=0x7ff6140fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.318+0000 7ff61c552700 1 --2- 172.21.15.77:0/397916933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fce70 0x7ff6140fd280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff610004940 tx=0x7ff610009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 -- 172.21.15.77:0/397916933 shutdown_connections 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 --2- 172.21.15.77:0/397916933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fdfe0 0x7ff6140fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 --2- 172.21.15.77:0/397916933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fce70 0x7ff6140fd280 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 -- 172.21.15.77:0/397916933 >> 172.21.15.77:0/397916933 conn(0x7ff6140f8510 msgr2=0x7ff6140fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 -- 172.21.15.77:0/397916933 shutdown_connections 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 -- 172.21.15.77:0/397916933 wait complete. 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 Processor -- start 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.319+0000 7ff61c552700 1 -- start start 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff61c552700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fce70 0x7ff61410fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff61c552700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff61c552700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6141105f0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff61c552700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff614110760 con 0x7ff6140fce70 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55720/0 (socket says 172.21.15.77:55720) 2024-04-26T04:53:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 -- 172.21.15.77:0/103935543 learned_addr learned my addr 172.21.15.77:0/103935543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff61a2ee700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fce70 0x7ff61410fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 -- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fce70 msgr2=0x7ff61410fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fce70 0x7ff61410fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 -- 172.21.15.77:0/103935543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610005040 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.320+0000 7ff619aed700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff604002a80 tx=0x7ff6040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.321+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff604007cc0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.321+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6141151c0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.321+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60400f040 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.321+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60400dac0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.321+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff614115780 con 0x7ff6140fdfe0 2024-04-26T04:53:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.322+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f80052e0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.323+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7ff60400dc20 con 0x7ff6140fdfe0 2024-04-26T04:53:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.323+0000 7ff60affd700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 0x7ff60006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.324+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff60408a1d0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.324+0000 7ff61a2ee700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 0x7ff60006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.324+0000 7ff61a2ee700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 0x7ff60006dc50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b3b0 tx=0x7ff610009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.329+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff6040577e0 con 0x7ff6140fdfe0 2024-04-26T04:53:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.635+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5f8000bf0 con 0x7ff60006b7a0 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.646+0000 7ff60affd700 1 -- 172.21.15.77:0/103935543 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff5f8000bf0 con 0x7ff60006b7a0 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 starting - - - - 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (9m) 62s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (8m) 64s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (7m) 62s ago 8m 85.2M - 9.4.7 954c08fa6188 b75d4cf89d5a 2024-04-26T04:53:40.647 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (3m) 62s ago 3m 177M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (3m) 62s ago 3m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (3m) 64s ago 3m 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (3m) 64s ago 3m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (10m) 62s ago 10m 344M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (87s) 64s ago 8m 490M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 70ff04585016 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (10m) 62s ago 10m 49.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (8m) 64s ago 8m 38.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 starting - - - - 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (8m) 64s ago 8m 16.4M - 1.5.0 0da6a335fe13 a654efa40eda 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (7m) 62s ago 7m 78.1M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (6m) 62s ago 6m 85.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (6m) 62s ago 6m 76.9M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (6m) 64s ago 6m 99.3M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:53:40.648 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (5m) 64s ago 5m 89.1M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:53:40.649 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (5m) 64s ago 5m 85.9M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:53:40.649 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (7m) 62s ago 8m 58.3M - 2.43.0 a07b618ecd1d 2dc67045dee6 2024-04-26T04:53:40.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.648+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 msgr2=0x7ff60006dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.648+0000 7ff61c552700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 0x7ff60006dc50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b3b0 tx=0x7ff610009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:40.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.648+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 msgr2=0x7ff61410fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.648+0000 7ff61c552700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff604002a80 tx=0x7ff6040046b0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 shutdown_connections 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff60006b7a0 0x7ff60006dc50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6140fce70 0x7ff61410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 --2- 172.21.15.77:0/103935543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6140fdfe0 0x7ff61410fff0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 >> 172.21.15.77:0/103935543 conn(0x7ff6140f8510 msgr2=0x7ff6140f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 shutdown_connections 2024-04-26T04:53:40.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.649+0000 7ff61c552700 1 -- 172.21.15.77:0/103935543 wait complete. 2024-04-26T04:53:40.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 -- 172.21.15.77:0/1035023234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c0fd270 msgr2=0x7fe08c0fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 --2- 172.21.15.77:0/1035023234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c0fd270 0x7fe08c0fd680 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe07c004940 tx=0x7fe07c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:40.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 -- 172.21.15.77:0/1035023234 shutdown_connections 2024-04-26T04:53:40.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 --2- 172.21.15.77:0/1035023234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fdbc0 0x7fe08c0fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 --2- 172.21.15.77:0/1035023234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c0fd270 0x7fe08c0fd680 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 -- 172.21.15.77:0/1035023234 >> 172.21.15.77:0/1035023234 conn(0x7fe08c0f6b80 msgr2=0x7fe08c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:40.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 -- 172.21.15.77:0/1035023234 shutdown_connections 2024-04-26T04:53:40.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.783+0000 7fe090ee3700 1 -- 172.21.15.77:0/1035023234 wait complete. 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 Processor -- start 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 -- start start 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 0x7fe08c10fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c110640 con 0x7fe08c110180 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe090ee3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c110780 con 0x7fe08c0fd270 2024-04-26T04:53:40.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.784+0000 7fe089d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55730/0 (socket says 172.21.15.77:55730) 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 -- 172.21.15.77:0/3018218197 learned_addr learned my addr 172.21.15.77:0/3018218197 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe08a59c700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 0x7fe08c10fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 -- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 msgr2=0x7fe08c10fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 0x7fe08c10fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 -- 172.21.15.77:0/3018218197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c005040 con 0x7fe08c110180 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe08a59c700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 0x7fe08c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe089d9b700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe080002a50 tx=0x7fe0800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:40.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.785+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe080007cc0 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.786+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c111630 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.786+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c111bc0 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.786+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe08000f040 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.786+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08000da30 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.787+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c0052e0 con 0x7fe08c110180 2024-04-26T04:53:40.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.788+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fe08000dc70 con 0x7fe08c110180 2024-04-26T04:53:40.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.788+0000 7fe07affd700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 0x7fe07406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:40.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.788+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe08008a090 con 0x7fe08c110180 2024-04-26T04:53:40.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.789+0000 7fe08a59c700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 0x7fe07406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:40.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.789+0000 7fe08a59c700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 0x7fe07406dc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:40.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:40.796+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe0800576b0 con 0x7fe08c110180 2024-04-26T04:53:40.872 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:40 smithi077 ceph-mon[26311]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:41.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:40 smithi078 ceph-mon[33746]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.231+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe06c005860 con 0x7fe08c110180 2024-04-26T04:53:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.231+0000 7fe07affd700 1 -- 172.21.15.77:0/3018218197 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fe080057240 con 0x7fe08c110180 2024-04-26T04:53:41.232 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:53:41.232 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:53:41.232 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-26T04:53:41.233 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:53:41.234 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:53:41.234 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:53:41.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 msgr2=0x7fe07406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 0x7fe07406dc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:41.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 msgr2=0x7fe08c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe080002a50 tx=0x7fe0800046b0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 shutdown_connections 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe07406b790 0x7fe07406dc40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe08c0fd270 0x7fe08c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 --2- 172.21.15.77:0/3018218197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08c110180 0x7fe08c110fd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 >> 172.21.15.77:0/3018218197 conn(0x7fe08c0f6b80 msgr2=0x7fe08c0fb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 shutdown_connections 2024-04-26T04:53:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.234+0000 7fe090ee3700 1 -- 172.21.15.77:0/3018218197 wait complete. 2024-04-26T04:53:41.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.378+0000 7f6e156d9700 1 -- 172.21.15.77:0/3626299826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 msgr2=0x7f6e100fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 --2- 172.21.15.77:0/3626299826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e100fc0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e00004940 tx=0x7f6e00009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 -- 172.21.15.77:0/3626299826 shutdown_connections 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 --2- 172.21.15.77:0/3626299826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 0x7f6e100fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 --2- 172.21.15.77:0/3626299826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e100fc0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 -- 172.21.15.77:0/3626299826 >> 172.21.15.77:0/3626299826 conn(0x7f6e100f7420 msgr2=0x7f6e100f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 -- 172.21.15.77:0/3626299826 shutdown_connections 2024-04-26T04:53:41.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.379+0000 7f6e156d9700 1 -- 172.21.15.77:0/3626299826 wait complete. 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 Processor -- start 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 -- start start 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 0x7f6e1010cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1010d3d0 con 0x7f6e100fcf70 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e156d9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1010dc20 con 0x7f6e100fbcc0 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e0e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 0x7f6e1010cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e0effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e0effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:43842/0 (socket says 172.21.15.77:43842) 2024-04-26T04:53:41.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.380+0000 7f6e0effd700 1 -- 172.21.15.77:0/2434617175 learned_addr learned my addr 172.21.15.77:0/2434617175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e0effd700 1 -- 172.21.15.77:0/2434617175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 msgr2=0x7f6e1010cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e0effd700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 0x7f6e1010cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e0effd700 1 -- 172.21.15.77:0/2434617175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00005040 con 0x7f6e100fbcc0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e0effd700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000b0d0 tx=0x7f6e00009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0000e030 con 0x7f6e100fbcc0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e0001c400 con 0x7f6e100fbcc0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0000b650 con 0x7f6e100fbcc0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1010dea0 con 0x7f6e100fbcc0 2024-04-26T04:53:41.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.381+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1010e3e0 con 0x7f6e100fbcc0 2024-04-26T04:53:41.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.382+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1005f5c0 con 0x7f6e100fbcc0 2024-04-26T04:53:41.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.383+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f6e0001c570 con 0x7f6e100fbcc0 2024-04-26T04:53:41.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.384+0000 7f6df77fe700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 0x7f6df806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.384+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6e0008bd20 con 0x7f6e100fbcc0 2024-04-26T04:53:41.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.384+0000 7f6e0e7fc700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 0x7f6df806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.385+0000 7f6e0e7fc700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 0x7f6df806dc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009b80 tx=0x7f6e0400b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:41.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.388+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6e000594e0 con 0x7f6e100fbcc0 2024-04-26T04:53:41.794 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:53:41.794 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:53:41.794 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:53:41.795 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:53:41.796 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:41.797 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:53:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.792+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e1010e030 con 0x7f6e100fbcc0 2024-04-26T04:53:41.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.793+0000 7f6df77fe700 1 -- 172.21.15.77:0/2434617175 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f6e1010e030 con 0x7f6e100fbcc0 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 msgr2=0x7f6df806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 0x7f6df806dc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009b80 tx=0x7f6e0400b040 comp rx=0 tx=0).stop 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 msgr2=0x7f6e1010c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000b0d0 tx=0x7f6e00009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 shutdown_connections 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f6df806b790 0x7f6df806dc40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e100fbcc0 0x7f6e1010c890 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000b0d0 tx=0x7f6e00009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 --2- 172.21.15.77:0/2434617175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e100fcf70 0x7f6e1010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.798+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 >> 172.21.15.77:0/2434617175 conn(0x7f6e100f7420 msgr2=0x7f6e100f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.799+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 shutdown_connections 2024-04-26T04:53:41.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.800+0000 7f6e156d9700 1 -- 172.21.15.77:0/2434617175 wait complete. 2024-04-26T04:53:41.804 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:53:41.912 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: pgmap v40: 65 pgs: 65 active+clean; 1.3 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 19 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3018218197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:41.913 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:41 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:41.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 -- 172.21.15.77:0/3901533635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e81022e0 msgr2=0x7f00e81046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 --2- 172.21.15.77:0/3901533635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e81022e0 0x7f00e81046c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004880 tx=0x7f00d8009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 -- 172.21.15.77:0/3901533635 shutdown_connections 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 --2- 172.21.15.77:0/3901533635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e81022e0 0x7f00e81046c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 --2- 172.21.15.77:0/3901533635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e8066ed0 0x7f00e8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 -- 172.21.15.77:0/3901533635 >> 172.21.15.77:0/3901533635 conn(0x7f00e80f7430 msgr2=0x7f00e80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.959+0000 7f00f017a700 1 -- 172.21.15.77:0/3901533635 shutdown_connections 2024-04-26T04:53:41.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.960+0000 7f00f017a700 1 -- 172.21.15.77:0/3901533635 wait complete. 2024-04-26T04:53:41.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.960+0000 7f00f017a700 1 Processor -- start 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00f017a700 1 -- start start 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00f017a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 0x7f00e810fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00f017a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 0x7f00e81100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00f017a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e81106a0 con 0x7f00e81022e0 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00f017a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8110810 con 0x7f00e8066ed0 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00edf16700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 0x7f00e810fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00edf16700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 0x7f00e810fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:43852/0 (socket says 172.21.15.77:43852) 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00edf16700 1 -- 172.21.15.77:0/2709416338 learned_addr learned my addr 172.21.15.77:0/2709416338 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.961+0000 7f00ed715700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 0x7f00e81100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00ed715700 1 -- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 msgr2=0x7f00e810fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00ed715700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 0x7f00e810fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:41.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00ed715700 1 -- 172.21.15.77:0/2709416338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8005040 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00ed715700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 0x7f00e81100a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00d80138f0 tx=0x7f00d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d800e030 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.962+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e8115210 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.963+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e81157d0 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.963+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00d8013e60 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.963+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d801ce80 con 0x7f00e81022e0 2024-04-26T04:53:41.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.964+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e804c710 con 0x7f00e81022e0 2024-04-26T04:53:41.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.964+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f00d801c690 con 0x7f00e81022e0 2024-04-26T04:53:41.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.965+0000 7f00de7fc700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 0x7f00d406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:41.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.965+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f00d808bb20 con 0x7f00e81022e0 2024-04-26T04:53:41.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.965+0000 7f00edf16700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 0x7f00d406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:41.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.966+0000 7f00edf16700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 0x7f00d406dc50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009b80 tx=0x7f00e400b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:41.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:41.972+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f00d8059460 con 0x7f00e81022e0 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: pgmap v40: 65 pgs: 65 active+clean; 1.3 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 19 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3018218197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:41 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:42.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.256+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00e80f8f20 con 0x7f00d406b7a0 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.257+0000 7f00de7fc700 1 -- 172.21.15.77:0/2709416338 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f00e80f8f20 con 0x7f00d406b7a0 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "1/21 daemons upgraded", 2024-04-26T04:53:42.258 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "", 2024-04-26T04:53:42.259 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:53:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 msgr2=0x7f00d406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 0x7f00d406dc50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009b80 tx=0x7f00e400b040 comp rx=0 tx=0).stop 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 msgr2=0x7f00e81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 0x7f00e81100a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00d80138f0 tx=0x7f00d8009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 shutdown_connections 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f00d406b7a0 0x7f00d406dc50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00e8066ed0 0x7f00e810fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 --2- 172.21.15.77:0/2709416338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e81022e0 0x7f00e81100a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 >> 172.21.15.77:0/2709416338 conn(0x7f00e80f7430 msgr2=0x7f00e80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:42.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 shutdown_connections 2024-04-26T04:53:42.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.260+0000 7f00f017a700 1 -- 172.21.15.77:0/2709416338 wait complete. 2024-04-26T04:53:42.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.375+0000 7f2ee7a73700 1 -- 172.21.15.77:0/3328577880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 msgr2=0x7f2ee0102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.375+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/3328577880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee0102330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0004940 tx=0x7f2ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 -- 172.21.15.77:0/3328577880 shutdown_connections 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/3328577880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee0102330 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/3328577880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 -- 172.21.15.77:0/3328577880 >> 172.21.15.77:0/3328577880 conn(0x7f2ee00f7440 msgr2=0x7f2ee00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 -- 172.21.15.77:0/3328577880 shutdown_connections 2024-04-26T04:53:42.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 -- 172.21.15.77:0/3328577880 wait complete. 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 Processor -- start 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.376+0000 7f2ee7a73700 1 -- start start 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee7a73700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee7a73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee7a73700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00fd7d0 con 0x7f2ee00fff10 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee7a73700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00fbe10 con 0x7f2ee0066f70 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55782/0 (socket says 172.21.15.77:55782) 2024-04-26T04:53:42.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 -- 172.21.15.77:0/2961327328 learned_addr learned my addr 172.21.15.77:0/2961327328 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee580f700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 -- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 msgr2=0x7f2ee00fd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00fd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 -- 172.21.15.77:0/2961327328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0005040 con 0x7f2ee00fff10 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee580f700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.377+0000 7f2ee500e700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0004060 tx=0x7f2ed0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.378+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed000e070 con 0x7f2ee00fff10 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.378+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee00fc090 con 0x7f2ee00fff10 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.378+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ed000bbf0 con 0x7f2ee00fff10 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.378+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee00fc4b0 con 0x7f2ee00fff10 2024-04-26T04:53:42.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.378+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed001c400 con 0x7f2ee00fff10 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.379+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee004c710 con 0x7f2ee00fff10 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.380+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f2ed0013420 con 0x7f2ee00fff10 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.380+0000 7f2ed67fc700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 0x7f2ecc06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.380+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2ed008bd00 con 0x7f2ee00fff10 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.380+0000 7f2ee580f700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 0x7f2ecc06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:42.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.381+0000 7f2ee580f700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 0x7f2ecc06dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fd0 tx=0x7f2edc009ee0 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.385+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2ed00593a0 con 0x7f2ee00fff10 2024-04-26T04:53:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.740+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2ee01058e0 con 0x7f2ee00fff10 2024-04-26T04:53:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.740+0000 7f2ed67fc700 1 -- 172.21.15.77:0/2961327328 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2ed0058f30 con 0x7f2ee00fff10 2024-04-26T04:53:42.741 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:53:42.741 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:53:42.741 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:53:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 msgr2=0x7f2ecc06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 0x7f2ecc06dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fd0 tx=0x7f2edc009ee0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 msgr2=0x7f2ee00fb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0004060 tx=0x7f2ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 shutdown_connections 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f2ecc06b870 0x7f2ecc06dd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee0066f70 0x7f2ee00fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 --2- 172.21.15.77:0/2961327328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ee00fff10 0x7f2ee00fb8d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 >> 172.21.15.77:0/2961327328 conn(0x7f2ee00f7440 msgr2=0x7f2ee00f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 shutdown_connections 2024-04-26T04:53:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:53:42.744+0000 7f2ee7a73700 1 -- 172.21.15.77:0/2961327328 wait complete. 2024-04-26T04:53:42.938 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:42 smithi077 ceph-mon[26311]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:53:42.938 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:42 smithi077 ceph-mon[26311]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:53:42.938 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:42 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2434617175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:53:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:42 smithi078 ceph-mon[33746]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:53:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:42 smithi078 ceph-mon[33746]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:53:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:42 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2434617175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:53:43.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:43 smithi077 ceph-mon[26311]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:43.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:43 smithi077 ceph-mon[26311]: pgmap v41: 65 pgs: 65 active+clean; 947 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-04-26T04:53:43.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:43 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2961327328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:43 smithi078 ceph-mon[33746]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:43 smithi078 ceph-mon[33746]: pgmap v41: 65 pgs: 65 active+clean; 947 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-04-26T04:53:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:43 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2961327328' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:44.682 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T04:53:44.682 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T04:53:45.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:45 smithi077 ceph-mon[26311]: pgmap v42: 65 pgs: 65 active+clean; 842 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-04-26T04:53:46.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:45 smithi078 ceph-mon[33746]: pgmap v42: 65 pgs: 65 active+clean; 842 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-04-26T04:53:47.928 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:47 smithi077 ceph-mon[26311]: pgmap v43: 65 pgs: 65 active+clean; 842 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-04-26T04:53:48.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:47 smithi078 ceph-mon[33746]: pgmap v43: 65 pgs: 65 active+clean; 842 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-04-26T04:53:49.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:49 smithi077 ceph-mon[26311]: pgmap v44: 65 pgs: 65 active+clean; 289 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.0 MiB/s wr, 183 op/s 2024-04-26T04:53:50.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:49 smithi078 ceph-mon[33746]: pgmap v44: 65 pgs: 65 active+clean; 289 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.0 MiB/s wr, 183 op/s 2024-04-26T04:53:50.969 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:50 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:50.969 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:50 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:50.969 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:50 smithi078 ceph-mon[33746]: Reconfiguring node-exporter.smithi078 (dependencies changed)... 2024-04-26T04:53:50.969 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:50 smithi078 ceph-mon[33746]: Reconfiguring daemon node-exporter.smithi078 on smithi078 2024-04-26T04:53:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:50 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:50 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:50 smithi077 ceph-mon[26311]: Reconfiguring node-exporter.smithi078 (dependencies changed)... 2024-04-26T04:53:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:50 smithi077 ceph-mon[26311]: Reconfiguring daemon node-exporter.smithi078 on smithi078 2024-04-26T04:53:52.049 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:51 smithi078 ceph-mon[33746]: pgmap v45: 65 pgs: 65 active+clean; 289 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-04-26T04:53:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:51 smithi077 ceph-mon[26311]: pgmap v45: 65 pgs: 65 active+clean; 289 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-04-26T04:53:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077.front.sepia.ceph.com:9093"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077.front.sepia.ceph.com:9093"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077.front.sepia.ceph.com:3000"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077.front.sepia.ceph.com:3000"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:53:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: Upgrade: Need to upgrade myself (mgr.smithi078.navvuw) 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: Upgrade: Need to upgrade myself (mgr.smithi078.navvuw) 2024-04-26T04:53:53.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:53 smithi077 ceph-mon[26311]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-04-26T04:53:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:53:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077.front.sepia.ceph.com:9093"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077.front.sepia.ceph.com:9093"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077.front.sepia.ceph.com:3000"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077.front.sepia.ceph.com:3000"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:53:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:53:53.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:53:53.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: Upgrade: Need to upgrade myself (mgr.smithi078.navvuw) 2024-04-26T04:53:53.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: Upgrade: Need to upgrade myself (mgr.smithi078.navvuw) 2024-04-26T04:53:53.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:53 smithi078 ceph-mon[33746]: pgmap v46: 65 pgs: 65 active+clean; 295 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2024-04-26T04:53:55.984 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:55 smithi077 ceph-mon[26311]: pgmap v47: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-26T04:53:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:55 smithi078 ceph-mon[33746]: pgmap v47: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-26T04:53:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:57 smithi077 ceph-mon[26311]: pgmap v48: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-26T04:53:57.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:57 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:57 smithi078 ceph-mon[33746]: pgmap v48: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-26T04:53:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:57 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:59.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:59 smithi077 ceph-mon[26311]: pgmap v49: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-26T04:53:59.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:53:59 smithi077 ceph-mon[26311]: mgrmap e26: smithi078.navvuw(active, since 92s), standbys: smithi077.hkvsor 2024-04-26T04:54:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:59 smithi078 ceph-mon[33746]: pgmap v49: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-26T04:54:00.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:53:59 smithi078 ceph-mon[33746]: mgrmap e26: smithi078.navvuw(active, since 92s), standbys: smithi077.hkvsor 2024-04-26T04:54:01.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:01 smithi077 ceph-mon[26311]: pgmap v50: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-26T04:54:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:01 smithi078 ceph-mon[33746]: pgmap v50: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-04-26T04:54:03.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:03 smithi077 ceph-mon[26311]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 140 op/s 2024-04-26T04:54:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:03 smithi078 ceph-mon[33746]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 140 op/s 2024-04-26T04:54:05.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: pgmap v52: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-04-26T04:54:05.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:05.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:54:05.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:54:05.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:54:05.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:05 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:06.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: pgmap v52: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-04-26T04:54:06.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:06.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:54:06.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:54:06.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:54:06.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:05 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:06.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:06 smithi077 ceph-mon[26311]: Upgrade: Updating mgr.smithi077.hkvsor 2024-04-26T04:54:06.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:06 smithi077 ceph-mon[26311]: Deploying daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:54:07.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:06 smithi078 ceph-mon[33746]: Upgrade: Updating mgr.smithi077.hkvsor 2024-04-26T04:54:07.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:06 smithi078 ceph-mon[33746]: Deploying daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:54:07.900 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:07 smithi077 ceph-mon[26311]: pgmap v53: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-04-26T04:54:08.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:07 smithi078 ceph-mon[33746]: pgmap v53: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-04-26T04:54:09.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:09 smithi077 ceph-mon[26311]: pgmap v54: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-04-26T04:54:10.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:09 smithi078 ceph-mon[33746]: pgmap v54: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-04-26T04:54:11.904 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:11 smithi077 ceph-mon[26311]: pgmap v55: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-04-26T04:54:11.905 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:11 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:12.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:11 smithi078 ceph-mon[33746]: pgmap v55: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-04-26T04:54:12.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:11 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:12.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 -- 172.21.15.77:0/1392297787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 msgr2=0x7f09f80ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:12.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 --2- 172.21.15.77:0/1392297787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80ffa30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f09e8004940 tx=0x7f09e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:12.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 -- 172.21.15.77:0/1392297787 shutdown_connections 2024-04-26T04:54:12.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 --2- 172.21.15.77:0/1392297787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 --2- 172.21.15.77:0/1392297787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80ffa30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 -- 172.21.15.77:0/1392297787 >> 172.21.15.77:0/1392297787 conn(0x7f09f80f7420 msgr2=0x7f09f80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 -- 172.21.15.77:0/1392297787 shutdown_connections 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.874+0000 7f09fe547700 1 -- 172.21.15.77:0/1392297787 wait complete. 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 Processor -- start 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 -- start start 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f80fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f8111660 con 0x7f09f8066ad0 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09fe547700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f81117a0 con 0x7f09f80fff70 2024-04-26T04:54:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55226/0 (socket says 172.21.15.77:55226) 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09f7fff700 1 -- 172.21.15.77:0/3443461937 learned_addr learned my addr 172.21.15.77:0/3443461937 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.875+0000 7f09f77fe700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f80fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f77fe700 1 -- 172.21.15.77:0/3443461937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 msgr2=0x7f09f80fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f77fe700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f77fe700 1 -- 172.21.15.77:0/3443461937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8005040 con 0x7f09f80fff70 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f77fe700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f80fb930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009fd0 tx=0x7f09ec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09ec004e90 con 0x7f09f80fff70 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09ec00f040 con 0x7f09f80fff70 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09ec00ca50 con 0x7f09f80fff70 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f80fbed0 con 0x7f09f80fff70 2024-04-26T04:54:12.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.876+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f80fc3c0 con 0x7f09f80fff70 2024-04-26T04:54:12.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.877+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f805f5c0 con 0x7f09f80fff70 2024-04-26T04:54:12.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.878+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f09ec00cc00 con 0x7f09f80fff70 2024-04-26T04:54:12.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.878+0000 7f09f4ff9700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 0x7f09e006db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:12.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.878+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f09ec089be0 con 0x7f09f80fff70 2024-04-26T04:54:12.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.878+0000 7f09f7fff700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 0x7f09e006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:12.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.879+0000 7f09f7fff700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 0x7f09e006db20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b0d0 tx=0x7f09e8009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:12.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:12.883+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f09ec050100 con 0x7f09f80fff70 2024-04-26T04:54:13.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.210+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09f80fc6d0 con 0x7f09e006b670 2024-04-26T04:54:13.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.211+0000 7f09f4ff9700 1 -- 172.21.15.77:0/3443461937 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f09f80fc6d0 con 0x7f09e006b670 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.214+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 msgr2=0x7f09e006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.214+0000 7f09fe547700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 0x7f09e006db20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b0d0 tx=0x7f09e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.214+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 msgr2=0x7f09f80fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.214+0000 7f09fe547700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f80fb930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009fd0 tx=0x7f09ec00b040 comp rx=0 tx=0).stop 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 shutdown_connections 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f09e006b670 0x7f09e006db20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09f8066ad0 0x7f09f80fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 --2- 172.21.15.77:0/3443461937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f80fff70 0x7f09f80fb930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 >> 172.21.15.77:0/3443461937 conn(0x7f09f80f7420 msgr2=0x7f09f80f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:13.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 shutdown_connections 2024-04-26T04:54:13.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.215+0000 7f09fe547700 1 -- 172.21.15.77:0/3443461937 wait complete. 2024-04-26T04:54:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:54:13.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 -- 172.21.15.77:0/460651351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 msgr2=0x7fd3900fc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 --2- 172.21.15.77:0/460651351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3900fc520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd388009ce0 tx=0x7fd388004d10 comp rx=0 tx=0).stop 2024-04-26T04:54:13.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 -- 172.21.15.77:0/460651351 shutdown_connections 2024-04-26T04:54:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 --2- 172.21.15.77:0/460651351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd3900fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 --2- 172.21.15.77:0/460651351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3900fc520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.382+0000 7fd396844700 1 -- 172.21.15.77:0/460651351 >> 172.21.15.77:0/460651351 conn(0x7fd3900f6b80 msgr2=0x7fd3900f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.383+0000 7fd396844700 1 -- 172.21.15.77:0/460651351 shutdown_connections 2024-04-26T04:54:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.383+0000 7fd396844700 1 -- 172.21.15.77:0/460651351 wait complete. 2024-04-26T04:54:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.384+0000 7fd396844700 1 Processor -- start 2024-04-26T04:54:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.385+0000 7fd396844700 1 -- start start 2024-04-26T04:54:13.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd396844700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3901141d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd396844700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd390114710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd396844700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390114d10 con 0x7fd3900fca60 2024-04-26T04:54:13.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd396844700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390115560 con 0x7fd3900fc110 2024-04-26T04:54:13.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3901141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd390114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3901141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:33474/0 (socket says 172.21.15.77:33474) 2024-04-26T04:54:13.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.386+0000 7fd38ffff700 1 -- 172.21.15.77:0/2582768044 learned_addr learned my addr 172.21.15.77:0/2582768044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.387+0000 7fd38f7fe700 1 -- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 msgr2=0x7fd3901141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.387+0000 7fd38f7fe700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3901141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.387+0000 7fd38f7fe700 1 -- 172.21.15.77:0/2582768044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388009940 con 0x7fd3900fca60 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.387+0000 7fd38f7fe700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd390114710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd384002820 tx=0x7fd384004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.388+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd384007cc0 con 0x7fd3900fca60 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.388+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd38400f040 con 0x7fd3900fca60 2024-04-26T04:54:13.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.388+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3840094b0 con 0x7fd3900fca60 2024-04-26T04:54:13.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.390+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd390115840 con 0x7fd3900fca60 2024-04-26T04:54:13.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.390+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390115d50 con 0x7fd3900fca60 2024-04-26T04:54:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.393+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39005f5c0 con 0x7fd3900fca60 2024-04-26T04:54:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.393+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7fd384009610 con 0x7fd3900fca60 2024-04-26T04:54:13.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.394+0000 7fd38cff9700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 0x7fd37806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.394+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd3840897f0 con 0x7fd3900fca60 2024-04-26T04:54:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.394+0000 7fd38ffff700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 0x7fd37806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.396+0000 7fd38ffff700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 0x7fd37806dbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd38800b740 tx=0x7fd388004d10 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.404+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd38404f100 con 0x7fd3900fca60 2024-04-26T04:54:13.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.729+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3901160d0 con 0x7fd37806b730 2024-04-26T04:54:13.731 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:13 smithi077 ceph-mon[26311]: pgmap v56: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.4 MiB/s wr, 137 op/s 2024-04-26T04:54:13.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.730+0000 7fd38cff9700 1 -- 172.21.15.77:0/2582768044 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fd3901160d0 con 0x7fd37806b730 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 msgr2=0x7fd37806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 0x7fd37806dbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd38800b740 tx=0x7fd388004d10 comp rx=0 tx=0).stop 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 msgr2=0x7fd390114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd390114710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd384002820 tx=0x7fd384004570 comp rx=0 tx=0).stop 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 shutdown_connections 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fd37806b730 0x7fd37806dbe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3900fc110 0x7fd3901141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 --2- 172.21.15.77:0/2582768044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900fca60 0x7fd390114710 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 >> 172.21.15.77:0/2582768044 conn(0x7fd3900f6b80 msgr2=0x7fd3900f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:13.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 shutdown_connections 2024-04-26T04:54:13.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.733+0000 7fd396844700 1 -- 172.21.15.77:0/2582768044 wait complete. 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 -- 172.21.15.77:0/882149925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 msgr2=0x7fc2fc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 --2- 172.21.15.77:0/882149925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc0fd3d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec004940 tx=0x7fc2ec009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 -- 172.21.15.77:0/882149925 shutdown_connections 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 --2- 172.21.15.77:0/882149925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc0fd3d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 --2- 172.21.15.77:0/882149925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 -- 172.21.15.77:0/882149925 >> 172.21.15.77:0/882149925 conn(0x7fc2fc0f7430 msgr2=0x7fc2fc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 -- 172.21.15.77:0/882149925 shutdown_connections 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.868+0000 7fc303861700 1 -- 172.21.15.77:0/882149925 wait complete. 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 Processor -- start 2024-04-26T04:54:13.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 -- start start 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc110770 con 0x7fc2fc0fcf60 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc303861700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc110f70 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc300dfc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc3015fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc300dfc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55256/0 (socket says 172.21.15.77:55256) 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc3015fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:33496/0 (socket says 172.21.15.77:33496) 2024-04-26T04:54:13.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.869+0000 7fc300dfc700 1 -- 172.21.15.77:0/3849877161 learned_addr learned my addr 172.21.15.77:0/3849877161 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc3015fd700 1 -- 172.21.15.77:0/3849877161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 msgr2=0x7fc2fc110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc3015fd700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc3015fd700 1 -- 172.21.15.77:0/3849877161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec005040 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc3015fd700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009fd0 tx=0x7fc2f800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f8004e90 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2f800f040 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc1111f0 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f800ca50 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.870+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1116e0 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.871+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc1018f0 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.872+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7fc2f800cc00 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.872+0000 7fc2f1ffb700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 0x7fc2e806db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:13.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.872+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc2f8089c60 con 0x7fc2fc0fbcb0 2024-04-26T04:54:13.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.872+0000 7fc300dfc700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 0x7fc2e806db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:13.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.873+0000 7fc300dfc700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 0x7fc2e806db80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fd0 tx=0x7fc2ec009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:13.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:13.877+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fc2f8050100 con 0x7fc2fc0fbcb0 2024-04-26T04:54:14.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:13 smithi078 ceph-mon[33746]: pgmap v56: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.4 MiB/s wr, 137 op/s 2024-04-26T04:54:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.198+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2fc111950 con 0x7fc2e806b6d0 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.208+0000 7fc2f1ffb700 1 -- 172.21.15.77:0/3849877161 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fc2fc111950 con 0x7fc2e806b6d0 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 starting - - - - 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (9m) 95s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (8m) 98s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 starting - - - - 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (4m) 95s ago 4m 177M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (4m) 95s ago 4m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:54:14.210 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (4m) 98s ago 4m 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (4m) 98s ago 4m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:9283 running (11m) 95s ago 10m 344M - 17.2.7-902-gc61472c8 0de407c1b3af 312fba87d35b 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (2m) 98s ago 8m 490M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 70ff04585016 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (11m) 95s ago 11m 49.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (8m) 98s ago 8m 38.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 starting - - - - 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 starting - - - - 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (7m) 95s ago 7m 78.1M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (7m) 95s ago 7m 85.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (7m) 95s ago 7m 76.9M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (6m) 98s ago 6m 99.3M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (6m) 98s ago 6m 89.1M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (5m) 98s ago 5m 85.9M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:54:14.211 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 starting - - - - 2024-04-26T04:54:14.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 msgr2=0x7fc2e806db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 0x7fc2e806db80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fd0 tx=0x7fc2ec009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:14.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 msgr2=0x7fc2fc10fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009fd0 tx=0x7fc2f800b040 comp rx=0 tx=0).stop 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 shutdown_connections 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fc2e806b6d0 0x7fc2e806db80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2fc0fbcb0 0x7fc2fc10fc30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 --2- 172.21.15.77:0/3849877161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2fc0fcf60 0x7fc2fc110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 >> 172.21.15.77:0/3849877161 conn(0x7fc2fc0f7430 msgr2=0x7fc2fc065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 shutdown_connections 2024-04-26T04:54:14.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.212+0000 7fc303861700 1 -- 172.21.15.77:0/3849877161 wait complete. 2024-04-26T04:54:14.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 -- 172.21.15.77:0/457646634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 msgr2=0x7ff5400fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 --2- 172.21.15.77:0/457646634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff5400fd270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff534004940 tx=0x7ff534009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 -- 172.21.15.77:0/457646634 shutdown_connections 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 --2- 172.21.15.77:0/457646634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff5400fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 --2- 172.21.15.77:0/457646634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff5400fd270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 -- 172.21.15.77:0/457646634 >> 172.21.15.77:0/457646634 conn(0x7ff5400f8540 msgr2=0x7ff5400fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 -- 172.21.15.77:0/457646634 shutdown_connections 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.347+0000 7ff546bbe700 1 -- 172.21.15.77:0/457646634 wait complete. 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.348+0000 7ff546bbe700 1 Processor -- start 2024-04-26T04:54:14.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.348+0000 7ff546bbe700 1 -- start start 2024-04-26T04:54:14.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.348+0000 7ff546bbe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff54010d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff546bbe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff546bbe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010e420 con 0x7ff5400fce60 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff546bbe700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010e590 con 0x7ff5400fdfd0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff54495a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff54010d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:33516/0 (socket says 172.21.15.77:33516) 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 -- 172.21.15.77:0/381795657 learned_addr learned my addr 172.21.15.77:0/381795657 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff54495a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff54010d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55276/0 (socket says 172.21.15.77:55276) 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 -- 172.21.15.77:0/381795657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 msgr2=0x7ff54010d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff54010d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 -- 172.21.15.77:0/381795657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534005040 con 0x7ff5400fdfd0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.349+0000 7ff53ffff700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00eb40 tx=0x7ff52c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.350+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c007cc0 con 0x7ff5400fdfd0 2024-04-26T04:54:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.350+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff52c00f040 con 0x7ff5400fdfd0 2024-04-26T04:54:14.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.350+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c009620 con 0x7ff5400fdfd0 2024-04-26T04:54:14.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.350+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540070190 con 0x7ff5400fdfd0 2024-04-26T04:54:14.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.350+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540070750 con 0x7ff5400fdfd0 2024-04-26T04:54:14.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.351+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7ff52c009780 con 0x7ff5400fdfd0 2024-04-26T04:54:14.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.352+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54004c710 con 0x7ff5400fdfd0 2024-04-26T04:54:14.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.352+0000 7ff53d7fa700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 0x7ff53006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.352+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff52c089db0 con 0x7ff5400fdfd0 2024-04-26T04:54:14.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.352+0000 7ff54495a700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 0x7ff53006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.353+0000 7ff54495a700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 0x7ff53006dbe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b0d0 tx=0x7ff534009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:14.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.357+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff52c050100 con 0x7ff5400fdfd0 2024-04-26T04:54:14.675 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:14 smithi077 ceph-mon[26311]: from='client.24501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:14.675 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:14 smithi077 ceph-mon[26311]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:14.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.819+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff540070ae0 con 0x7ff5400fdfd0 2024-04-26T04:54:14.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.820+0000 7ff53d7fa700 1 -- 172.21.15.77:0/381795657 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7ff52c057720 con 0x7ff5400fdfd0 2024-04-26T04:54:14.821 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:54:14.821 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:54:14.821 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:54:14.822 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-26T04:54:14.823 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:54:14.823 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:54:14.823 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 msgr2=0x7ff53006dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 0x7ff53006dbe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff53400b0d0 tx=0x7ff534009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 msgr2=0x7ff54010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00eb40 tx=0x7ff52c00b040 comp rx=0 tx=0).stop 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 shutdown_connections 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7ff53006b730 0x7ff53006dbe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5400fce60 0x7ff54010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 --2- 172.21.15.77:0/381795657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5400fdfd0 0x7ff54010de20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 >> 172.21.15.77:0/381795657 conn(0x7ff5400f8540 msgr2=0x7ff5400f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:14.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 shutdown_connections 2024-04-26T04:54:14.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.822+0000 7ff546bbe700 1 -- 172.21.15.77:0/381795657 wait complete. 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.960+0000 7f643b0d5700 1 -- 172.21.15.77:0/1360579626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 msgr2=0x7f64340fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.960+0000 7f643b0d5700 1 --2- 172.21.15.77:0/1360579626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f64340fc0c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6428004940 tx=0x7f6428009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 -- 172.21.15.77:0/1360579626 shutdown_connections 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 --2- 172.21.15.77:0/1360579626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 0x7f64340fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 --2- 172.21.15.77:0/1360579626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f64340fc0c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 -- 172.21.15.77:0/1360579626 >> 172.21.15.77:0/1360579626 conn(0x7f64340f7430 msgr2=0x7f64340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 -- 172.21.15.77:0/1360579626 shutdown_connections 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 -- 172.21.15.77:0/1360579626 wait complete. 2024-04-26T04:54:14.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 Processor -- start 2024-04-26T04:54:14.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.961+0000 7f643b0d5700 1 -- start start 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f643b0d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f643b0d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 0x7f643410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f643b0d5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643410e620 con 0x7f64340fbcb0 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f643b0d5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643410edd0 con 0x7f64340fcf60 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55286/0 (socket says 172.21.15.77:55286) 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 -- 172.21.15.77:0/2977465801 learned_addr learned my addr 172.21.15.77:0/2977465801 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6433fff700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 0x7f643410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 -- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 msgr2=0x7f643410e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 0x7f643410e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.962+0000 7f6438e71700 1 -- 172.21.15.77:0/2977465801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428005040 con 0x7f64340fbcb0 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f6438e71700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6428009150 tx=0x7f6428009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:14.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642800e040 con 0x7f64340fbcb0 2024-04-26T04:54:14.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643410f050 con 0x7f64340fbcb0 2024-04-26T04:54:14.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642800bbf0 con 0x7f64340fbcb0 2024-04-26T04:54:14.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642801c630 con 0x7f64340fbcb0 2024-04-26T04:54:14.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.963+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643410f530 con 0x7f64340fbcb0 2024-04-26T04:54:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.964+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643405f5c0 con 0x7f64340fbcb0 2024-04-26T04:54:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.965+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f642800bd60 con 0x7f64340fbcb0 2024-04-26T04:54:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.965+0000 7f64317fa700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 0x7f642406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.965+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f642808b4d0 con 0x7f64340fbcb0 2024-04-26T04:54:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.965+0000 7f6433fff700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 0x7f642406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:14.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.966+0000 7f6433fff700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 0x7f642406db30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64200041d0 tx=0x7f64200043c0 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:14.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:14.973+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6428058e60 con 0x7f64340fbcb0 2024-04-26T04:54:15.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:14 smithi078 ceph-mon[33746]: from='client.24501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:15.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:14 smithi078 ceph-mon[33746]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.330+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f643402b4a0 con 0x7f64340fbcb0 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.331+0000 7f64317fa700 1 -- 172.21.15.77:0/2977465801 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f6428013e60 con 0x7f64340fbcb0 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:54:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:54:15.334 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:15.335 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 msgr2=0x7f642406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 0x7f642406db30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64200041d0 tx=0x7f64200043c0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 msgr2=0x7f643410dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6428009150 tx=0x7f6428009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 shutdown_connections 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f642406b680 0x7f642406db30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64340fbcb0 0x7f643410dae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 --2- 172.21.15.77:0/2977465801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64340fcf60 0x7f643410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 >> 172.21.15.77:0/2977465801 conn(0x7f64340f7430 msgr2=0x7f6434065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:15.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 shutdown_connections 2024-04-26T04:54:15.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.334+0000 7f643b0d5700 1 -- 172.21.15.77:0/2977465801 wait complete. 2024-04-26T04:54:15.337 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 -- 172.21.15.77:0/136106253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fce60 msgr2=0x7f36880fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 --2- 172.21.15.77:0/136106253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fce60 0x7f36880fd270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3684004940 tx=0x7f3684009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 -- 172.21.15.77:0/136106253 shutdown_connections 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 --2- 172.21.15.77:0/136106253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fdfd0 0x7f36880fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 --2- 172.21.15.77:0/136106253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fce60 0x7f36880fd270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 -- 172.21.15.77:0/136106253 >> 172.21.15.77:0/136106253 conn(0x7f36880f8540 msgr2=0x7f36880fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 -- 172.21.15.77:0/136106253 shutdown_connections 2024-04-26T04:54:15.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.465+0000 7f368f4b8700 1 -- 172.21.15.77:0/136106253 wait complete. 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.466+0000 7f368f4b8700 1 Processor -- start 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.466+0000 7f368f4b8700 1 -- start start 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.466+0000 7f368f4b8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 0x7f368810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368f4b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 0x7f368810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368f4b8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368810e4d0 con 0x7f36880fdfd0 2024-04-26T04:54:15.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368f4b8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368810ed20 con 0x7f36880fce60 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368d254700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 0x7f368810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368d254700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 0x7f368810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:33576/0 (socket says 172.21.15.77:33576) 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368d254700 1 -- 172.21.15.77:0/3521042256 learned_addr learned my addr 172.21.15.77:0/3521042256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368ca53700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 0x7f368810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368ca53700 1 -- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 msgr2=0x7f368810d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368ca53700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 0x7f368810d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.467+0000 7f368ca53700 1 -- 172.21.15.77:0/3521042256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684005040 con 0x7f36880fdfd0 2024-04-26T04:54:15.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.468+0000 7f368ca53700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 0x7f368810ded0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3678009fd0 tx=0x7f367800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:15.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.468+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3678007cc0 con 0x7f36880fdfd0 2024-04-26T04:54:15.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.468+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f367800f040 con 0x7f36880fdfd0 2024-04-26T04:54:15.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.468+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367800cad0 con 0x7f36880fdfd0 2024-04-26T04:54:15.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.469+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368810efa0 con 0x7f36880fdfd0 2024-04-26T04:54:15.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.469+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368810f4e0 con 0x7f36880fdfd0 2024-04-26T04:54:15.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.470+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368805f5c0 con 0x7f36880fdfd0 2024-04-26T04:54:15.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.471+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f367800cc30 con 0x7f36880fdfd0 2024-04-26T04:54:15.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.471+0000 7f367dffb700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 0x7f367406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:15.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.472+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3678089b50 con 0x7f36880fdfd0 2024-04-26T04:54:15.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.472+0000 7f368d254700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 0x7f367406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:15.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.472+0000 7f368d254700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 0x7f367406dbf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f368400b0d0 tx=0x7f3684009e30 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:15.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.477+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3678057410 con 0x7f36880fdfd0 2024-04-26T04:54:15.622 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: pgmap v57: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/381795657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:15.623 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:15 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2977465801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:15.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.933+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 --> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f368810f860 con 0x7f367406b740 2024-04-26T04:54:15.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.935+0000 7f367dffb700 1 -- 172.21.15.77:0/3521042256 <== mgr.14634 v2:172.21.15.78:6828/1876826264 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f368810f860 con 0x7f367406b740 2024-04-26T04:54:15.936 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:54:15.936 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "", 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mgr daemons", 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:54:15.937 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:54:15.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.937+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 msgr2=0x7f367406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.937+0000 7f368f4b8700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 0x7f367406dbf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f368400b0d0 tx=0x7f3684009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:15.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 msgr2=0x7f368810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:15.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 0x7f368810ded0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3678009fd0 tx=0x7f367800b040 comp rx=0 tx=0).stop 2024-04-26T04:54:15.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 shutdown_connections 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7f367406b740 0x7f367406dbf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36880fce60 0x7f368810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 --2- 172.21.15.77:0/3521042256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36880fdfd0 0x7f368810ded0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 >> 172.21.15.77:0/3521042256 conn(0x7f36880f8540 msgr2=0x7f36880f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 shutdown_connections 2024-04-26T04:54:15.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:15.938+0000 7f368f4b8700 1 -- 172.21.15.77:0/3521042256 wait complete. 2024-04-26T04:54:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: pgmap v57: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-04-26T04:54:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/381795657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:16.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:16.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:16.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:15 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2977465801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:16.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/422280765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 msgr2=0x7fe8c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:16.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/422280765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c80fd3d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc004940 tx=0x7fe8bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:16.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/422280765 shutdown_connections 2024-04-26T04:54:16.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/422280765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c80fd3d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/422280765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 0x7fe8c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/422280765 >> 172.21.15.77:0/422280765 conn(0x7fe8c80f7430 msgr2=0x7fe8c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:16.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/422280765 shutdown_connections 2024-04-26T04:54:16.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.086+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/422280765 wait complete. 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.087+0000 7fe8cd5ed700 1 Processor -- start 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.087+0000 7fe8cd5ed700 1 -- start start 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.087+0000 7fe8cd5ed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 0x7fe8c810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8cd5ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8cd5ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c810d3e0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8cd5ed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c810dc30 con 0x7fe8c80fbcb0 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:16.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55338/0 (socket says 172.21.15.77:55338) 2024-04-26T04:54:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 -- 172.21.15.77:0/4114826919 learned_addr learned my addr 172.21.15.77:0/4114826919 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c6ffd700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 0x7fe8c810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 -- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 msgr2=0x7fe8c810c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 0x7fe8c810c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 -- 172.21.15.77:0/4114826919 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc005040 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.088+0000 7fe8c67fc700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.089+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8bc00e070 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.089+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8bc013d00 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.089+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8bc023650 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.089+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c810deb0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.089+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c810e2d0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.090+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c805f5c0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.091+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) v1 ==== 89277+0+0 (secure 0 0 0) 0x7fe8bc01aa50 con 0x7fe8c80fcf60 2024-04-26T04:54:16.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.092+0000 7fe8af7fe700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 0x7fe8b006dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:16.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.092+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe8bc08dde0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.092+0000 7fe8c6ffd700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 0x7fe8b006dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.093+0000 7fe8c6ffd700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 0x7fe8b006dcc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009b80 tx=0x7fe8b800b040 comp rx=0 tx=0).ready entity=mgr.14634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:16.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.100+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe8bc05b6a0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.516+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8c804bed0 con 0x7fe8c80fcf60 2024-04-26T04:54:16.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.518+0000 7fe8af7fe700 1 -- 172.21.15.77:0/4114826919 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe8bc004490 con 0x7fe8c80fcf60 2024-04-26T04:54:16.520 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:54:16.520 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:54:16.520 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:54:16.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.522+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 msgr2=0x7fe8b006dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.522+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 0x7fe8b006dcc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009b80 tx=0x7fe8b800b040 comp rx=0 tx=0).stop 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.522+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 msgr2=0x7fe8c810cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.522+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc000c00 tx=0x7fe8bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 shutdown_connections 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:6828/1876826264,v1:172.21.15.78:6829/1876826264] conn(0x7fe8b006b810 0x7fe8b006dcc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c80fbcb0 0x7fe8c810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 --2- 172.21.15.77:0/4114826919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c80fcf60 0x7fe8c810cde0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 >> 172.21.15.77:0/4114826919 conn(0x7fe8c80f7430 msgr2=0x7fe8c80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 shutdown_connections 2024-04-26T04:54:16.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:16.523+0000 7fe8cd5ed700 1 -- 172.21.15.77:0/4114826919 wait complete. 2024-04-26T04:54:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:17 smithi078 ceph-mon[33746]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:17 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:17 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:17 smithi078 ceph-mon[33746]: pgmap v58: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-04-26T04:54:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:17 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/4114826919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:54:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:17 smithi077 ceph-mon[26311]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:17 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:17 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:17 smithi077 ceph-mon[26311]: pgmap v58: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-04-26T04:54:17.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:17 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/4114826919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:54:19.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:19 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:19.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:19 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:19 smithi077 ceph-mon[26311]: pgmap v59: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.3 MiB/s wr, 145 op/s 2024-04-26T04:54:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:19 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:19 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:19.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:19 smithi078 ceph-mon[33746]: pgmap v59: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.3 MiB/s wr, 145 op/s 2024-04-26T04:54:21.911 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:21 smithi077 ceph-mon[26311]: pgmap v60: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-04-26T04:54:22.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:21 smithi078 ceph-mon[33746]: pgmap v60: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-04-26T04:54:23.831 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:23 smithi077 ceph-mon[26311]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-04-26T04:54:24.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:23 smithi078 ceph-mon[33746]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: Standby manager daemon smithi077.hkvsor restarted 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: Standby manager daemon smithi077.hkvsor started 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/crt"}]: dispatch 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/key"}]: dispatch 2024-04-26T04:54:24.828 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:24 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:54:25.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: Standby manager daemon smithi077.hkvsor restarted 2024-04-26T04:54:25.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: Standby manager daemon smithi077.hkvsor started 2024-04-26T04:54:25.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/crt"}]: dispatch 2024-04-26T04:54:25.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:54:25.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi077.hkvsor/key"}]: dispatch 2024-04-26T04:54:25.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:24 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:54:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:25 smithi077 ceph-mon[26311]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-04-26T04:54:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:25 smithi077 ceph-mon[26311]: mgrmap e27: smithi078.navvuw(active, since 118s), standbys: smithi077.hkvsor 2024-04-26T04:54:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:25 smithi078 ceph-mon[33746]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-04-26T04:54:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:25 smithi078 ceph-mon[33746]: mgrmap e27: smithi078.navvuw(active, since 118s), standbys: smithi077.hkvsor 2024-04-26T04:54:26.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:26 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:27.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:26 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:27.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:27 smithi077 ceph-mon[26311]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-04-26T04:54:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:27 smithi078 ceph-mon[33746]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-04-26T04:54:29.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:29 smithi077 ceph-mon[26311]: pgmap v64: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.2 MiB/s wr, 382 op/s 2024-04-26T04:54:29.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:29 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:29.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:29 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:29 smithi078 ceph-mon[33746]: pgmap v64: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.2 MiB/s wr, 382 op/s 2024-04-26T04:54:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:29 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:30.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:29 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:31 smithi077 ceph-mon[26311]: pgmap v65: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 330 op/s 2024-04-26T04:54:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:31 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:31 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:31 smithi078 ceph-mon[33746]: pgmap v65: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 330 op/s 2024-04-26T04:54:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:31 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:31 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:33.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:33 smithi077 ceph-mon[26311]: pgmap v66: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-04-26T04:54:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:33 smithi078 ceph-mon[33746]: pgmap v66: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-04-26T04:54:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:35 smithi077 ceph-mon[26311]: pgmap v67: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T04:54:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:35 smithi078 ceph-mon[33746]: pgmap v67: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T04:54:37.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:37 smithi077 ceph-mon[26311]: pgmap v68: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T04:54:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:37 smithi078 ceph-mon[33746]: pgmap v68: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:39.618 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd='[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]': finished 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: mgrmap e28: smithi077.hkvsor(active, starting, since 0.0062396s) 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:54:39.619 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:54:39.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:39 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:54:39.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 172.21.15.78:0/866763517' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14634 ' entity='mgr.smithi078.navvuw' cmd='[{"prefix": "mgr fail", "who": "smithi078.navvuw"}]': finished 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: mgrmap e28: smithi077.hkvsor(active, starting, since 0.0062396s) 2024-04-26T04:54:39.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:54:39.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:54:39.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:39 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: mgrmap e29: smithi077.hkvsor(active, since 1.01201s) 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: [26/Apr/2024:04:54:39] ENGINE Bus STARTING 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: [26/Apr/2024:04:54:39] ENGINE Serving on http://172.21.15.77:8765 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: [26/Apr/2024:04:54:39] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:54:40.720 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:40 smithi077 ceph-mon[26311]: [26/Apr/2024:04:54:39] ENGINE Bus STARTED 2024-04-26T04:54:40.748 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:54:40.749 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: mgrmap e29: smithi077.hkvsor(active, since 1.01201s) 2024-04-26T04:54:40.749 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: [26/Apr/2024:04:54:39] ENGINE Bus STARTING 2024-04-26T04:54:40.749 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: [26/Apr/2024:04:54:39] ENGINE Serving on http://172.21.15.77:8765 2024-04-26T04:54:40.749 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: [26/Apr/2024:04:54:39] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:54:40.749 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:40 smithi078 ceph-mon[33746]: [26/Apr/2024:04:54:39] ENGINE Bus STARTED 2024-04-26T04:54:41.560 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:41 smithi077 ceph-mon[26311]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T04:54:41.561 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:41 smithi077 ceph-mon[26311]: pgmap v4: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:54:41.561 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:41 smithi077 ceph-mon[26311]: mgrmap e30: smithi077.hkvsor(active, since 2s) 2024-04-26T04:54:41.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:41 smithi078 ceph-mon[33746]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T04:54:41.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:41 smithi078 ceph-mon[33746]: pgmap v4: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:54:41.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:41 smithi078 ceph-mon[33746]: mgrmap e30: smithi077.hkvsor(active, since 2s) 2024-04-26T04:54:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:43 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:43 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:43 smithi078 ceph-mon[33746]: pgmap v5: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:54:43.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:43 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:43.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:43 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:43.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:43 smithi077 ceph-mon[26311]: pgmap v5: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:54:44.130 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-26T04:54:44.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-26T04:54:44.431 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:44 smithi078 ceph-mon[33746]: mgrmap e31: smithi077.hkvsor(active, since 4s) 2024-04-26T04:54:44.431 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:44 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:44.431 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:44 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:44.432 DEBUG:teuthology.parallel:result is None 2024-04-26T04:54:44.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:44 smithi077 ceph-mon[26311]: mgrmap e31: smithi077.hkvsor(active, since 4s) 2024-04-26T04:54:44.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:44 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:44.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:44 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:45.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:45 smithi077 ceph-mon[26311]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 615 B/s wr, 51 op/s 2024-04-26T04:54:45.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:45 smithi078 ceph-mon[33746]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 615 B/s wr, 51 op/s 2024-04-26T04:54:46.379 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:46 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:54:46.380 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:46 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:54:46.380 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:46 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:54:46.380 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:46 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:54:46.678 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:46 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:54:46.678 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:46 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:54:46.679 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:46 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:54:46.679 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:46 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:54:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 -- 172.21.15.77:0/771311536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06ed90 msgr2=0x7fb16c108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 --2- 172.21.15.77:0/771311536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06ed90 0x7fb16c108f80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009ce0 tx=0x7fb15c004d10 comp rx=0 tx=0).stop 2024-04-26T04:54:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 -- 172.21.15.77:0/771311536 shutdown_connections 2024-04-26T04:54:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 --2- 172.21.15.77:0/771311536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06ed90 0x7fb16c108f80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 --2- 172.21.15.77:0/771311536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06e440 0x7fb16c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.676+0000 7fb17230f700 1 -- 172.21.15.77:0/771311536 >> 172.21.15.77:0/771311536 conn(0x7fb16c069dc0 msgr2=0x7fb16c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.677+0000 7fb17230f700 1 -- 172.21.15.77:0/771311536 shutdown_connections 2024-04-26T04:54:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.677+0000 7fb17230f700 1 -- 172.21.15.77:0/771311536 wait complete. 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.677+0000 7fb17230f700 1 Processor -- start 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.677+0000 7fb17230f700 1 -- start start 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb17230f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06e440 0x7fb16c11c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb17230f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb17230f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c11d440 con 0x7fb16c06e440 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb17230f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c11d5b0 con 0x7fb16c06ed90 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb16b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb16b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:49222/0 (socket says 172.21.15.77:49222) 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb16b7fe700 1 -- 172.21.15.77:0/1332632147 learned_addr learned my addr 172.21.15.77:0/1332632147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.678+0000 7fb16bfff700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06e440 0x7fb16c11c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.679+0000 7fb16b7fe700 1 -- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06e440 msgr2=0x7fb16c11c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.679+0000 7fb16b7fe700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06e440 0x7fb16c11c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.679+0000 7fb16b7fe700 1 -- 172.21.15.77:0/1332632147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c009940 con 0x7fb16c06ed90 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.679+0000 7fb16b7fe700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009380 tx=0x7fb15c004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.680+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb15c00e040 con 0x7fb16c06ed90 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.680+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb15c008e20 con 0x7fb16c06ed90 2024-04-26T04:54:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.680+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb15c01c750 con 0x7fb16c06ed90 2024-04-26T04:54:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.680+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c0742f0 con 0x7fb16c06ed90 2024-04-26T04:54:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.680+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c0748b0 con 0x7fb16c06ed90 2024-04-26T04:54:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.681+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c04c710 con 0x7fb16c06ed90 2024-04-26T04:54:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.683+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fb15c012b00 con 0x7fb16c06ed90 2024-04-26T04:54:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.683+0000 7fb168ff9700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 0x7fb1580701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.683+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fb15c08f2c0 con 0x7fb16c06ed90 2024-04-26T04:54:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.684+0000 7fb16bfff700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 0x7fb1580701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.685+0000 7fb16bfff700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 0x7fb1580701f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb16c11ddd0 tx=0x7fb16400b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.691+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb15c058d30 con 0x7fb16c06ed90 2024-04-26T04:54:46.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.993+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb16c1087e0 con 0x7fb15806dd40 2024-04-26T04:54:46.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.995+0000 7fb168ff9700 1 -- 172.21.15.77:0/1332632147 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb16c1087e0 con 0x7fb15806dd40 2024-04-26T04:54:46.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.997+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 msgr2=0x7fb1580701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.997+0000 7fb17230f700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 0x7fb1580701f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb16c11ddd0 tx=0x7fb16400b040 comp rx=0 tx=0).stop 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.997+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 msgr2=0x7fb16c11cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.997+0000 7fb17230f700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009380 tx=0x7fb15c004d10 comp rx=0 tx=0).stop 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 shutdown_connections 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fb15806dd40 0x7fb1580701f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb16c06e440 0x7fb16c11c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 --2- 172.21.15.77:0/1332632147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c06ed90 0x7fb16c11cdf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 >> 172.21.15.77:0/1332632147 conn(0x7fb16c069dc0 msgr2=0x7fb16c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 shutdown_connections 2024-04-26T04:54:47.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:46.998+0000 7fb17230f700 1 -- 172.21.15.77:0/1332632147 wait complete. 2024-04-26T04:54:47.011 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:54:47.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- 172.21.15.77:0/1965748947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fcf60 msgr2=0x7fdd3c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 --2- 172.21.15.77:0/1965748947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c0fd3d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c004940 tx=0x7fdd2c009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:47.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- 172.21.15.77:0/1965748947 shutdown_connections 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 --2- 172.21.15.77:0/1965748947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c0fd3d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 --2- 172.21.15.77:0/1965748947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- 172.21.15.77:0/1965748947 >> 172.21.15.77:0/1965748947 conn(0x7fdd3c0f7430 msgr2=0x7fdd3c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- 172.21.15.77:0/1965748947 shutdown_connections 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- 172.21.15.77:0/1965748947 wait complete. 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 Processor -- start 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.120+0000 7fdd44419700 1 -- start start 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd44419700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd44419700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd44419700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c112bc0 con 0x7fdd3c0fbcb0 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd44419700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c112d00 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd421b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd421b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36504/0 (socket says 172.21.15.77:36504) 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd421b5700 1 -- 172.21.15.77:0/254267366 learned_addr learned my addr 172.21.15.77:0/254267366 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd419b4700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd419b4700 1 -- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 msgr2=0x7fdd3c066c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd419b4700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd419b4700 1 -- 172.21.15.77:0/254267366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c005040 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.121+0000 7fdd421b5700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd419b4700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c112680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd2c00e030 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c112f00 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd2c013de0 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd2c01ce50 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.122+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c113410 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.123+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd200052e0 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.124+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fdd2c013910 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.124+0000 7fdd32ffd700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 0x7fdd2806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.124+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fdd2c08eba0 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.125+0000 7fdd421b5700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 0x7fdd2806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.125+0000 7fdd421b5700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 0x7fdd2806ffb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdd380041d0 tx=0x7fdd3800b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.129+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdd2c05c0e0 con 0x7fdd3c0fcf60 2024-04-26T04:54:47.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.432+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd200052e0 con 0x7fdd2806db00 2024-04-26T04:54:47.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.433+0000 7fdd32ffd700 1 -- 172.21.15.77:0/254267366 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdd200052e0 con 0x7fdd2806db00 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 msgr2=0x7fdd2806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 0x7fdd2806ffb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdd380041d0 tx=0x7fdd3800b040 comp rx=0 tx=0).stop 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 msgr2=0x7fdd3c112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c112680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000c00 tx=0x7fdd2c009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 shutdown_connections 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fdd2806db00 0x7fdd2806ffb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd3c0fbcb0 0x7fdd3c066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 --2- 172.21.15.77:0/254267366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd3c0fcf60 0x7fdd3c112680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 >> 172.21.15.77:0/254267366 conn(0x7fdd3c0f7430 msgr2=0x7fdd3c0f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 shutdown_connections 2024-04-26T04:54:47.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.436+0000 7fdd44419700 1 -- 172.21.15.77:0/254267366 wait complete. 2024-04-26T04:54:47.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 -- 172.21.15.77:0/20321209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fcf60 msgr2=0x7f99c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 --2- 172.21.15.77:0/20321209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fcf60 0x7f99c80fd3d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99b8004940 tx=0x7f99b8009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:47.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 -- 172.21.15.77:0/20321209 shutdown_connections 2024-04-26T04:54:47.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 --2- 172.21.15.77:0/20321209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fcf60 0x7f99c80fd3d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 --2- 172.21.15.77:0/20321209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fbcb0 0x7f99c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 -- 172.21.15.77:0/20321209 >> 172.21.15.77:0/20321209 conn(0x7f99c80f7430 msgr2=0x7f99c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.558 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:47 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:54:47.558 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:47 smithi077 ceph-mon[26311]: mgrmap e32: smithi077.hkvsor(active, since 7s), standbys: smithi078.navvuw 2024-04-26T04:54:47.558 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:47 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:47.558 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:47 smithi077 ceph-mon[26311]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 971 KiB/s wr, 174 op/s 2024-04-26T04:54:47.559 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:47 smithi077 ceph-mon[26311]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 -- 172.21.15.77:0/20321209 shutdown_connections 2024-04-26T04:54:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 -- 172.21.15.77:0/20321209 wait complete. 2024-04-26T04:54:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.556+0000 7f99cf82b700 1 Processor -- start 2024-04-26T04:54:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cf82b700 1 -- start start 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cf82b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49034/0 (socket says 172.21.15.77:49034) 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cf82b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fcf60 0x7f99c8112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8066330 con 0x7f99c80fbcb0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 -- 172.21.15.77:0/1827440777 learned_addr learned my addr 172.21.15.77:0/1827440777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8066470 con 0x7f99c80fcf60 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99ccdc6700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fcf60 0x7f99c8112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 -- 172.21.15.77:0/1827440777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fcf60 msgr2=0x7f99c8112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fcf60 0x7f99c8112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8005040 con 0x7f99c80fbcb0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.557+0000 7f99cd5c7700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99c400eb40 tx=0x7f99c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.558+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c4007cc0 con 0x7f99c80fbcb0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.558+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99c400f040 con 0x7f99c80fbcb0 2024-04-26T04:54:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.558+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c8113150 con 0x7f99c80fbcb0 2024-04-26T04:54:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.558+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c400ca10 con 0x7f99c80fbcb0 2024-04-26T04:54:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.558+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c8113710 con 0x7f99c80fbcb0 2024-04-26T04:54:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.559+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c805f5c0 con 0x7f99c80fbcb0 2024-04-26T04:54:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.559+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f99c400cc80 con 0x7f99c80fbcb0 2024-04-26T04:54:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.560+0000 7f99bdffb700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 0x7f99b4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.560+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f99c408cfd0 con 0x7f99c80fbcb0 2024-04-26T04:54:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.560+0000 7f99ccdc6700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 0x7f99b4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.560+0000 7f99ccdc6700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 0x7f99b4070010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b0d0 tx=0x7f99b8009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.566+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f99c4056a90 con 0x7f99c80fbcb0 2024-04-26T04:54:47.581 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:47 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/737888258' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:54:47.581 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:47 smithi078 ceph-mon[33746]: mgrmap e32: smithi077.hkvsor(active, since 7s), standbys: smithi078.navvuw 2024-04-26T04:54:47.581 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:47 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:54:47.581 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:47 smithi078 ceph-mon[33746]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 971 KiB/s wr, 174 op/s 2024-04-26T04:54:47.581 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:47 smithi078 ceph-mon[33746]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:47.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.834+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c8101290 con 0x7f99b406db60 2024-04-26T04:54:47.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.844+0000 7f99bdffb700 1 -- 172.21.15.77:0/1827440777 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f99c8101290 con 0x7f99b406db60 2024-04-26T04:54:47.846 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:54:47.846 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (85s) 18s ago 10m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:54:47.846 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (10m) 18s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:54:47.846 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (9m) 5s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 18s ago 9m - - 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (4m) 18s ago 4m 232M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (4m) 18s ago 4m 15.2M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (4m) 5s ago 4m 15.6M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (4m) 5s ago 4m 15.6M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (33s) 18s ago 11m 114M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283 running (2m) 5s ago 9m 79.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 70ff04585016 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (11m) 18s ago 11m 52.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:54:47.847 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (9m) 5s ago 9m 43.6M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (114s) 18s ago 10m 16.0M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (56s) 5s ago 9m 16.6M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (8m) 18s ago 8m 181M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (8m) 18s ago 8m 192M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (7m) 18s ago 7m 161M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (7m) 5s ago 7m 227M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (6m) 5s ago 6m 207M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (6m) 5s ago 6m 185M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:54:47.848 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (60s) 18s ago 9m 42.9M - 2.43.0 a07b618ecd1d 3c1a74f581e2 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 msgr2=0x7f99b4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 0x7f99b4070010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b0d0 tx=0x7f99b8009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 msgr2=0x7f99c81106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99c400eb40 tx=0x7f99c400b040 comp rx=0 tx=0).stop 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 shutdown_connections 2024-04-26T04:54:47.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f99b406db60 0x7f99b4070010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c80fbcb0 0x7f99c81106c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 --2- 172.21.15.77:0/1827440777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99c80fcf60 0x7f99c8112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 >> 172.21.15.77:0/1827440777 conn(0x7f99c80f7430 msgr2=0x7f99c80f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 shutdown_connections 2024-04-26T04:54:47.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.847+0000 7f99cf82b700 1 -- 172.21.15.77:0/1827440777 wait complete. 2024-04-26T04:54:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 -- 172.21.15.77:0/2798863137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 msgr2=0x7fd1940fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/2798863137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd1940fd320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd190004940 tx=0x7fd190009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 -- 172.21.15.77:0/2798863137 shutdown_connections 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/2798863137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd1940fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/2798863137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd1940fd320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 -- 172.21.15.77:0/2798863137 >> 172.21.15.77:0/2798863137 conn(0x7fd1940f8560 msgr2=0x7fd1940fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 -- 172.21.15.77:0/2798863137 shutdown_connections 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 -- 172.21.15.77:0/2798863137 wait complete. 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.977+0000 7fd19c4cf700 1 Processor -- start 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19c4cf700 1 -- start start 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19c4cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19c4cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd19410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19c4cf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19410e490 con 0x7fd1940fcf10 2024-04-26T04:54:47.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19c4cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19410ece0 con 0x7fd1940fe080 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49056/0 (socket says 172.21.15.77:49056) 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 -- 172.21.15.77:0/265105085 learned_addr learned my addr 172.21.15.77:0/265105085 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd199a6a700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd19410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 -- 172.21.15.77:0/265105085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 msgr2=0x7fd19410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd19410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd19a26b700 1 -- 172.21.15.77:0/265105085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190005040 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.978+0000 7fd199a6a700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd19410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd19a26b700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd190013870 tx=0x7fd190009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19000e070 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19410ef60 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd190013de0 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19001ce70 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.979+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19410f440 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.980+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19405f5c0 con 0x7fd1940fcf10 2024-04-26T04:54:47.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.981+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fd19001c690 con 0x7fd1940fcf10 2024-04-26T04:54:47.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.981+0000 7fd18affd700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 0x7fd180070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.981+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fd19008eae0 con 0x7fd1940fcf10 2024-04-26T04:54:47.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.981+0000 7fd199a6a700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 0x7fd180070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.982+0000 7fd199a6a700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 0x7fd180070010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd184009b80 tx=0x7fd18400b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:47.988+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd19005c020 con 0x7fd1940fcf10 2024-04-26T04:54:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.372+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd19410f720 con 0x7fd1940fcf10 2024-04-26T04:54:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.372+0000 7fd18affd700 1 -- 172.21.15.77:0/265105085 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fd190004490 con 0x7fd1940fcf10 2024-04-26T04:54:48.374 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:54:48.374 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:54:48.374 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:54:48.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:54:48.376 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.375+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 msgr2=0x7fd180070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.375+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 0x7fd180070010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd184009b80 tx=0x7fd18400b040 comp rx=0 tx=0).stop 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.375+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 msgr2=0x7fd19410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.375+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd190013870 tx=0x7fd190009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 shutdown_connections 2024-04-26T04:54:48.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd18006db60 0x7fd180070010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1940fcf10 0x7fd19410d950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 --2- 172.21.15.77:0/265105085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1940fe080 0x7fd19410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 >> 172.21.15.77:0/265105085 conn(0x7fd1940f8560 msgr2=0x7fd1940f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 shutdown_connections 2024-04-26T04:54:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.376+0000 7fd19c4cf700 1 -- 172.21.15.77:0/265105085 wait complete. 2024-04-26T04:54:48.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 -- 172.21.15.77:0/839179390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 msgr2=0x7f79a01047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 --2- 172.21.15.77:0/839179390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01047b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7994004940 tx=0x7f7994009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:48.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 -- 172.21.15.77:0/839179390 shutdown_connections 2024-04-26T04:54:48.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 --2- 172.21.15.77:0/839179390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01047b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 --2- 172.21.15.77:0/839179390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a0101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 -- 172.21.15.77:0/839179390 >> 172.21.15.77:0/839179390 conn(0x7f79a00f74c0 msgr2=0x7f79a00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 -- 172.21.15.77:0/839179390 shutdown_connections 2024-04-26T04:54:48.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.500+0000 7f79a6995700 1 -- 172.21.15.77:0/839179390 wait complete. 2024-04-26T04:54:48.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 Processor -- start 2024-04-26T04:54:48.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 -- start start 2024-04-26T04:54:48.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a010fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0110880 con 0x7f79a01023d0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f79a6995700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0111020 con 0x7f79a0066af0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.501+0000 7f799f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49076/0 (socket says 172.21.15.77:49076) 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799f7fe700 1 -- 172.21.15.77:0/3301782442 learned_addr learned my addr 172.21.15.77:0/3301782442 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799ffff700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a010fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799ffff700 1 -- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 msgr2=0x7f79a01101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799ffff700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799ffff700 1 -- 172.21.15.77:0/3301782442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994005040 con 0x7f79a0066af0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.502+0000 7f799ffff700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a010fcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f799000b550 tx=0x7f79900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.503+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f799000f6f0 con 0x7f79a0066af0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.503+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f799000fd20 con 0x7f79a0066af0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.503+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a0111300 con 0x7f79a0066af0 2024-04-26T04:54:48.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.503+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79900103a0 con 0x7f79a0066af0 2024-04-26T04:54:48.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.503+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01118c0 con 0x7f79a0066af0 2024-04-26T04:54:48.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.504+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a005f5c0 con 0x7f79a0066af0 2024-04-26T04:54:48.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.504+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f7990010500 con 0x7f79a0066af0 2024-04-26T04:54:48.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.505+0000 7f799cff9700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 0x7f7988070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.505+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f799008cff0 con 0x7f79a0066af0 2024-04-26T04:54:48.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.505+0000 7f799f7fe700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 0x7f7988070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.506+0000 7f799f7fe700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 0x7f7988070010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7994004f90 tx=0x7f7994009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.510+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f79900c1100 con 0x7f79a0066af0 2024-04-26T04:54:48.672 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:48 smithi077 ceph-mon[26311]: from='client.24557 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:48.673 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:48 smithi077 ceph-mon[26311]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:48.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:48 smithi078 ceph-mon[33746]: from='client.24557 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:48.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:48 smithi078 ceph-mon[33746]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:48.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.853+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79a0107b50 con 0x7f79a0066af0 2024-04-26T04:54:48.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.853+0000 7f799cff9700 1 -- 172.21.15.77:0/3301782442 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f799005a730 con 0x7f79a0066af0 2024-04-26T04:54:48.855 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:54:48.855 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:54:48.855 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:54:48.855 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:54:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:48.857 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:54:48.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 msgr2=0x7f7988070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 0x7f7988070010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7994004f90 tx=0x7f7994009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 msgr2=0x7f79a010fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a010fcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f799000b550 tx=0x7f79900091a0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 shutdown_connections 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f798806db60 0x7f7988070010 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0066af0 0x7f79a010fcb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 --2- 172.21.15.77:0/3301782442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a01023d0 0x7f79a01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 >> 172.21.15.77:0/3301782442 conn(0x7f79a00f74c0 msgr2=0x7f79a00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 shutdown_connections 2024-04-26T04:54:48.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.857+0000 7f79a6995700 1 -- 172.21.15.77:0/3301782442 wait complete. 2024-04-26T04:54:48.860 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 -- 172.21.15.77:0/3786514903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fce60 msgr2=0x7f37bc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 --2- 172.21.15.77:0/3786514903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fce60 0x7f37bc0fd270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f37ac004940 tx=0x7f37ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 -- 172.21.15.77:0/3786514903 shutdown_connections 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 --2- 172.21.15.77:0/3786514903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 --2- 172.21.15.77:0/3786514903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fce60 0x7f37bc0fd270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 -- 172.21.15.77:0/3786514903 >> 172.21.15.77:0/3786514903 conn(0x7f37bc0f8540 msgr2=0x7f37bc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 -- 172.21.15.77:0/3786514903 shutdown_connections 2024-04-26T04:54:48.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.988+0000 7f37c14a6700 1 -- 172.21.15.77:0/3786514903 wait complete. 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.989+0000 7f37c14a6700 1 Processor -- start 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.989+0000 7f37c14a6700 1 -- start start 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.989+0000 7f37c14a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fce60 0x7f37bc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37c14a6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37c14a6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc10e450 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37c14a6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc10e5c0 con 0x7f37bc0fce60 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49094/0 (socket says 172.21.15.77:49094) 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 -- 172.21.15.77:0/4269997941 learned_addr learned my addr 172.21.15.77:0/4269997941 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37baffd700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fce60 0x7f37bc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 -- 172.21.15.77:0/4269997941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fce60 msgr2=0x7f37bc10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fce60 0x7f37bc10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 -- 172.21.15.77:0/4269997941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac005040 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37ba7fc700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37b0002a50 tx=0x7f37b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b0007cc0 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37c14a6700 1 -- 172.21.15.77:0/4269997941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc113020 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37b000f040 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b000da30 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.990+0000 7f37c14a6700 1 -- 172.21.15.77:0/4269997941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1135b0 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.991+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c0052b0 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.992+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f37b000db90 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.992+0000 7f37a37fe700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 0x7f37a4070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.993+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f37b008d0b0 con 0x7f37bc0fdfd0 2024-04-26T04:54:48.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.993+0000 7f37baffd700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 0x7f37a4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:48.993+0000 7f37baffd700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 0x7f37a4070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b970 tx=0x7f37ac009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.000+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f37b005a670 con 0x7f37bc0fdfd0 2024-04-26T04:54:49.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.319+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f379c000bc0 con 0x7f37a406db70 2024-04-26T04:54:49.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.321+0000 7f37a37fe700 1 -- 172.21.15.77:0/4269997941 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f379c000bc0 con 0x7f37a406db70 2024-04-26T04:54:49.322 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr" 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "2/21 daemons upgraded", 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "", 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:54:49.323 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:54:49.324 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:49 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/265105085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:49.324 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:49 smithi077 ceph-mon[26311]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 755 KiB/s wr, 135 op/s 2024-04-26T04:54:49.324 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:49 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/3301782442' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 msgr2=0x7f37a4070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 0x7f37a4070020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b970 tx=0x7f37ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 msgr2=0x7f37bc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37b0002a50 tx=0x7f37b00046b0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 shutdown_connections 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f37a406db70 0x7f37a4070020 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37bc0fce60 0x7f37bc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 --2- 172.21.15.77:0/4269997941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37bc0fdfd0 0x7f37bc10de50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.323+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 >> 172.21.15.77:0/4269997941 conn(0x7f37bc0f8540 msgr2=0x7f37bc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.324+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 shutdown_connections 2024-04-26T04:54:49.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.324+0000 7f37a17fa700 1 -- 172.21.15.77:0/4269997941 wait complete. 2024-04-26T04:54:49.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/265497545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 msgr2=0x7f4fc4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/265497545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc4102330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4004940 tx=0x7f4fb4009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:49.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/265497545 shutdown_connections 2024-04-26T04:54:49.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/265497545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc4102330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/265497545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 0x7f4fc40ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/265497545 >> 172.21.15.77:0/265497545 conn(0x7f4fc40f7440 msgr2=0x7f4fc40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/265497545 shutdown_connections 2024-04-26T04:54:49.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.465+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/265497545 wait complete. 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 Processor -- start 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 -- start start 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 0x7f4fc40fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40fd810 con 0x7f4fc40fff50 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fcb5bb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40fbe50 con 0x7f4fc4066ef0 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fc8b56700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fc8b56700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49098/0 (socket says 172.21.15.77:49098) 2024-04-26T04:54:49.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fc8b56700 1 -- 172.21.15.77:0/2595977329 learned_addr learned my addr 172.21.15.77:0/2595977329 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.466+0000 7f4fc9357700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 0x7f4fc40fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fc8b56700 1 -- 172.21.15.77:0/2595977329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 msgr2=0x7f4fc40fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fc8b56700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 0x7f4fc40fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fc8b56700 1 -- 172.21.15.77:0/2595977329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4005040 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fc8b56700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009150 tx=0x7f4fb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb400e040 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb4004290 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb401c630 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc40fc0d0 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.467+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40fc5c0 con 0x7f4fc40fff50 2024-04-26T04:54:49.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.468+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc404c710 con 0x7f4fc40fff50 2024-04-26T04:54:49.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.469+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f4fb400bbf0 con 0x7f4fc40fff50 2024-04-26T04:54:49.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.470+0000 7f4fb9ffb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 0x7f4fb0070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.470+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f4fb408df80 con 0x7f4fc40fff50 2024-04-26T04:54:49.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.470+0000 7f4fc9357700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 0x7f4fb0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.471+0000 7f4fc9357700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 0x7f4fb0070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4066c30 tx=0x7f4fc000b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.475+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f4fb405b540 con 0x7f4fc40fff50 2024-04-26T04:54:49.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:49 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/265105085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:49.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:49 smithi078 ceph-mon[33746]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 755 KiB/s wr, 135 op/s 2024-04-26T04:54:49.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:49 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/3301782442' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:49.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.890+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fc4115bc0 con 0x7f4fc40fff50 2024-04-26T04:54:49.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.891+0000 7f4fb9ffb700 1 -- 172.21.15.77:0/2595977329 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4fb405b0d0 con 0x7f4fc40fff50 2024-04-26T04:54:49.893 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T04:54:49.893 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:54:49.893 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 msgr2=0x7f4fb0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 0x7f4fb0070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4066c30 tx=0x7f4fc000b040 comp rx=0 tx=0).stop 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 msgr2=0x7f4fc40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009150 tx=0x7f4fb4009e30 comp rx=0 tx=0).stop 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 shutdown_connections 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f4fb006db70 0x7f4fb0070020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc4066ef0 0x7f4fc40fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 --2- 172.21.15.77:0/2595977329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc40fff50 0x7f4fc40fb910 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 >> 172.21.15.77:0/2595977329 conn(0x7f4fc40f7440 msgr2=0x7f4fc40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 shutdown_connections 2024-04-26T04:54:49.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:54:49.893+0000 7f4fcb5bb700 1 -- 172.21.15.77:0/2595977329 wait complete. 2024-04-26T04:54:50.967 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:50 smithi077 ceph-mon[26311]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:50.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:50 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:50.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:50 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:50.968 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:50 smithi077 ceph-mon[26311]: from='client.? 172.21.15.77:0/2595977329' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:54:50.969 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-26T04:54:50.969 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T04:54:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:50 smithi078 ceph-mon[33746]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:50 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:50 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:50 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/2595977329' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:54:51.272 DEBUG:teuthology.parallel:result is None 2024-04-26T04:54:51.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T04:54:51.307 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T04:54:51.309 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T04:54:51.342 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T04:54:51.342 DEBUG:teuthology.parallel:result is None 2024-04-26T04:54:51.882 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:51 smithi077 ceph-mon[26311]: pgmap v9: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.0 MiB/s wr, 202 op/s 2024-04-26T04:54:51.882 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:51 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:51.882 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:51 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:51 smithi078 ceph-mon[33746]: pgmap v9: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.0 MiB/s wr, 202 op/s 2024-04-26T04:54:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:51 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:52.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:51 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:53.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:53 smithi077 ceph-mon[26311]: pgmap v10: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 971 KiB/s wr, 185 op/s 2024-04-26T04:54:54.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:53 smithi078 ceph-mon[33746]: pgmap v10: 65 pgs: 65 active+clean; 262 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 971 KiB/s wr, 185 op/s 2024-04-26T04:54:54.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:54 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:55.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:54 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:55.971 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:55 smithi077 ceph-mon[26311]: pgmap v11: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.1 MiB/s wr, 206 op/s 2024-04-26T04:54:56.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:55 smithi078 ceph-mon[33746]: pgmap v11: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.1 MiB/s wr, 206 op/s 2024-04-26T04:54:58.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-04-26T04:54:58.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:57 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:57 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.986 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:54:58.986 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:54:58.986 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:58.986 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:58.986 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 640 KiB/s wr, 130 op/s 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:58.987 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:58 smithi077 ceph-mon[26311]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:54:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:54:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:54:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 640 KiB/s wr, 130 op/s 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:54:59.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:58 smithi078 ceph-mon[33746]: Reconfiguring prometheus.smithi077 (dependencies changed)... 2024-04-26T04:55:00.063 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:59 smithi077 ceph-mon[26311]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:55:00.063 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:54:59 smithi077 ceph-mon[26311]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:55:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:59 smithi078 ceph-mon[33746]: Reconfiguring daemon prometheus.smithi077 on smithi077 2024-04-26T04:55:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:54:59 smithi078 ceph-mon[33746]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:55:00.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:00 smithi077 ceph-mon[26311]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 641 KiB/s wr, 131 op/s 2024-04-26T04:55:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:00 smithi078 ceph-mon[33746]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 641 KiB/s wr, 131 op/s 2024-04-26T04:55:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:02 smithi077 ceph-mon[26311]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 208 KiB/s wr, 42 op/s 2024-04-26T04:55:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:02 smithi078 ceph-mon[33746]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 208 KiB/s wr, 42 op/s 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:04 smithi077 ceph-mon[26311]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 208 KiB/s wr, 42 op/s 2024-04-26T04:55:04.674 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.674 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.674 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:55:04.674 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:55:04.675 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:55:04.675 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi077:9095"}]: dispatch 2024-04-26T04:55:04.675 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:04.675 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:04.675 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:04 smithi078 ceph-mon[33746]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 208 KiB/s wr, 42 op/s 2024-04-26T04:55:05.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:05 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:55:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:05 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:55:05.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:05 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:05 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:55:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:05 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:55:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:05 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:06.716 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:06 smithi078 ceph-mon[33746]: Upgrade: Updating mgr.smithi078.navvuw 2024-04-26T04:55:06.716 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:06 smithi078 ceph-mon[33746]: Deploying daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:55:06.716 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:06 smithi078 ceph-mon[33746]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 8.9 KiB/s wr, 19 op/s 2024-04-26T04:55:06.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:06 smithi077 ceph-mon[26311]: Upgrade: Updating mgr.smithi078.navvuw 2024-04-26T04:55:06.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:06 smithi077 ceph-mon[26311]: Deploying daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:55:06.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:06 smithi077 ceph-mon[26311]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.5 KiB/s rd, 8.9 KiB/s wr, 19 op/s 2024-04-26T04:55:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 182 B/s rd, 638 B/s wr, 1 op/s 2024-04-26T04:55:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:08 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:08.995 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 182 B/s rd, 638 B/s wr, 1 op/s 2024-04-26T04:55:08.995 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.995 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:08.995 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.995 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:08.996 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:08 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:11.098 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:10 smithi078 ceph-mon[33746]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 597 B/s wr, 0 op/s 2024-04-26T04:55:11.147 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:10 smithi077 ceph-mon[26311]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 597 B/s wr, 0 op/s 2024-04-26T04:55:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:12 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:12 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:12 smithi077 ceph-mon[26311]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:12 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:12 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:12 smithi078 ceph-mon[33746]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:13.983 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:13 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:13.983 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:13 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:13.983 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:13 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:13.983 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:13 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:13 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:14.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:13 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:14.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:13 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:14.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:13 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:15.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:14 smithi078 ceph-mon[33746]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:14 smithi077 ceph-mon[26311]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:15.965 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:55:15.965 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:55:15.965 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:55:15.966 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:55:15.966 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:55:15.966 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:15 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:55:16.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:55:16.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:55:16.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:55:16.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:55:16.111 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:55:16.111 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:15 smithi077 ceph-mon[26311]: from='mgr.? 172.21.15.78:0/2223678111' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: mgrmap e33: smithi077.hkvsor(active, since 37s), standbys: smithi078.navvuw 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:17.179 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi077.hkvsor"}]: dispatch 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi077.hkvsor"}]': finished 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi078.navvuw"}]: dispatch 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi078.navvuw"}]': finished 2024-04-26T04:55:17.180 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:16 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:55:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: mgrmap e33: smithi077.hkvsor(active, since 37s), standbys: smithi078.navvuw 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi077.hkvsor"}]: dispatch 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi077.hkvsor"}]': finished 2024-04-26T04:55:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi078.navvuw"}]: dispatch 2024-04-26T04:55:17.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi078.navvuw"}]': finished 2024-04-26T04:55:17.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:16 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:55:18.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:17 smithi077 ceph-mon[26311]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:18.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:17 smithi077 ceph-mon[26311]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:18.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:17 smithi077 ceph-mon[26311]: Upgrade: Setting container_image for all mgr 2024-04-26T04:55:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:17 smithi078 ceph-mon[33746]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:17 smithi078 ceph-mon[33746]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:18.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:17 smithi078 ceph-mon[33746]: Upgrade: Setting container_image for all mgr 2024-04-26T04:55:19.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: Upgrade: Updating mon.smithi077 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:19 smithi077 ceph-mon[26311]: Deploying daemon mon.smithi077 on smithi077 2024-04-26T04:55:20.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 -- 172.21.15.77:0/1517115237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 msgr2=0x7f76fc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 --2- 172.21.15.77:0/1517115237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc0fd3d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f76f0004940 tx=0x7f76f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 -- 172.21.15.77:0/1517115237 shutdown_connections 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 --2- 172.21.15.77:0/1517115237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc0fd3d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 --2- 172.21.15.77:0/1517115237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 0x7f76fc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 -- 172.21.15.77:0/1517115237 >> 172.21.15.77:0/1517115237 conn(0x7f76fc0f7430 msgr2=0x7f76fc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 -- 172.21.15.77:0/1517115237 shutdown_connections 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 -- 172.21.15.77:0/1517115237 wait complete. 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.007+0000 7f770138a700 1 Processor -- start 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f770138a700 1 -- start start 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f770138a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 0x7f76fc10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f770138a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f770138a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc10e5c0 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f770138a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc10edc0 con 0x7f76fc0fbcb0 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50006/0 (socket says 172.21.15.77:50006) 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 -- 172.21.15.77:0/1990174600 learned_addr learned my addr 172.21.15.77:0/1990174600 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76faffd700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 0x7f76fc10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 -- 172.21.15.77:0/1990174600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 msgr2=0x7f76fc10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 0x7f76fc10da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 -- 172.21.15.77:0/1990174600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f0005040 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76fa7fc700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009150 tx=0x7f76f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.008+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f000e040 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.009+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc10f040 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.009+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76f0004290 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.009+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f001c630 con 0x7f76fc0fcf60 2024-04-26T04:55:20.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.009+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc10f580 con 0x7f76fc0fcf60 2024-04-26T04:55:20.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.009+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc101af0 con 0x7f76fc0fcf60 2024-04-26T04:55:20.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.010+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f76f000bbf0 con 0x7f76fc0fcf60 2024-04-26T04:55:20.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.010+0000 7f76e37fe700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 0x7f76e4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.011+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f76f008deb0 con 0x7f76fc0fcf60 2024-04-26T04:55:20.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.011+0000 7f76faffd700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 0x7f76e4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.011+0000 7f76faffd700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 0x7f76e4070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007a40 tx=0x7f76f4009da0 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.015+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f76f005b510 con 0x7f76fc0fcf60 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: Upgrade: Updating mon.smithi077 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:19 smithi078 ceph-mon[33746]: Deploying daemon mon.smithi077 on smithi077 2024-04-26T04:55:20.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.231+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76fc1048b0 con 0x7f76e406db60 2024-04-26T04:55:20.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.233+0000 7f76e37fe700 1 -- 172.21.15.77:0/1990174600 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f76fc1048b0 con 0x7f76e406db60 2024-04-26T04:55:20.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 msgr2=0x7f76e4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 0x7f76e4070010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007a40 tx=0x7f76f4009da0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 msgr2=0x7f76fc10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009150 tx=0x7f76f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 shutdown_connections 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f76e406db60 0x7f76e4070010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc0fbcb0 0x7f76fc10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 --2- 172.21.15.77:0/1990174600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76fc0fcf60 0x7f76fc10dfc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 >> 172.21.15.77:0/1990174600 conn(0x7f76fc0f7430 msgr2=0x7f76fc065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 shutdown_connections 2024-04-26T04:55:20.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.235+0000 7f770138a700 1 -- 172.21.15.77:0/1990174600 wait complete. 2024-04-26T04:55:20.246 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:55:20.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- 172.21.15.77:0/2050197607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 msgr2=0x7f11b40fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 --2- 172.21.15.77:0/2050197607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b40fe7f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f11b0004940 tx=0x7f11b0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- 172.21.15.77:0/2050197607 shutdown_connections 2024-04-26T04:55:20.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 --2- 172.21.15.77:0/2050197607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 0x7f11b40fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 --2- 172.21.15.77:0/2050197607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b40fe7f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- 172.21.15.77:0/2050197607 >> 172.21.15.77:0/2050197607 conn(0x7f11b40f6b80 msgr2=0x7f11b40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- 172.21.15.77:0/2050197607 shutdown_connections 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- 172.21.15.77:0/2050197607 wait complete. 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 Processor -- start 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 -- start start 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.340+0000 7f11bc608700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 0x7f11b410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11bc608700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b410e510 con 0x7f11b40fe3e0 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11bc608700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b410ed60 con 0x7f11b40fed30 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50026/0 (socket says 172.21.15.77:50026) 2024-04-26T04:55:20.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 -- 172.21.15.77:0/191260417 learned_addr learned my addr 172.21.15.77:0/191260417 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11b9ba3700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 0x7f11b410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 -- 172.21.15.77:0/191260417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 msgr2=0x7f11b410df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 0x7f11b410df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 -- 172.21.15.77:0/191260417 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b0005040 con 0x7f11b40fe3e0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11ba3a4700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f11b000b3b0 tx=0x7f11b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b000e040 con 0x7f11b40fe3e0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b410efe0 con 0x7f11b40fe3e0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11b000bbf0 con 0x7f11b40fe3e0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b001c630 con 0x7f11b40fe3e0 2024-04-26T04:55:20.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.341+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b410f520 con 0x7f11b40fe3e0 2024-04-26T04:55:20.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.342+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b405f5c0 con 0x7f11b40fe3e0 2024-04-26T04:55:20.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.343+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f11b000bd60 con 0x7f11b40fe3e0 2024-04-26T04:55:20.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.343+0000 7f11aaffd700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 0x7f11a0070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.343+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f11b008dfa0 con 0x7f11b40fe3e0 2024-04-26T04:55:20.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.343+0000 7f11b9ba3700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 0x7f11a0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.344+0000 7f11b9ba3700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 0x7f11a0070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11a4004f40 tx=0x7f11a400b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.348+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f11b005b560 con 0x7f11b40fe3e0 2024-04-26T04:55:20.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.580+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b410f8c0 con 0x7f11a006db60 2024-04-26T04:55:20.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.581+0000 7f11aaffd700 1 -- 172.21.15.77:0/191260417 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f11b410f8c0 con 0x7f11a006db60 2024-04-26T04:55:20.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 msgr2=0x7f11a0070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 0x7f11a0070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11a4004f40 tx=0x7f11a400b040 comp rx=0 tx=0).stop 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 msgr2=0x7f11b410d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f11b000b3b0 tx=0x7f11b0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 shutdown_connections 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f11a006db60 0x7f11a0070010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11b40fe3e0 0x7f11b410d9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 --2- 172.21.15.77:0/191260417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11b40fed30 0x7f11b410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 >> 172.21.15.77:0/191260417 conn(0x7f11b40f6b80 msgr2=0x7f11b40f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 shutdown_connections 2024-04-26T04:55:20.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.585+0000 7f11bc608700 1 -- 172.21.15.77:0/191260417 wait complete. 2024-04-26T04:55:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 -- 172.21.15.77:0/2186790861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fd3d0 msgr2=0x7fd3e40fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 --2- 172.21.15.77:0/2186790861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e40fd7e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0004940 tx=0x7fd3e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 -- 172.21.15.77:0/2186790861 shutdown_connections 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 --2- 172.21.15.77:0/2186790861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fdd20 0x7fd3e4104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 --2- 172.21.15.77:0/2186790861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e40fd7e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 -- 172.21.15.77:0/2186790861 >> 172.21.15.77:0/2186790861 conn(0x7fd3e40f6b80 msgr2=0x7fd3e40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 -- 172.21.15.77:0/2186790861 shutdown_connections 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.693+0000 7fd3ec564700 1 -- 172.21.15.77:0/2186790861 wait complete. 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 Processor -- start 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 -- start start 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e410fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e41107a0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ec564700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e4110fa0 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50040/0 (socket says 172.21.15.77:50040) 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 learned_addr learned my addr 172.21.15.77:0/3265430586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ea300700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e410fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 msgr2=0x7fd3e410fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e410fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0005040 con 0x7fd3e40fdd20 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.694+0000 7fd3ea300700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3e410fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.695+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4002a50 tx=0x7fd3d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.695+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3d4007cc0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.695+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3d400f040 con 0x7fd3e40fdd20 2024-04-26T04:55:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.695+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e4111280 con 0x7fd3e40fdd20 2024-04-26T04:55:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.696+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3d400da30 con 0x7fd3e40fdd20 2024-04-26T04:55:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.696+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e41117c0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.696+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fd3d400dc70 con 0x7fd3e40fdd20 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.697+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e405f5c0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.697+0000 7fd3daffd700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 0x7fd3d0070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.697+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fd3d408c9f0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.697+0000 7fd3ea300700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 0x7fd3d0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.697+0000 7fd3ea300700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 0x7fd3d0070010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b970 tx=0x7fd3e0009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.705+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd3d4059ea0 con 0x7fd3e40fdd20 2024-04-26T04:55:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 msgr2=0x7fd3e41101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:55:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 msgr2=0x7fd3e41101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:55:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4002a50 tx=0x7fd3d40046b0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-26T04:55:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4002a50 tx=0x7fd3d40046b0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 msgr2=0x7fd3e41101a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3daffd700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3e40fdd20 0x7fd3e41101a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3daffd700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3d0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.826+0000 7fd3daffd700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 0x7fd3d007a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3eab01700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 msgr2=0x7fd3d007a2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:3300/0 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3eab01700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 0x7fd3d007a2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d007ad10 con 0x7fd3d0077e20 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d007ae50 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3d0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 msgr2=0x7fd3d007a2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:55:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 0x7fd3d007a2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.827+0000 7fd3e9aff700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=3+}) v3 -- 0x7fd3d007afe0 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.828+0000 7fd3e9aff700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3d0077720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3d408ab70 tx=0x7fd3d40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.828+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3d407a0c0 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.829+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3d40097c0 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.831+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fd3d405e170 con 0x7fd3e40fd3d0 2024-04-26T04:55:20.831 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:20 smithi077 systemd[1]: Stopping Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:55:20.832 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:20 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:55:20.825+0000 7ff943393700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi077 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:20.832 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:20 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[26307]: 2024-04-26T04:55:20.825+0000 7ff943393700 -1 mon.smithi077@0(leader) e2 *** Got Signal Terminated *** 2024-04-26T04:55:20.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.928+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3e404bed0 con 0x7fd3d006db60 2024-04-26T04:55:20.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.938+0000 7fd3daffd700 1 -- 172.21.15.77:0/3265430586 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd3e404bed0 con 0x7fd3d006db60 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (118s) 7s ago 10m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (10m) 7s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (10m) 9s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 7s ago 10m - - 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (5m) 7s ago 5m 154M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (5m) 7s ago 5m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (5m) 9s ago 5m 15.8M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:55:20.939 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (5m) 9s ago 5m 15.8M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (66s) 7s ago 12m 493M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (12s) 9s ago 9m 65.1M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (12m) 7s ago 12m 54.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af f1ae32d34661 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (9m) 9s ago 9m 42.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 1b87cf9e8e17 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (2m) 7s ago 10m 16.1M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (89s) 9s ago 9m 16.2M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (9m) 7s ago 9m 203M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:55:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (8m) 7s ago 8m 212M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:55:20.941 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (8m) 7s ago 8m 177M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:55:20.941 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (7m) 9s ago 7m 232M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:55:20.941 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (7m) 9s ago 7m 209M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:55:20.941 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (6m) 9s ago 6m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:55:20.941 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (17s) 7s ago 10m 39.8M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:55:20.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 msgr2=0x7fd3d0070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 0x7fd3d0070010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000b970 tx=0x7fd3e0009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:20.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 msgr2=0x7fd3d0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:20.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3d0077720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3d408ab70 tx=0x7fd3d40046b0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 shutdown_connections 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0077e20 0x7fd3d007a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3e40fd3d0 0x7fd3d0077720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 --2- 172.21.15.77:0/3265430586 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7fd3d006db60 0x7fd3d0070010 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 >> 172.21.15.77:0/3265430586 conn(0x7fd3e40f6b80 msgr2=0x7fd3e40f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 shutdown_connections 2024-04-26T04:55:20.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:20.941+0000 7fd3ec564700 1 -- 172.21.15.77:0/3265430586 wait complete. 2024-04-26T04:55:21.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 -- 172.21.15.77:0/327023906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fbcb0 msgr2=0x7f3f640fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/327023906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fbcb0 0x7f3f640fc0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f58004940 tx=0x7f3f58009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 -- 172.21.15.77:0/327023906 shutdown_connections 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/327023906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fcf60 0x7f3f640fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/327023906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fbcb0 0x7f3f640fc0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 -- 172.21.15.77:0/327023906 >> 172.21.15.77:0/327023906 conn(0x7f3f640f7430 msgr2=0x7f3f640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 -- 172.21.15.77:0/327023906 shutdown_connections 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.041+0000 7f3f6afaa700 1 -- 172.21.15.77:0/327023906 wait complete. 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 Processor -- start 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 -- start start 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 0x7f3f64110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640663c0 con 0x7f3f640fbcb0 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f6afaa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64066500 con 0x7f3f640fcf60 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f68d46700 1 -- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 msgr2=0x7f3f64110790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:3300/0 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f68d46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 0x7f3f64110790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f63fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:35208/0 (socket says 172.21.15.77:35208) 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.042+0000 7f3f63fff700 1 -- 172.21.15.77:0/697881352 learned_addr learned my addr 172.21.15.77:0/697881352 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f63fff700 1 -- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 msgr2=0x7f3f64110790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f63fff700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 0x7f3f64110790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f63fff700 1 -- 172.21.15.77:0/697881352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58005040 con 0x7f3f640fcf60 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f63fff700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000b760 tx=0x7f3f50009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f50010790 con 0x7f3f640fcf60 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f64113340 con 0x7f3f640fcf60 2024-04-26T04:55:21.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f50010dc0 con 0x7f3f640fcf60 2024-04-26T04:55:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5000f300 con 0x7f3f640fcf60 2024-04-26T04:55:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.043+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f641138d0 con 0x7f3f640fcf60 2024-04-26T04:55:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.044+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6405f5c0 con 0x7f3f640fcf60 2024-04-26T04:55:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.045+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f3f500117f0 con 0x7f3f640fcf60 2024-04-26T04:55:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.045+0000 7f3f617fa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 0x7f3f5406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.045+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f3f5008d3f0 con 0x7f3f640fcf60 2024-04-26T04:55:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.045+0000 7f3f68d46700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 0x7f3f5406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.045+0000 7f3f68d46700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 0x7f3f5406ff60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f58004890 tx=0x7f3f58009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.050+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3f50056eb0 con 0x7f3f640fcf60 2024-04-26T04:55:21.096 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:20 smithi077 podman[91565]: 2024-04-26 04:55:20.906794431 +0000 UTC m=+0.173997898 container died f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:55:21.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.332+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f64113ca0 con 0x7f3f640fcf60 2024-04-26T04:55:21.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.333+0000 7f3f617fa700 1 -- 172.21.15.77:0/697881352 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f3f5005a570 con 0x7f3f640fcf60 2024-04-26T04:55:21.333 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:55:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 msgr2=0x7f3f5406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 0x7f3f5406ff60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f58004890 tx=0x7f3f58009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 msgr2=0x7f3f64112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000b760 tx=0x7f3f50009220 comp rx=0 tx=0).stop 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 shutdown_connections 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f3f5406dab0 0x7f3f5406ff60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f640fbcb0 0x7f3f64110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 --2- 172.21.15.77:0/697881352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f640fcf60 0x7f3f64112ce0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 >> 172.21.15.77:0/697881352 conn(0x7f3f640f7430 msgr2=0x7f3f640f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 shutdown_connections 2024-04-26T04:55:21.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.335+0000 7f3f6afaa700 1 -- 172.21.15.77:0/697881352 wait complete. 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 -- 172.21.15.77:0/2198238166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fce80 msgr2=0x7f45900fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 --2- 172.21.15.77:0/2198238166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fce80 0x7f45900fd290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f458c004940 tx=0x7f458c009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 -- 172.21.15.77:0/2198238166 shutdown_connections 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 --2- 172.21.15.77:0/2198238166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fe080 0x7f45900fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 --2- 172.21.15.77:0/2198238166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fce80 0x7f45900fd290 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 -- 172.21.15.77:0/2198238166 >> 172.21.15.77:0/2198238166 conn(0x7f45900f8590 msgr2=0x7f45900fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 -- 172.21.15.77:0/2198238166 shutdown_connections 2024-04-26T04:55:21.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.441+0000 7f4598377700 1 -- 172.21.15.77:0/2198238166 wait complete. 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 Processor -- start 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 -- start start 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 0x7f4590071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45900708f0 con 0x7f45900fce80 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4598377700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4590070a60 con 0x7f45900fe080 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4596113700 1 -- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 msgr2=0x7f4590071de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:3300/0 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4596113700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 0x7f4590071de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4595912700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4595912700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:35228/0 (socket says 172.21.15.77:35228) 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.442+0000 7f4595912700 1 -- 172.21.15.77:0/746828468 learned_addr learned my addr 172.21.15.77:0/746828468 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4595912700 1 -- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 msgr2=0x7f4590071de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4595912700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 0x7f4590071de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4595912700 1 -- 172.21.15.77:0/746828468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c005040 con 0x7f45900fe080 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4595912700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f458000b760 tx=0x7f4580009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4580010790 con 0x7f45900fe080 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4580010dc0 con 0x7f45900fe080 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590070d40 con 0x7f45900fe080 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458000f300 con 0x7f45900fe080 2024-04-26T04:55:21.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.443+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590115b60 con 0x7f45900fe080 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.444+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459005f5c0 con 0x7f45900fe080 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.445+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f45800117f0 con 0x7f45900fe080 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.445+0000 7f4586ffd700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 0x7f457c06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.445+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f458008d400 con 0x7f45900fe080 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.445+0000 7f4596113700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 0x7f457c06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.446+0000 7f4596113700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 0x7f457c06ff60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.449+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f4580056f00 con 0x7f45900fe080 2024-04-26T04:55:21.678 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 podman[91565]: 2024-04-26 04:55:21.376248666 +0000 UTC m=+0.643452132 container cleanup f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T04:55:21.678 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 bash[91565]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077 2024-04-26T04:55:21.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.696+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4590105940 con 0x7f45900fe080 2024-04-26T04:55:21.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.697+0000 7f4586ffd700 1 -- 172.21.15.77:0/746828468 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1804 (secure 0 0 0) 0x7f458005a5c0 con 0x7f45900fe080 2024-04-26T04:55:21.697 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:55:21.697 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:55:21.698 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:55:21.699 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:21.700 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 msgr2=0x7f457c06ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 0x7f457c06ff60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f458c000c00 tx=0x7f458c009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 msgr2=0x7f45900702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f458000b760 tx=0x7f4580009220 comp rx=0 tx=0).stop 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 shutdown_connections 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f457c06dab0 0x7f457c06ff60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45900fce80 0x7f4590071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 --2- 172.21.15.77:0/746828468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45900fe080 0x7f45900702b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 >> 172.21.15.77:0/746828468 conn(0x7f45900f8590 msgr2=0x7f45900f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 shutdown_connections 2024-04-26T04:55:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.699+0000 7f4598377700 1 -- 172.21.15.77:0/746828468 wait complete. 2024-04-26T04:55:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:55:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 -- 172.21.15.77:0/3140746820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684066b10 msgr2=0x7f66840ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 --2- 172.21.15.77:0/3140746820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684066b10 0x7f66840ffa70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6678004940 tx=0x7f6678009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 -- 172.21.15.77:0/3140746820 shutdown_connections 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 --2- 172.21.15.77:0/3140746820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66840fffb0 0x7f6684102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 --2- 172.21.15.77:0/3140746820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684066b10 0x7f66840ffa70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.806+0000 7f668af9c700 1 -- 172.21.15.77:0/3140746820 >> 172.21.15.77:0/3140746820 conn(0x7f66840f74c0 msgr2=0x7f66840f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 -- 172.21.15.77:0/3140746820 shutdown_connections 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 -- 172.21.15.77:0/3140746820 wait complete. 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 Processor -- start 2024-04-26T04:55:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 -- start start 2024-04-26T04:55:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 0x7f668410dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668410e6b0 con 0x7f6684066b10 2024-04-26T04:55:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f668af9c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668410eea0 con 0x7f66840fffb0 2024-04-26T04:55:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f6688d38700 1 -- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 msgr2=0x7f668410dae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:3300/0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.807+0000 7f6688d38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 0x7f668410dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:35248/0 (socket says 172.21.15.77:35248) 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 -- 172.21.15.77:0/3896144048 learned_addr learned my addr 172.21.15.77:0/3896144048 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 -- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 msgr2=0x7f668410dae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 0x7f668410dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 -- 172.21.15.77:0/3896144048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678005040 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f6683fff700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f667000b760 tx=0x7f6670009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6670010790 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6670010dc0 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667000f300 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668410f180 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.808+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668410f740 con 0x7f66840fffb0 2024-04-26T04:55:21.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.810+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668405f5c0 con 0x7f66840fffb0 2024-04-26T04:55:21.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.810+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f66700117f0 con 0x7f66840fffb0 2024-04-26T04:55:21.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.810+0000 7f66817fa700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 0x7f667406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:21.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.810+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f667008d480 con 0x7f66840fffb0 2024-04-26T04:55:21.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.810+0000 7f6688d38700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 0x7f667406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:21.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.811+0000 7f6688d38700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 0x7f667406ff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6678004890 tx=0x7f6678009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:21.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:21.815+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f667005aa30 con 0x7f66840fffb0 2024-04-26T04:55:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 podman[91591]: 2024-04-26 04:55:21.677815171 +0000 UTC m=+0.772102287 container remove f1ae32d34661c07d37b4b931ff13b3a6ef10ad4776bf69765d835b2a91f8ad04 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, RELEASE=quincy-c61472c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T04:55:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service: Deactivated successfully. 2024-04-26T04:55:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 systemd[1]: Stopped Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:55:21.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:21 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service: Consumed 11.294s CPU time. 2024-04-26T04:55:22.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.055+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 --> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f668410fa60 con 0x7f667406dab0 2024-04-26T04:55:22.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.056+0000 7f66817fa700 1 -- 172.21.15.77:0/3896144048 <== mgr.14764 v2:172.21.15.77:6800/3186647962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f668410fa60 con 0x7f667406dab0 2024-04-26T04:55:22.057 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:55:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:55:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr" 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "2/21 daemons upgraded", 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mon daemons", 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:55:22.058 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:55:22.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 msgr2=0x7f667406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:22.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 0x7f667406ff60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6678004890 tx=0x7f6678009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 msgr2=0x7f668410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f667000b760 tx=0x7f6670009220 comp rx=0 tx=0).stop 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 shutdown_connections 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f667406dab0 0x7f667406ff60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6684066b10 0x7f668410dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 --2- 172.21.15.77:0/3896144048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66840fffb0 0x7f668410e020 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 >> 172.21.15.77:0/3896144048 conn(0x7f66840f74c0 msgr2=0x7f66840f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 shutdown_connections 2024-04-26T04:55:22.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.059+0000 7f668af9c700 1 -- 172.21.15.77:0/3896144048 wait complete. 2024-04-26T04:55:22.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 -- 172.21.15.77:0/1104718505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fcf60 msgr2=0x7f1c600fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:22.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 --2- 172.21.15.77:0/1104718505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fcf60 0x7f1c600fd3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1c54004940 tx=0x7f1c54009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:22.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 -- 172.21.15.77:0/1104718505 shutdown_connections 2024-04-26T04:55:22.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 --2- 172.21.15.77:0/1104718505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fcf60 0x7f1c600fd3d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 --2- 172.21.15.77:0/1104718505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fbcb0 0x7f1c600fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.169+0000 7f1c67736700 1 -- 172.21.15.77:0/1104718505 >> 172.21.15.77:0/1104718505 conn(0x7f1c600f7430 msgr2=0x7f1c600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 -- 172.21.15.77:0/1104718505 shutdown_connections 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 -- 172.21.15.77:0/1104718505 wait complete. 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 Processor -- start 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 -- start start 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 0x7f1c6010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6010e530 con 0x7f1c600fcf60 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c67736700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60112ed0 con 0x7f1c600fbcb0 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c64cd1700 1 -- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 msgr2=0x7f1c6010dea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:3300/0 2024-04-26T04:55:22.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.170+0000 7f1c64cd1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 0x7f1c6010dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:35266/0 (socket says 172.21.15.77:35266) 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 -- 172.21.15.77:0/750088559 learned_addr learned my addr 172.21.15.77:0/750088559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 -- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 msgr2=0x7f1c6010dea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 0x7f1c6010dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 -- 172.21.15.77:0/750088559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54005040 con 0x7f1c600fbcb0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c654d2700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800fd80 tx=0x7f1c58009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.171+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c58010af0 con 0x7f1c600fbcb0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.172+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c58003300 con 0x7f1c600fbcb0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.172+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c600fe1e0 con 0x7f1c600fbcb0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.172+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c58001690 con 0x7f1c600fbcb0 2024-04-26T04:55:22.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.172+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c600fe700 con 0x7f1c600fbcb0 2024-04-26T04:55:22.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.173+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6004c710 con 0x7f1c600fbcb0 2024-04-26T04:55:22.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.173+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f1c58016070 con 0x7f1c600fbcb0 2024-04-26T04:55:22.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.173+0000 7f1c5dffb700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 0x7f1c4806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:22.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.174+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f1c58091530 con 0x7f1c600fbcb0 2024-04-26T04:55:22.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.174+0000 7f1c64cd1700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 0x7f1c4806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:22.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.174+0000 7f1c64cd1700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 0x7f1c4806ff60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c54004890 tx=0x7f1c54009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.178+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f1c5805eba0 con 0x7f1c600fbcb0 2024-04-26T04:55:22.387 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 systemd[1]: Starting Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:55:22.388 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 podman[91792]: 2024-04-26 04:55:22.181441697 +0000 UTC m=+0.019250761 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:22.388 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 podman[91792]: 2024-04-26 04:55:22.295538721 +0000 UTC m=+0.133347766 container create d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.469+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c60104840 con 0x7f1c600fbcb0 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.470+0000 7f1c5dffb700 1 -- 172.21.15.77:0/750088559 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+347 (secure 0 0 0) 0x7f1c5805e730 con 0x7f1c600fbcb0 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 msgr2=0x7f1c4806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 0x7f1c4806ff60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c54004890 tx=0x7f1c54009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 msgr2=0x7f1c6010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800fd80 tx=0x7f1c58009810 comp rx=0 tx=0).stop 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 shutdown_connections 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:6800/3186647962,v1:172.21.15.77:6801/3186647962] conn(0x7f1c4806dab0 0x7f1c4806ff60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c600fbcb0 0x7f1c6010d960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 --2- 172.21.15.77:0/750088559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c600fcf60 0x7f1c6010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 >> 172.21.15.77:0/750088559 conn(0x7f1c600f7430 msgr2=0x7f1c600f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:22.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 shutdown_connections 2024-04-26T04:55:22.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:22.472+0000 7f1c67736700 1 -- 172.21.15.77:0/750088559 wait complete. 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 podman[91792]: 2024-04-26 04:55:22.506377848 +0000 UTC m=+0.344186896 container init d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 podman[91792]: 2024-04-26 04:55:22.509552835 +0000 UTC m=+0.347361881 container start d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS) 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev), process ceph-mon, pid 2 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: pidfile_write: ignore empty --pid-file 2024-04-26T04:55:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: load: jerasure load: lrc 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: RocksDB version: 7.9.2 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Git sha 0 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Compile date 2024-04-25 16:43:49 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: DB SUMMARY 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: DB Session ID: 8YHPSLHJW2F5XIUZDQHO 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi077/store.db dir, Total Num: 1, files: 000030.sst 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi077/store.db: 000028.log size: 2556363 ; 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T04:55:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:55:22.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T04:55:22.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.env: 0x55cb70a86c60 2024-04-26T04:55:22.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T04:55:22.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.info_log: 0x55cb71f7b960 2024-04-26T04:55:22.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.statistics: (nil) 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.use_fsync: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:55:22.747 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.db_log_dir: 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.wal_dir: 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.write_buffer_manager: 0x55cb71f4e1e0 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:55:22.748 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.unordered_write: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.row_cache: None 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.wal_filter: None 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:55:22.749 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.wal_compression: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:55:22.750 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:55:22.751 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_open_files: -1 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Compression algorithms supported: 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kZSTD supported: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:55:22.752 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi077/store.db/MANIFEST-000015 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.merge_operator: 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_filter: None 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:55:22.753 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cb71f7ba60) 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_type: 0 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: data_block_index_type: 0 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_shortening: 1 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: checksum: 4 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: no_block_cache: 0 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache: 0x55cb71f7f090 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:55:22.754 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_options: 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: capacity : 536870912 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: num_shard_bits : 4 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: strict_capacity_limit : 0 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_cache_compressed: (nil) 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: persistent_cache: (nil) 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_size: 4096 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_size_deviation: 10 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_restart_interval: 16 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: index_block_restart_interval: 1 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: metadata_block_size: 4096 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: partition_filters: 0 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: use_delta_encoding: 1 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: filter_policy: bloomfilter 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: whole_key_filtering: 1 2024-04-26T04:55:22.755 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: verify_compression: 0 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: format_version: 5 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: enable_index_compression: 1 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: block_align: 0 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: max_auto_readahead_size: 262144 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: prepopulate_block_cache: 0 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: initial_auto_readahead_size: 8192 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression: NoCompression 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:55:22.756 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.num_levels: 7 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:55:22.757 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:55:22.758 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:55:22.759 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:55:22.760 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:55:22.761 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:55:22.762 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.ttl: 2592000 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T04:55:22.763 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi077/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10513, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d06d1a6d-30ac-41dd-aafb-0e5d6a18e0d8 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107322539413, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107322548753, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2347646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10514, "largest_seqno": 11590, "table_properties": {"data_size": 2342096, "index_size": 3315, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1221, "raw_key_size": 11363, "raw_average_key_size": 23, "raw_value_size": 2331426, "raw_average_value_size": 4797, "num_data_blocks": 152, "num_entries": 486, "num_filter_entries": 486, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714107322, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d06d1a6d-30ac-41dd-aafb-0e5d6a18e0d8", "db_session_id": "8YHPSLHJW2F5XIUZDQHO", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107322548883, "job": 1, "event": "recovery_finished"} 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi077/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:55:22.764 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cb72070000 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: DB pointer 0x55cb7205a000 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** DB Stats ** 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:55:22.765 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: L0 1/0 2.24 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 601.7 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: L6 1/0 5.42 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Sum 2/0 7.66 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 601.7 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 601.7 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** Compaction Stats [default] ** 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 601.7 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:55:22.766 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Cumulative compaction: 0.00 GB write, 186.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Interval compaction: 0.00 GB write, 186.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Block cache BinnedLRUCache@0x55cb71f7f090#2 capacity: 512.00 MB usage: 4.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.28 KB,0.000244379%) IndexBlock(1,3.33 KB,0.000634789%) Misc(1,0.00 KB,0%) 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: starting mon.smithi077 rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi077 fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???) e2 preinit fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:22.767 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).mds e14 new map 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).mds e14 print_map 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: e14 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: legacy client fscid: 1 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Filesystem 'cephfs' (1) 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: fs_name cephfs 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: epoch 13 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: tableserver 0 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: root 0 2024-04-26T04:55:22.768 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: session_timeout 60 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: session_autoclose 300 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: max_file_size 1099511627776 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: max_xattr_size 65536 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: required_client_features {} 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: last_failure 0 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: last_failure_osd_epoch 0 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: max_mds 1 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: in 0 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: up {0=14458} 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: failed 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: damaged 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: stopped 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: data_pools [3] 2024-04-26T04:55:22.769 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: metadata_pool 2 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: inline_data enabled 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: balancer 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: bal_rank_mask -1 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: standby_count_wanted 1 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: qdb_cluster leader: 0 members: 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: [mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.770 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: Standby daemons: 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: [mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: [mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout: [mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:22.771 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 ceph-mon[91833]: mon.smithi077@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-26T04:55:22.772 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 bash[91792]: d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f 2024-04-26T04:55:22.772 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:22 smithi077 systemd[1]: Started Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:55:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/697881352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/746828468' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.24603 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='client.? 172.21.15.77:0/750088559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: mon.smithi077 calling monitor election 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: mon.smithi077 is new leader, mons smithi077,smithi078 in quorum (ranks 0,1) 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: monmap epoch 2 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: last_changed 2024-04-26T04:45:34.756719+0000 2024-04-26T04:55:24.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: created 2024-04-26T04:42:57.348674+0000 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: min_mon_release 17 (quincy) 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: election_strategy: 1 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.smithi077 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.smithi078 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: mgrmap e33: smithi077.hkvsor(active, since 45s), standbys: smithi078.navvuw 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: mgrmap e34: smithi077.hkvsor(active, since 45s), standbys: smithi078.navvuw 2024-04-26T04:55:24.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:24 smithi078 ceph-mon[33746]: from='mgr.14764 ' entity='' 2024-04-26T04:55:24.341 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/697881352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/746828468' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.24603 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/750088559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:55:24.342 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='mgr.14764 172.21.15.77:0/3037930522' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: mon.smithi077 calling monitor election 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: mon.smithi077 is new leader, mons smithi077,smithi078 in quorum (ranks 0,1) 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: monmap epoch 2 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: last_changed 2024-04-26T04:45:34.756719+0000 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: created 2024-04-26T04:42:57.348674+0000 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: min_mon_release 17 (quincy) 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: election_strategy: 1 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.smithi077 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.smithi078 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:55:24.343 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: mgrmap e33: smithi077.hkvsor(active, since 45s), standbys: smithi078.navvuw 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: mgrmap e34: smithi077.hkvsor(active, since 45s), standbys: smithi078.navvuw 2024-04-26T04:55:24.344 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:24 smithi077 ceph-mon[91833]: from='mgr.14764 ' entity='' 2024-04-26T04:55:31.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:55:31.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:55:31.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:55:31.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:55:31.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:55:31.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:30 smithi077 ceph-mon[91833]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw restarted 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: Standby manager daemon smithi078.navvuw started 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/crt"}]: dispatch 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi078.navvuw/key"}]: dispatch 2024-04-26T04:55:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:30 smithi078 ceph-mon[33746]: from='mgr.? 172.21.15.78:0/361178601' entity='mgr.smithi078.navvuw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: mgrmap e35: smithi077.hkvsor(active, since 52s), standbys: smithi078.navvuw 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: Active manager daemon smithi077.hkvsor restarted 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: mgrmap e36: smithi077.hkvsor(active, starting, since 0.0151171s), standbys: smithi078.navvuw 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:32.198 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:55:32.199 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:32.200 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: mgrmap e35: smithi077.hkvsor(active, since 52s), standbys: smithi078.navvuw 2024-04-26T04:55:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: Active manager daemon smithi077.hkvsor restarted 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: Activating manager daemon smithi077.hkvsor 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: mgrmap e36: smithi077.hkvsor(active, starting, since 0.0151171s), standbys: smithi078.navvuw 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi077.hkvsor", "id": "smithi077.hkvsor"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr metadata", "who": "smithi078.navvuw", "id": "smithi078.navvuw"}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:55:32.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: Manager daemon smithi077.hkvsor is now available 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi077.hkvsor/trash_purge_schedule"}]: dispatch 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:32.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:31 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: [26/Apr/2024:04:55:31] ENGINE Bus STARTING 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: [26/Apr/2024:04:55:31] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: [26/Apr/2024:04:55:31] ENGINE Serving on http://172.21.15.77:8765 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: [26/Apr/2024:04:55:31] ENGINE Bus STARTED 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: mgrmap e37: smithi077.hkvsor(active, since 1.0175s), standbys: smithi078.navvuw 2024-04-26T04:55:33.230 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:32 smithi078 ceph-mon[33746]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: [26/Apr/2024:04:55:31] ENGINE Bus STARTING 2024-04-26T04:55:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: [26/Apr/2024:04:55:31] ENGINE Serving on https://172.21.15.77:7150 2024-04-26T04:55:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: [26/Apr/2024:04:55:31] ENGINE Serving on http://172.21.15.77:8765 2024-04-26T04:55:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: [26/Apr/2024:04:55:31] ENGINE Bus STARTED 2024-04-26T04:55:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: mgrmap e37: smithi077.hkvsor(active, since 1.0175s), standbys: smithi078.navvuw 2024-04-26T04:55:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:32 smithi077 ceph-mon[91833]: pgmap v3: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:34.143 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:33 smithi077 ceph-mon[91833]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T04:55:34.143 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:33 smithi077 ceph-mon[91833]: pgmap v4: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:34.144 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:33 smithi077 ceph-mon[91833]: mgrmap e38: smithi077.hkvsor(active, since 2s), standbys: smithi078.navvuw 2024-04-26T04:55:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:33 smithi078 ceph-mon[33746]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T04:55:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:33 smithi078 ceph-mon[33746]: pgmap v4: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:34.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:33 smithi078 ceph-mon[33746]: mgrmap e38: smithi077.hkvsor(active, since 2s), standbys: smithi078.navvuw 2024-04-26T04:55:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:35 smithi077 ceph-mon[91833]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:35 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:35 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:35 smithi078 ceph-mon[33746]: pgmap v5: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:36 smithi078 ceph-mon[33746]: mgrmap e39: smithi077.hkvsor(active, since 4s), standbys: smithi078.navvuw 2024-04-26T04:55:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:36 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:36 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:36 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:36 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:36 smithi077 ceph-mon[91833]: mgrmap e39: smithi077.hkvsor(active, since 4s), standbys: smithi078.navvuw 2024-04-26T04:55:36.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:36.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:37 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:37 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:38.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:37 smithi078 ceph-mon[33746]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-04-26T04:55:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:37 smithi077 ceph-mon[91833]: pgmap v6: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-04-26T04:55:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:55:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:55:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:39 smithi077 ceph-mon[91833]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.conf 2024-04-26T04:55:40.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:39 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-26T04:55:41.055 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:55:41.055 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:41.056 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:55:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:55:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:55:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:55:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:55:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: Updating smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:40 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:55:42.084 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:41 smithi077 ceph-mon[91833]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:55:42.084 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:42.084 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:42.084 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:42.084 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:42.208 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:41 smithi078 ceph-mon[33746]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T04:55:42.209 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:41 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:42.209 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:41 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:42.209 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:41 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:42.209 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:41 smithi078 ceph-mon[33746]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:42 smithi077 ceph-mon[91833]: Upgrade: Updating mon.smithi078 2024-04-26T04:55:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:42 smithi077 ceph-mon[91833]: Deploying daemon mon.smithi078 on smithi078 2024-04-26T04:55:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:42 smithi077 ceph-mon[91833]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-26T04:55:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:42 smithi078 ceph-mon[33746]: Upgrade: Updating mon.smithi078 2024-04-26T04:55:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:42 smithi078 ceph-mon[33746]: Deploying daemon mon.smithi078 on smithi078 2024-04-26T04:55:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:42 smithi078 ceph-mon[33746]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-26T04:55:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 systemd[1]: Stopping Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:55:43.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078[33725]: 2024-04-26T04:55:43.391+0000 7f4f7ccb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi078 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:43.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078[33725]: 2024-04-26T04:55:43.391+0000 7f4f7ccb9700 -1 mon.smithi078@1(peon) e2 *** Got Signal Terminated *** 2024-04-26T04:55:43.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 podman[79524]: 2024-04-26 04:55:43.505004978 +0000 UTC m=+0.211099792 container died 1b87cf9e8e17dc0f4861b5c02ef90ac8884e74df13307657f99a88871f9e7035 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-26T04:55:44.102 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 podman[79524]: 2024-04-26 04:55:43.875935456 +0000 UTC m=+0.582030276 container cleanup 1b87cf9e8e17dc0f4861b5c02ef90ac8884e74df13307657f99a88871f9e7035 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=quincy-c61472c) 2024-04-26T04:55:44.102 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:43 smithi078 bash[79524]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078 2024-04-26T04:55:44.440 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 podman[79538]: 2024-04-26 04:55:44.102265037 +0000 UTC m=+0.600274136 container remove 1b87cf9e8e17dc0f4861b5c02ef90ac8884e74df13307657f99a88871f9e7035 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T04:55:44.440 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service: Deactivated successfully. 2024-04-26T04:55:44.440 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 systemd[1]: Stopped Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:55:44.440 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service: Consumed 6.001s CPU time. 2024-04-26T04:55:44.440 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 systemd[1]: Starting Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:55:44.750 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 podman[79670]: 2024-04-26 04:55:44.65312587 +0000 UTC m=+0.113526379 container create ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb) 2024-04-26T04:55:44.750 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 podman[79670]: 2024-04-26 04:55:44.558181452 +0000 UTC m=+0.018581973 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 podman[79670]: 2024-04-26 04:55:44.847670817 +0000 UTC m=+0.308071331 container init ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 podman[79670]: 2024-04-26 04:55:44.851081981 +0000 UTC m=+0.311482491 container start ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev), process ceph-mon, pid 2 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: pidfile_write: ignore empty --pid-file 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: load: jerasure load: lrc 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: RocksDB version: 7.9.2 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Git sha 0 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Compile date 2024-04-25 16:43:49 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: DB SUMMARY 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: DB Session ID: 7SN5WUMTPQBVW4RMZYUG 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:55:45.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi078/store.db dir, Total Num: 1, files: 000027.sst 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi078/store.db: 000025.log size: 872699 ; 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.env: 0x55c7f8e88c60 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.info_log: 0x55c7fa2e1960 2024-04-26T04:55:45.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.statistics: (nil) 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.use_fsync: 0 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:55:45.271 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.db_log_dir: 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.wal_dir: 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:55:45.272 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.write_buffer_manager: 0x55c7fa2b41e0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.unordered_write: 0 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.row_cache: None 2024-04-26T04:55:45.273 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.wal_filter: None 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.wal_compression: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:55:45.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:55:45.275 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_open_files: -1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Compression algorithms supported: 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kZSTD supported: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:55:45.276 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi078/store.db/MANIFEST-000009 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.merge_operator: 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_filter: None 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c7fa2e1a60) 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:55:45.277 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: index_type: 0 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: data_block_index_type: 0 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: index_shortening: 1 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: checksum: 4 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: no_block_cache: 0 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_cache: 0x55c7fa2e5090 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_cache_options: 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: capacity : 536870912 2024-04-26T04:55:45.278 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: num_shard_bits : 4 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: strict_capacity_limit : 0 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_cache_compressed: (nil) 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: persistent_cache: (nil) 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_size: 4096 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_size_deviation: 10 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_restart_interval: 16 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: index_block_restart_interval: 1 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: metadata_block_size: 4096 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: partition_filters: 0 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: use_delta_encoding: 1 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: filter_policy: bloomfilter 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: whole_key_filtering: 1 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: verify_compression: 0 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:55:45.279 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: format_version: 5 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: enable_index_compression: 1 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: block_align: 0 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: max_auto_readahead_size: 262144 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: prepopulate_block_cache: 0 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: initial_auto_readahead_size: 8192 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression: NoCompression 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.num_levels: 7 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:55:45.280 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:55:45.281 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:55:45.282 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:55:45.283 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:55:45.284 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.ttl: 2592000 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:55:45.285 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi078/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12348, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 77f8dcd4-295a-41d2-bfc5-d522afeef42d 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107344891063, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-26T04:55:45.286 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107344894033, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 529305, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12341, "largest_seqno": 12575, "table_properties": {"data_size": 527190, "index_size": 783, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2630, "raw_average_key_size": 24, "raw_value_size": 524626, "raw_average_value_size": 4857, "num_data_blocks": 34, "num_entries": 108, "num_filter_entries": 108, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714107344, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "77f8dcd4-295a-41d2-bfc5-d522afeef42d", "db_session_id": "7SN5WUMTPQBVW4RMZYUG", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714107344894120, "job": 1, "event": "recovery_finished"} 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi078/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c7fa3d6000 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: DB pointer 0x55c7fa3c0000 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: ** DB Stats ** 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:45.287 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: L0 1/0 516.90 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 311.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: L6 1/0 7.86 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Sum 2/0 8.37 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 311.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 311.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.288 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 311.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:55:45.289 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Cumulative compaction: 0.00 GB write, 32.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Interval compaction: 0.00 GB write, 32.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Block cache BinnedLRUCache@0x55c7fa2e5090#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.95 KB,0.000181794%) Misc(1,0.00 KB,0%) 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: starting mon.smithi078 rank 1 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi078 fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???) e2 preinit fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).mds e14 new map 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).mds e14 print_map 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: e14 2024-04-26T04:55:45.290 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: legacy client fscid: 1 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Filesystem 'cephfs' (1) 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: fs_name cephfs 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: epoch 13 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: tableserver 0 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: root 0 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: session_timeout 60 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: session_autoclose 300 2024-04-26T04:55:45.291 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: max_file_size 1099511627776 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: max_xattr_size 65536 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: required_client_features {} 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: last_failure 0 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: last_failure_osd_epoch 0 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: max_mds 1 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: in 0 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: up {0=14458} 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: failed 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: damaged 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: stopped 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: data_pools [3] 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: metadata_pool 2 2024-04-26T04:55:45.292 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: inline_data enabled 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: balancer 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: bal_rank_mask -1 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: standby_count_wanted 1 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: qdb_cluster leader: 0 members: 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: [mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: Standby daemons: 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: [mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: [mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout: [mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-26T04:55:45.293 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:45.294 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:45.294 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:55:45.294 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 ceph-mon[79689]: mon.smithi078@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-26T04:55:45.294 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 bash[79670]: ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 2024-04-26T04:55:45.294 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:44 smithi078 systemd[1]: Started Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: mgrmap e39: smithi077.hkvsor(active, since 14s), standbys: smithi078.navvuw 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: mon.smithi078 calling monitor election 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: mon.smithi077 calling monitor election 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:46.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: mon.smithi077 is new leader, mons smithi077,smithi078 in quorum (ranks 0,1) 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: monmap epoch 3 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: last_changed 2024-04-26T04:55:45.382061+0000 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: created 2024-04-26T04:42:57.348674+0000 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: min_mon_release 19 (squid) 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: election_strategy: 1 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.smithi077 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.smithi078 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: mgrmap e39: smithi077.hkvsor(active, since 14s), standbys: smithi078.navvuw 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:46.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:46.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:46 smithi078 ceph-mon[79689]: pgmap v11: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: mgrmap e39: smithi077.hkvsor(active, since 14s), standbys: smithi078.navvuw 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: mon.smithi078 calling monitor election 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: mon.smithi077 calling monitor election 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi077"}]: dispatch 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: mon.smithi077 is new leader, mons smithi077,smithi078 in quorum (ranks 0,1) 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: monmap epoch 3 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: fsid 3c1ee652-0387-11ef-bc93-c7b262605968 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: last_changed 2024-04-26T04:55:45.382061+0000 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: created 2024-04-26T04:42:57.348674+0000 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: min_mon_release 19 (squid) 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: election_strategy: 1 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.smithi077 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: 1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.smithi078 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:55:46.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: mgrmap e39: smithi077.hkvsor(active, since 14s), standbys: smithi078.navvuw 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mon metadata", "id": "smithi078"}]: dispatch 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:55:46.745 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:46 smithi077 ceph-mon[91833]: pgmap v11: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:55:49.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:49 smithi077 ceph-mon[91833]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:49.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:49 smithi078 ceph-mon[79689]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:50.514 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:50 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:50.514 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:50 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:50.514 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:50 smithi077 ceph-mon[91833]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:50.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:50 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:50.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:50 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:50.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:50 smithi078 ceph-mon[79689]: pgmap v13: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- 172.21.15.77:0/745156290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fbd40 msgr2=0x7f8ed40fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 --2- 172.21.15.77:0/745156290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fbd40 0x7f8ed40fc150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4004940 tx=0x7f8ec4009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- 172.21.15.77:0/745156290 shutdown_connections 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 --2- 172.21.15.77:0/745156290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fcff0 0x7f8ed40fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 --2- 172.21.15.77:0/745156290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fbd40 0x7f8ed40fc150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- 172.21.15.77:0/745156290 >> 172.21.15.77:0/745156290 conn(0x7f8ed40f74c0 msgr2=0x7f8ed40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- 172.21.15.77:0/745156290 shutdown_connections 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- 172.21.15.77:0/745156290 wait complete. 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 Processor -- start 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.578+0000 7f8ed9668700 1 -- start start 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed9668700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fbd40 0x7f8ed41106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed9668700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed9668700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4066c70 con 0x7f8ed40fcff0 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed9668700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4066db0 con 0x7f8ed40fbd40 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41004/0 (socket says 172.21.15.77:41004) 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 -- 172.21.15.77:0/1861863584 learned_addr learned my addr 172.21.15.77:0/1861863584 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed2ffd700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fbd40 0x7f8ed41106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 -- 172.21.15.77:0/1861863584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fbd40 msgr2=0x7f8ed41106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fbd40 0x7f8ed41106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 -- 172.21.15.77:0/1861863584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4005040 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ed27fc700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800b7c0 tx=0x7f8ec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.579+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec800f6f0 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.580+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ec800fd20 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.580+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed4113290 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.580+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec80103a0 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.580+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed4113850 con 0x7f8ed40fcff0 2024-04-26T04:55:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.581+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed405f5c0 con 0x7f8ed40fcff0 2024-04-26T04:55:52.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.581+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8ec8011400 con 0x7f8ed40fcff0 2024-04-26T04:55:52.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.581+0000 7f8ebb7fe700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 0x7f8ebc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.582+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ec808da60 con 0x7f8ed40fcff0 2024-04-26T04:55:52.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.582+0000 7f8ed2ffd700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 0x7f8ebc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.582+0000 7f8ed2ffd700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 0x7f8ebc070710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b970 tx=0x7f8ec4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:52.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.587+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8ec8056da0 con 0x7f8ed40fcff0 2024-04-26T04:55:52.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.820+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ed41012c0 con 0x7f8ebc06e260 2024-04-26T04:55:52.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.822+0000 7f8ebb7fe700 1 -- 172.21.15.77:0/1861863584 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8ed41012c0 con 0x7f8ebc06e260 2024-04-26T04:55:52.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 msgr2=0x7f8ebc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 0x7f8ebc070710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b970 tx=0x7f8ec4009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:52.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 msgr2=0x7f8ed4112c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800b7c0 tx=0x7f8ec8009250 comp rx=0 tx=0).stop 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 shutdown_connections 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8ebc06e260 0x7f8ebc070710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ed40fbd40 0x7f8ed41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 --2- 172.21.15.77:0/1861863584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ed40fcff0 0x7f8ed4112c30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 >> 172.21.15.77:0/1861863584 conn(0x7f8ed40f74c0 msgr2=0x7f8ed40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 shutdown_connections 2024-04-26T04:55:52.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.824+0000 7f8ed9668700 1 -- 172.21.15.77:0/1861863584 wait complete. 2024-04-26T04:55:52.835 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 -- 172.21.15.77:0/2476415172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c1002e0 msgr2=0x7f975c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 --2- 172.21.15.77:0/2476415172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c1002e0 0x7f975c1026c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9748004940 tx=0x7f9748009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 -- 172.21.15.77:0/2476415172 shutdown_connections 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 --2- 172.21.15.77:0/2476415172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c1002e0 0x7f975c1026c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 --2- 172.21.15.77:0/2476415172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c0fd9c0 0x7f975c0ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 -- 172.21.15.77:0/2476415172 >> 172.21.15.77:0/2476415172 conn(0x7f975c0f7440 msgr2=0x7f975c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 -- 172.21.15.77:0/2476415172 shutdown_connections 2024-04-26T04:55:52.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.952+0000 7f9762fa3700 1 -- 172.21.15.77:0/2476415172 wait complete. 2024-04-26T04:55:52.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 Processor -- start 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 -- start start 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 0x7f975c0fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0fbe60 con 0x7f975c0fd9c0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9762fa3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0fbfa0 con 0x7f975c1002e0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.953+0000 7f9760d3f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41024/0 (socket says 172.21.15.77:41024) 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 -- 172.21.15.77:0/3566782784 learned_addr learned my addr 172.21.15.77:0/3566782784 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f975bfff700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 0x7f975c0fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 -- 172.21.15.77:0/3566782784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 msgr2=0x7f975c0fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 0x7f975c0fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 -- 172.21.15.77:0/3566782784 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748005040 con 0x7f975c0fd9c0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f975bfff700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 0x7f975c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9760d3f700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9750002a50 tx=0x7f97500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:52.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9750007cc0 con 0x7f975c0fd9c0 2024-04-26T04:55:52.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c0fc280 con 0x7f975c0fd9c0 2024-04-26T04:55:52.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f975000f040 con 0x7f975c0fd9c0 2024-04-26T04:55:52.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9750009630 con 0x7f975c0fd9c0 2024-04-26T04:55:52.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.954+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c115a30 con 0x7f975c0fd9c0 2024-04-26T04:55:52.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.955+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c05f5c0 con 0x7f975c0fd9c0 2024-04-26T04:55:52.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.957+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9750009790 con 0x7f975c0fd9c0 2024-04-26T04:55:52.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.957+0000 7f97597fa700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 0x7f974c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:52.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.958+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f975008ded0 con 0x7f975c0fd9c0 2024-04-26T04:55:52.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.958+0000 7f975bfff700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 0x7f974c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:52.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.958+0000 7f975bfff700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 0x7f974c070770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f974800b3b0 tx=0x7f9748009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:52.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:52.961+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9750057100 con 0x7f975c0fd9c0 2024-04-26T04:55:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.193+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f975c0f8f80 con 0x7f974c06e2c0 2024-04-26T04:55:53.195 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:53.195 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:53.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.194+0000 7f97597fa700 1 -- 172.21.15.77:0/3566782784 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f975c0f8f80 con 0x7f974c06e2c0 2024-04-26T04:55:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.196+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 msgr2=0x7f974c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.196+0000 7f9762fa3700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 0x7f974c070770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f974800b3b0 tx=0x7f9748009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.196+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 msgr2=0x7f975c0fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.196+0000 7f9762fa3700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9750002a50 tx=0x7f97500046b0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 shutdown_connections 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f974c06e2c0 0x7f974c070770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c0fd9c0 0x7f975c0fd3b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 --2- 172.21.15.77:0/3566782784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f975c1002e0 0x7f975c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 >> 172.21.15.77:0/3566782784 conn(0x7f975c0f7440 msgr2=0x7f975c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 shutdown_connections 2024-04-26T04:55:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.197+0000 7f9762fa3700 1 -- 172.21.15.77:0/3566782784 wait complete. 2024-04-26T04:55:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 -- 172.21.15.77:0/3550180635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c1002e0 msgr2=0x7f938c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 --2- 172.21.15.77:0/3550180635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c1002e0 0x7f938c1026c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9378004940 tx=0x7f9378009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 -- 172.21.15.77:0/3550180635 shutdown_connections 2024-04-26T04:55:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 --2- 172.21.15.77:0/3550180635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c1002e0 0x7f938c1026c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 --2- 172.21.15.77:0/3550180635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c0fd9c0 0x7f938c0ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 -- 172.21.15.77:0/3550180635 >> 172.21.15.77:0/3550180635 conn(0x7f938c0f7440 msgr2=0x7f938c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 -- 172.21.15.77:0/3550180635 shutdown_connections 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.302+0000 7f9392b37700 1 -- 172.21.15.77:0/3550180635 wait complete. 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 Processor -- start 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 -- start start 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 0x7f938c0fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0fbef0 con 0x7f938c0fd9c0 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f9392b37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0fc030 con 0x7f938c1002e0 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41044/0 (socket says 172.21.15.77:41044) 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 -- 172.21.15.77:0/580905939 learned_addr learned my addr 172.21.15.77:0/580905939 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f938bfff700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 0x7f938c0fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 -- 172.21.15.77:0/580905939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 msgr2=0x7f938c0fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 0x7f938c0fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f93908d3700 1 -- 172.21.15.77:0/580905939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378005040 con 0x7f938c0fd9c0 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.303+0000 7f938bfff700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 0x7f938c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f93908d3700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f938000b790 tx=0x7f9380009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f938000f6f0 con 0x7f938c0fd9c0 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f938000fd20 con 0x7f938c0fd9c0 2024-04-26T04:55:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c0fc310 con 0x7f938c0fd9c0 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c115ae0 con 0x7f938c0fd9c0 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.304+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93800103a0 con 0x7f938c0fd9c0 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.305+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c05f5c0 con 0x7f938c0fd9c0 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.305+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9380011400 con 0x7f938c0fd9c0 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.306+0000 7f93897fa700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 0x7f937c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.306+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f938008da20 con 0x7f938c0fd9c0 2024-04-26T04:55:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.306+0000 7f938bfff700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 0x7f937c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.306+0000 7f938bfff700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 0x7f937c070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f938c0fd990 tx=0x7f9378009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.310+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9380056d60 con 0x7f938c0fd9c0 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:55:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:55:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:55:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:53.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.521+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f938c0f8f80 con 0x7f937c06e260 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.531+0000 7f93897fa700 1 -- 172.21.15.77:0/580905939 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f938c0f8f80 con 0x7f937c06e260 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (2m) 17s ago 11m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (11m) 17s ago 11m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af 5bae0c6f56ec 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (10m) 5s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 17s ago 11m - - 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (5m) 17s ago 5m 154M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (5m) 17s ago 5m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:55:53.533 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (5m) 5s ago 5m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (5m) 5s ago 5m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (98s) 17s ago 12m 474M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (45s) 5s ago 10m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (31s) 17s ago 12m 27.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (8s) 5s ago 10m 18.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (3m) 17s ago 11m 15.8M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (2m) 5s ago 10m 16.5M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (9m) 17s ago 9m 203M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (9m) 17s ago 9m 213M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (8m) 17s ago 8m 178M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (8m) 5s ago 8m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (7m) 5s ago 7m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (7m) 5s ago 7m 192M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:55:53.534 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (50s) 17s ago 10m 52.3M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:55:53.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 msgr2=0x7f937c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 0x7f937c070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f938c0fd990 tx=0x7f9378009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:53.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 msgr2=0x7f938c0fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f938000b790 tx=0x7f9380009220 comp rx=0 tx=0).stop 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 shutdown_connections 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937c06e260 0x7f937c070710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f938c0fd9c0 0x7f938c0fd3b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 --2- 172.21.15.77:0/580905939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938c1002e0 0x7f938c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 >> 172.21.15.77:0/580905939 conn(0x7f938c0f7440 msgr2=0x7f938c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 shutdown_connections 2024-04-26T04:55:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.534+0000 7f9392b37700 1 -- 172.21.15.77:0/580905939 wait complete. 2024-04-26T04:55:53.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.632+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/2083168212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fcf70 msgr2=0x7f6a980fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.632+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/2083168212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fcf70 0x7f6a980fd3e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c004940 tx=0x7f6a8c009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/2083168212 shutdown_connections 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/2083168212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fcf70 0x7f6a980fd3e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/2083168212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fbcc0 0x7f6a980fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/2083168212 >> 172.21.15.77:0/2083168212 conn(0x7f6a980f7440 msgr2=0x7f6a980f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/2083168212 shutdown_connections 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/2083168212 wait complete. 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 Processor -- start 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- start start 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 0x7f6a981126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98112be0 con 0x7f6a980fbcc0 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9ce8f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98112d20 con 0x7f6a980fcf70 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41056/0 (socket says 172.21.15.77:41056) 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.633+0000 7f6a9659c700 1 -- 172.21.15.77:0/3609128568 learned_addr learned my addr 172.21.15.77:0/3609128568 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a95d9b700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 0x7f6a981126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9659c700 1 -- 172.21.15.77:0/3609128568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 msgr2=0x7f6a981126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9659c700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 0x7f6a981126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9659c700 1 -- 172.21.15.77:0/3609128568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c005040 con 0x7f6a980fbcc0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a95d9b700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 0x7f6a981126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9659c700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800eb40 tx=0x7f6a8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88007cc0 con 0x7f6a980fbcc0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a8800f040 con 0x7f6a980fbcc0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a98112f50 con 0x7f6a980fbcc0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88009580 con 0x7f6a980fbcc0 2024-04-26T04:55:53.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.634+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98113400 con 0x7f6a980fbcc0 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.635+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9804c710 con 0x7f6a980fbcc0 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.636+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6a88009840 con 0x7f6a980fbcc0 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.636+0000 7f6a86ffd700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 0x7f6a80070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.636+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6a8808da70 con 0x7f6a980fbcc0 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.636+0000 7f6a95d9b700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 0x7f6a80070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:53.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.636+0000 7f6a95d9b700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 0x7f6a80070720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000c00 tx=0x7f6a8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:53.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.641+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a8805a800 con 0x7f6a980fbcc0 2024-04-26T04:55:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.969+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a981138d0 con 0x7f6a980fbcc0 2024-04-26T04:55:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.969+0000 7f6a86ffd700 1 -- 172.21.15.77:0/3609128568 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f6a8805a390 con 0x7f6a980fbcc0 2024-04-26T04:55:53.971 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:55:53.971 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-26T04:55:53.972 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 msgr2=0x7f6a80070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 0x7f6a80070720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000c00 tx=0x7f6a8c009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 msgr2=0x7f6a98066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:53.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800eb40 tx=0x7f6a8800b040 comp rx=0 tx=0).stop 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 shutdown_connections 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6a8006e270 0x7f6a80070720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.971+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a980fbcc0 0x7f6a98066cd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.972+0000 7f6a9ce8f700 1 --2- 172.21.15.77:0/3609128568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a980fcf70 0x7f6a981126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.972+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 >> 172.21.15.77:0/3609128568 conn(0x7f6a980f7440 msgr2=0x7f6a980f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.972+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 shutdown_connections 2024-04-26T04:55:53.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:53.972+0000 7f6a9ce8f700 1 -- 172.21.15.77:0/3609128568 wait complete. 2024-04-26T04:55:54.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.075+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/1762138465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fdfd0 msgr2=0x7f4ed00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.075+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/1762138465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00fe440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4004940 tx=0x7f4ec4009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:54.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/1762138465 shutdown_connections 2024-04-26T04:55:54.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/1762138465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00fe440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/1762138465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fce60 0x7f4ed00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/1762138465 >> 172.21.15.77:0/1762138465 conn(0x7f4ed00f8520 msgr2=0x7f4ed00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:54.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/1762138465 shutdown_connections 2024-04-26T04:55:54.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/1762138465 wait complete. 2024-04-26T04:55:54.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 Processor -- start 2024-04-26T04:55:54.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- start start 2024-04-26T04:55:54.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00707f0 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.076+0000 7f4ed4ae2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0070960 con 0x7f4ed00fdfd0 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41076/0 (socket says 172.21.15.77:41076) 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 -- 172.21.15.77:0/91339060 learned_addr learned my addr 172.21.15.77:0/91339060 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 -- 172.21.15.77:0/91339060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fdfd0 msgr2=0x7f4ed00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ecdd9b700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 -- 172.21.15.77:0/91339060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4005040 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ece59c700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b790 tx=0x7f4ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec000f770 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed0070c40 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ec000fda0 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec000d310 con 0x7f4ed00fce60 2024-04-26T04:55:54.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.077+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed0115a50 con 0x7f4ed00fce60 2024-04-26T04:55:54.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.078+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed004c710 con 0x7f4ed00fce60 2024-04-26T04:55:54.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.079+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4ec0010420 con 0x7f4ed00fce60 2024-04-26T04:55:54.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.079+0000 7f4ebeffd700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 0x7f4eb8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.079+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4ec008dc70 con 0x7f4ed00fce60 2024-04-26T04:55:54.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.079+0000 7f4ecdd9b700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 0x7f4eb8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.079+0000 7f4ecdd9b700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 0x7f4eb8070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b970 tx=0x7f4ec4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.084+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4ec005aae0 con 0x7f4ed00fce60 2024-04-26T04:55:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: Reconfiguring mon.smithi077 (monmap changed)... 2024-04-26T04:55:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: Reconfiguring daemon mon.smithi077 on smithi077 2024-04-26T04:55:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:54 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3609128568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:54.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.338+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ed0115be0 con 0x7f4ed00fce60 2024-04-26T04:55:54.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.339+0000 7f4ebeffd700 1 -- 172.21.15.77:0/91339060 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f4ec005a670 con 0x7f4ed00fce60 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:55:54.341 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:55:54.342 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:54.343 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 msgr2=0x7f4eb8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 0x7f4eb8070720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b970 tx=0x7f4ec4009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 msgr2=0x7f4ed0071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b790 tx=0x7f4ec0009250 comp rx=0 tx=0).stop 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 shutdown_connections 2024-04-26T04:55:54.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4eb806e270 0x7f4eb8070720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ed00fce60 0x7f4ed0071c00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 --2- 172.21.15.77:0/91339060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ed00fdfd0 0x7f4ed00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 >> 172.21.15.77:0/91339060 conn(0x7f4ed00f8520 msgr2=0x7f4ed00f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 shutdown_connections 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.341+0000 7f4ed4ae2700 1 -- 172.21.15.77:0/91339060 wait complete. 2024-04-26T04:55:54.345 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:55:54.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- 172.21.15.77:0/2312435041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fcf70 msgr2=0x7f29980fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 --2- 172.21.15.77:0/2312435041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fcf70 0x7f29980fd3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2994004940 tx=0x7f2994009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- 172.21.15.77:0/2312435041 shutdown_connections 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 --2- 172.21.15.77:0/2312435041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fcf70 0x7f29980fd3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 --2- 172.21.15.77:0/2312435041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fbcc0 0x7f29980fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- 172.21.15.77:0/2312435041 >> 172.21.15.77:0/2312435041 conn(0x7f29980f7420 msgr2=0x7f29980f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- 172.21.15.77:0/2312435041 shutdown_connections 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- 172.21.15.77:0/2312435041 wait complete. 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 Processor -- start 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.444+0000 7f29a06eb700 1 -- start start 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fcf70 0x7f2998114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299810e520 con 0x7f29980fbcc0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2998115080 con 0x7f29980fcf70 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41100/0 (socket says 172.21.15.77:41100) 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 -- 172.21.15.77:0/3599736513 learned_addr learned my addr 172.21.15.77:0/3599736513 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299dc86700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fcf70 0x7f2998114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 -- 172.21.15.77:0/3599736513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fcf70 msgr2=0x7f2998114b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fcf70 0x7f2998114b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 -- 172.21.15.77:0/3599736513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994005040 con 0x7f29980fbcc0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f299e487700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f298c00b7c0 tx=0x7f298c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f298c00f6f0 con 0x7f29980fbcc0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29981152e0 con 0x7f29980fbcc0 2024-04-26T04:55:54.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f298c00fd20 con 0x7f29980fbcc0 2024-04-26T04:55:54.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f298c0103a0 con 0x7f29980fbcc0 2024-04-26T04:55:54.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.445+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29981158a0 con 0x7f29980fbcc0 2024-04-26T04:55:54.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.446+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299804c710 con 0x7f29980fbcc0 2024-04-26T04:55:54.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.447+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f298c011400 con 0x7f29980fbcc0 2024-04-26T04:55:54.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.447+0000 7f298affd700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 0x7f2984070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.447+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f298c08dad0 con 0x7f29980fbcc0 2024-04-26T04:55:54.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.447+0000 7f299dc86700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 0x7f2984070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.448+0000 7f299dc86700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 0x7f2984070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.452+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f298c05a940 con 0x7f29980fbcc0 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: Reconfiguring mon.smithi077 (monmap changed)... 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: Reconfiguring daemon mon.smithi077 on smithi077 2024-04-26T04:55:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:54 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3609128568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:55:54.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.676+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2998101250 con 0x7f298406e270 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.677+0000 7f298affd700 1 -- 172.21.15.77:0/3599736513 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2998101250 con 0x7f298406e270 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:55:54.679 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:55:54.680 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "4/21 daemons upgraded", 2024-04-26T04:55:54.680 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mon daemons", 2024-04-26T04:55:54.680 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:55:54.680 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 msgr2=0x7f2984070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 0x7f2984070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 msgr2=0x7f2998066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f298c00b7c0 tx=0x7f298c009250 comp rx=0 tx=0).stop 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 shutdown_connections 2024-04-26T04:55:54.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f298406e270 0x7f2984070720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29980fbcc0 0x7f2998066cf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 --2- 172.21.15.77:0/3599736513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29980fcf70 0x7f2998114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.679+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 >> 172.21.15.77:0/3599736513 conn(0x7f29980f7420 msgr2=0x7f29980f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:54.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.680+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 shutdown_connections 2024-04-26T04:55:54.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.680+0000 7f29a06eb700 1 -- 172.21.15.77:0/3599736513 wait complete. 2024-04-26T04:55:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.782+0000 7f0dff736700 1 -- 172.21.15.77:0/415203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 msgr2=0x7f0df80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.782+0000 7f0dff736700 1 --2- 172.21.15.77:0/415203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80fe440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0de8004940 tx=0x7f0de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- 172.21.15.77:0/415203 shutdown_connections 2024-04-26T04:55:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 --2- 172.21.15.77:0/415203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80fe440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 --2- 172.21.15.77:0/415203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- 172.21.15.77:0/415203 >> 172.21.15.77:0/415203 conn(0x7f0df80f8540 msgr2=0x7f0df80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- 172.21.15.77:0/415203 shutdown_connections 2024-04-26T04:55:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- 172.21.15.77:0/415203 wait complete. 2024-04-26T04:55:54.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 Processor -- start 2024-04-26T04:55:54.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- start start 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df8070860 con 0x7f0df80fdfd0 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.783+0000 7f0dff736700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df80709d0 con 0x7f0df80fce60 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41128/0 (socket says 172.21.15.77:41128) 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 -- 172.21.15.77:0/2282687963 learned_addr learned my addr 172.21.15.77:0/2282687963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 -- 172.21.15.77:0/2282687963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 msgr2=0x7f0df8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfd4d2700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df8071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 -- 172.21.15.77:0/2282687963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8005040 con 0x7f0df80fdfd0 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfd4d2700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df8071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:55:54.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dfccd1700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0de800b3b0 tx=0x7f0de8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de800e040 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0de8004290 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df8070c50 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de801c630 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.784+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df8115a30 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.785+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df805f5c0 con 0x7f0df80fdfd0 2024-04-26T04:55:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.786+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0de800bbf0 con 0x7f0df80fdfd0 2024-04-26T04:55:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.786+0000 7f0dedffb700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 0x7f0de4070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:55:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.786+0000 7f0dfd4d2700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 0x7f0de4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:55:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.787+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0de808f1e0 con 0x7f0df80fdfd0 2024-04-26T04:55:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.787+0000 7f0dfd4d2700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 0x7f0de4070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0df4007fd0 tx=0x7f0df4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:55:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:54.791+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0de8058410 con 0x7f0df80fdfd0 2024-04-26T04:55:55.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.079+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0df81058c0 con 0x7f0df80fdfd0 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.080+0000 7f0dedffb700 1 -- 172.21.15.77:0/2282687963 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+347 (secure 0 0 0) 0x7f0de805bad0 con 0x7f0df80fdfd0 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:55:55.082 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 msgr2=0x7f0de4070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 0x7f0de4070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0df4007fd0 tx=0x7f0df4009040 comp rx=0 tx=0).stop 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 msgr2=0x7f0df80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0de800b3b0 tx=0x7f0de8009e30 comp rx=0 tx=0).stop 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 shutdown_connections 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0de406e260 0x7f0de4070710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0df80fce60 0x7f0df8071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 --2- 172.21.15.77:0/2282687963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df80fdfd0 0x7f0df80702b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 >> 172.21.15.77:0/2282687963 conn(0x7f0df80f8540 msgr2=0x7f0df80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 shutdown_connections 2024-04-26T04:55:55.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:55:55.082+0000 7f0dff736700 1 -- 172.21.15.77:0/2282687963 wait complete. 2024-04-26T04:55:55.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:55 smithi077 ceph-mon[91833]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:55.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:55 smithi077 ceph-mon[91833]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:55.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:55 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/91339060' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:55:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:55 smithi078 ceph-mon[79689]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:55 smithi078 ceph-mon[79689]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:55 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/91339060' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:55:56.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:56.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2282687963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:55:56.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2282687963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi077.hkvsor", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:55:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: Reconfiguring mgr.smithi077.hkvsor (monmap changed)... 2024-04-26T04:55:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: Reconfiguring daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:55:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:57.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:57.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:55:57.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: Reconfiguring mgr.smithi077.hkvsor (monmap changed)... 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: Reconfiguring daemon mgr.smithi077.hkvsor on smithi077 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:55:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:58.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:58 smithi077 ceph-mon[91833]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:58.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:58 smithi077 ceph-mon[91833]: Reconfiguring crash.smithi077 (monmap changed)... 2024-04-26T04:55:58.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:58 smithi077 ceph-mon[91833]: Reconfiguring daemon crash.smithi077 on smithi077 2024-04-26T04:55:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:58 smithi078 ceph-mon[79689]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:58 smithi078 ceph-mon[79689]: Reconfiguring crash.smithi077 (monmap changed)... 2024-04-26T04:55:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:58 smithi078 ceph-mon[79689]: Reconfiguring daemon crash.smithi077 on smithi077 2024-04-26T04:55:59.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: Reconfiguring osd.0 (monmap changed)... 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.0 on smithi077 2024-04-26T04:55:59.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:55:59 smithi077 ceph-mon[91833]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: Reconfiguring osd.0 (monmap changed)... 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.0 on smithi077 2024-04-26T04:55:59.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:55:59 smithi078 ceph-mon[79689]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: Reconfiguring osd.1 (monmap changed)... 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.1 on smithi077 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: Reconfiguring osd.1 (monmap changed)... 2024-04-26T04:56:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:56:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:01.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.1 on smithi077 2024-04-26T04:56:01.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:01.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: Reconfiguring osd.2 (monmap changed)... 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:02.917 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:02 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.2 on smithi077 2024-04-26T04:56:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: Reconfiguring osd.2 (monmap changed)... 2024-04-26T04:56:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:56:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:02 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.2 on smithi077 2024-04-26T04:56:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:03 smithi078 ceph-mon[79689]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:04.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:04.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:04.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:04.067 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:03 smithi077 ceph-mon[91833]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:04.067 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:04.067 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:04.067 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:04.067 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:04.944 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:04 smithi077 ceph-mon[91833]: Reconfiguring mds.cephfs.smithi077.quaiyg (monmap changed)... 2024-04-26T04:56:05.127 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:04 smithi077 ceph-mon[91833]: Reconfiguring daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:56:05.128 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:04 smithi078 ceph-mon[79689]: Reconfiguring mds.cephfs.smithi077.quaiyg (monmap changed)... 2024-04-26T04:56:05.128 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:04 smithi078 ceph-mon[79689]: Reconfiguring daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:56:06.101 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:06.101 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:06.101 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:06.101 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: Reconfiguring mds.cephfs.smithi077.urxtgp (monmap changed)... 2024-04-26T04:56:06.101 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:06.102 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:06.102 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:05 smithi077 ceph-mon[91833]: Reconfiguring daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T04:56:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: Reconfiguring mds.cephfs.smithi077.urxtgp (monmap changed)... 2024-04-26T04:56:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:06.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:05 smithi078 ceph-mon[79689]: Reconfiguring daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T04:56:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:07 smithi078 ceph-mon[79689]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:08.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:08.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:08.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:08.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:08.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:07 smithi077 ceph-mon[91833]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:08.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:08.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:08.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:08.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:09 smithi078 ceph-mon[79689]: Reconfiguring crash.smithi078 (monmap changed)... 2024-04-26T04:56:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:09 smithi078 ceph-mon[79689]: Reconfiguring daemon crash.smithi078 on smithi078 2024-04-26T04:56:09.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:09 smithi077 ceph-mon[91833]: Reconfiguring crash.smithi078 (monmap changed)... 2024-04-26T04:56:09.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:09 smithi077 ceph-mon[91833]: Reconfiguring daemon crash.smithi078 on smithi078 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:56:10.140 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:10 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi078.navvuw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:56:10.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:10 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: Reconfiguring mgr.smithi078.navvuw (monmap changed)... 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: Reconfiguring daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:56:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:56:11.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:11.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: Reconfiguring mgr.smithi078.navvuw (monmap changed)... 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: Reconfiguring daemon mgr.smithi078.navvuw on smithi078 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:56:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:12 smithi078 ceph-mon[79689]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:12 smithi078 ceph-mon[79689]: Reconfiguring mon.smithi078 (monmap changed)... 2024-04-26T04:56:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:12 smithi078 ceph-mon[79689]: Reconfiguring daemon mon.smithi078 on smithi078 2024-04-26T04:56:12.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:12 smithi077 ceph-mon[91833]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:12.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:12 smithi077 ceph-mon[91833]: Reconfiguring mon.smithi078 (monmap changed)... 2024-04-26T04:56:12.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:12 smithi077 ceph-mon[91833]: Reconfiguring daemon mon.smithi078 on smithi078 2024-04-26T04:56:13.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:13.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:13.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: Reconfiguring osd.3 (monmap changed)... 2024-04-26T04:56:13.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:56:13.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:13.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.3 on smithi078 2024-04-26T04:56:13.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:13 smithi078 ceph-mon[79689]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:13.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:13.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:13.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: Reconfiguring osd.3 (monmap changed)... 2024-04-26T04:56:13.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:56:13.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:13.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.3 on smithi078 2024-04-26T04:56:13.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:13 smithi077 ceph-mon[91833]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: Reconfiguring osd.4 (monmap changed)... 2024-04-26T04:56:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:56:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:15 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.4 on smithi078 2024-04-26T04:56:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:15.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: Reconfiguring osd.4 (monmap changed)... 2024-04-26T04:56:15.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:56:15.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:15.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:15 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.4 on smithi078 2024-04-26T04:56:16.022 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:16.022 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:16.022 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:16.022 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:56:16.023 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:16.023 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:16.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:16.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:56:16.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:16.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:17.203 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:17 smithi078 ceph-mon[79689]: Reconfiguring osd.5 (monmap changed)... 2024-04-26T04:56:17.204 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:17 smithi078 ceph-mon[79689]: Reconfiguring daemon osd.5 on smithi078 2024-04-26T04:56:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:17 smithi077 ceph-mon[91833]: Reconfiguring osd.5 (monmap changed)... 2024-04-26T04:56:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:17 smithi077 ceph-mon[91833]: Reconfiguring daemon osd.5 on smithi078 2024-04-26T04:56:18.174 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:18 smithi078 ceph-mon[79689]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:18.174 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:18.174 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:18.174 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:18.174 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:18.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:18 smithi077 ceph-mon[91833]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:18.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:18.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:18.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:18.257 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:19.273 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: Reconfiguring mds.cephfs.smithi078.wkrgkp (monmap changed)... 2024-04-26T04:56:19.273 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: Reconfiguring daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T04:56:19.273 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:19.273 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:19.274 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:19.274 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:19.386 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: Reconfiguring mds.cephfs.smithi078.wkrgkp (monmap changed)... 2024-04-26T04:56:19.386 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: Reconfiguring daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T04:56:19.386 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:19.387 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:19.387 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:56:19.387 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:20.334 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:20 smithi078 ceph-mon[79689]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:20.334 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:20 smithi078 ceph-mon[79689]: Reconfiguring mds.cephfs.smithi078.nihewy (monmap changed)... 2024-04-26T04:56:20.334 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:20 smithi078 ceph-mon[79689]: Reconfiguring daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T04:56:20.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:20 smithi077 ceph-mon[91833]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:20.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:20 smithi077 ceph-mon[91833]: Reconfiguring mds.cephfs.smithi078.nihewy (monmap changed)... 2024-04-26T04:56:20.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:20 smithi077 ceph-mon[91833]: Reconfiguring daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T04:56:21.710 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all mon 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi077"}]: dispatch 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi077"}]': finished 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi078"}]: dispatch 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi078"}]': finished 2024-04-26T04:56:21.711 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:21 smithi077 ceph-mon[91833]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all mon 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi077"}]: dispatch 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi077"}]': finished 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi078"}]: dispatch 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi078"}]': finished 2024-04-26T04:56:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:21 smithi078 ceph-mon[79689]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: Upgrade: Updating crash.smithi077 (1/2) 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: Deploying daemon crash.smithi077 on smithi077 2024-04-26T04:56:23.328 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:23 smithi077 ceph-mon[91833]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: Upgrade: Updating crash.smithi077 (1/2) 2024-04-26T04:56:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi077", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: Deploying daemon crash.smithi077 on smithi077 2024-04-26T04:56:23.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:23 smithi078 ceph-mon[79689]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 -- 172.21.15.77:0/1476455722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fbce0 msgr2=0x7f76480fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 --2- 172.21.15.77:0/1476455722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fbce0 0x7f76480fc0f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7638004940 tx=0x7f7638009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 -- 172.21.15.77:0/1476455722 shutdown_connections 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 --2- 172.21.15.77:0/1476455722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fcf90 0x7f76480fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 --2- 172.21.15.77:0/1476455722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fbce0 0x7f76480fc0f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 -- 172.21.15.77:0/1476455722 >> 172.21.15.77:0/1476455722 conn(0x7f76480f7440 msgr2=0x7f76480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 -- 172.21.15.77:0/1476455722 shutdown_connections 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.196+0000 7f764d2eb700 1 -- 172.21.15.77:0/1476455722 wait complete. 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 Processor -- start 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 -- start start 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fbce0 0x7f7648110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648066c50 con 0x7f76480fcf90 2024-04-26T04:56:25.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.197+0000 7f764d2eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648066d90 con 0x7f76480fbce0 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47036/0 (socket says 172.21.15.77:47036) 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 -- 172.21.15.77:0/465685842 learned_addr learned my addr 172.21.15.77:0/465685842 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f7646ffd700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fbce0 0x7f7648110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 -- 172.21.15.77:0/465685842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fbce0 msgr2=0x7f7648110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fbce0 0x7f7648110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 -- 172.21.15.77:0/465685842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638005040 con 0x7f76480fcf90 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f76467fc700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b790 tx=0x7f763c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c00f770 con 0x7f76480fcf90 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76481132c0 con 0x7f76480fcf90 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.198+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763c00fda0 con 0x7f76480fcf90 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.199+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c00d310 con 0x7f76480fcf90 2024-04-26T04:56:25.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.199+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7648113850 con 0x7f76480fcf90 2024-04-26T04:56:25.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.199+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764805f5c0 con 0x7f76480fcf90 2024-04-26T04:56:25.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.200+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f763c010420 con 0x7f76480fcf90 2024-04-26T04:56:25.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.200+0000 7f762f7fe700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 0x7f7630070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.201+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f763c08dc70 con 0x7f76480fcf90 2024-04-26T04:56:25.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.201+0000 7f7646ffd700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 0x7f7630070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.201+0000 7f7646ffd700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 0x7f7630070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7638009fd0 tx=0x7f7638009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.209+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f763c056fb0 con 0x7f76480fcf90 2024-04-26T04:56:25.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.437+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7648101230 con 0x7f763006e260 2024-04-26T04:56:25.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.438+0000 7f762f7fe700 1 -- 172.21.15.77:0/465685842 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f7648101230 con 0x7f763006e260 2024-04-26T04:56:25.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.440+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 msgr2=0x7f7630070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.440+0000 7f764d2eb700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 0x7f7630070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7638009fd0 tx=0x7f7638009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:25.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.440+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 msgr2=0x7f7648112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.440+0000 7f764d2eb700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b790 tx=0x7f763c009250 comp rx=0 tx=0).stop 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 shutdown_connections 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f763006e260 0x7f7630070710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76480fbce0 0x7f7648110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 --2- 172.21.15.77:0/465685842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76480fcf90 0x7f7648112c60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 >> 172.21.15.77:0/465685842 conn(0x7f76480f7440 msgr2=0x7f76480f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 shutdown_connections 2024-04-26T04:56:25.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.441+0000 7f764d2eb700 1 -- 172.21.15.77:0/465685842 wait complete. 2024-04-26T04:56:25.451 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.543+0000 7f4344f2e700 1 -- 172.21.15.77:0/3854889965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 msgr2=0x7f43400fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.543+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3854889965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f43400fd3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4334004940 tx=0x7f4334009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- 172.21.15.77:0/3854889965 shutdown_connections 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3854889965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f43400fd3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3854889965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f43400fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- 172.21.15.77:0/3854889965 >> 172.21.15.77:0/3854889965 conn(0x7f43400f7430 msgr2=0x7f43400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- 172.21.15.77:0/3854889965 shutdown_connections 2024-04-26T04:56:25.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- 172.21.15.77:0/3854889965 wait complete. 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 Processor -- start 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- start start 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f434010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434010d440 con 0x7f43400fcf60 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.544+0000 7f4344f2e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434010dc90 con 0x7f43400fbcb0 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47040/0 (socket says 172.21.15.77:47040) 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 -- 172.21.15.77:0/3642444799 learned_addr learned my addr 172.21.15.77:0/3642444799 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433e59c700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f434010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 -- 172.21.15.77:0/3642444799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 msgr2=0x7f434010c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f434010c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 -- 172.21.15.77:0/3642444799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334005040 con 0x7f43400fcf60 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433e59c700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f434010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f433dd9b700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f433400b3b0 tx=0x7f4334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433400e040 con 0x7f43400fcf60 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f433400bcc0 con 0x7f43400fcf60 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434010df10 con 0x7f43400fcf60 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433401c630 con 0x7f43400fcf60 2024-04-26T04:56:25.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.545+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434010e450 con 0x7f43400fcf60 2024-04-26T04:56:25.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.546+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434005f5c0 con 0x7f43400fcf60 2024-04-26T04:56:25.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.547+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4334013980 con 0x7f43400fcf60 2024-04-26T04:56:25.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.547+0000 7f432effd700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 0x7f4328070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.547+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f433408e9b0 con 0x7f43400fcf60 2024-04-26T04:56:25.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.547+0000 7f433e59c700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 0x7f4328070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.548+0000 7f433e59c700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 0x7f4328070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4330009fd0 tx=0x7f4330009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.552+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f433405b710 con 0x7f43400fcf60 2024-04-26T04:56:25.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.782+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f434010e7d0 con 0x7f432806e260 2024-04-26T04:56:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.783+0000 7f432effd700 1 -- 172.21.15.77:0/3642444799 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f434010e7d0 con 0x7f432806e260 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 msgr2=0x7f4328070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 0x7f4328070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4330009fd0 tx=0x7f4330009ee0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 msgr2=0x7f434010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f433400b3b0 tx=0x7f4334009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 shutdown_connections 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f432806e260 0x7f4328070710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43400fbcb0 0x7f434010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 --2- 172.21.15.77:0/3642444799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43400fcf60 0x7f434010ce40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 >> 172.21.15.77:0/3642444799 conn(0x7f43400f7430 msgr2=0x7f43400f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 shutdown_connections 2024-04-26T04:56:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.785+0000 7f4344f2e700 1 -- 172.21.15.77:0/3642444799 wait complete. 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 -- 172.21.15.77:0/991808539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a44066260 msgr2=0x7f9a440ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/991808539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a44066260 0x7f9a440ffa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a38004940 tx=0x7f9a38009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 -- 172.21.15.77:0/991808539 shutdown_connections 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/991808539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a440fff40 0x7f9a44102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/991808539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a44066260 0x7f9a440ffa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 -- 172.21.15.77:0/991808539 >> 172.21.15.77:0/991808539 conn(0x7f9a440f7430 msgr2=0x7f9a440f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 -- 172.21.15.77:0/991808539 shutdown_connections 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 -- 172.21.15.77:0/991808539 wait complete. 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.886+0000 7f9a4a48f700 1 Processor -- start 2024-04-26T04:56:25.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a4a48f700 1 -- start start 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a4a48f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a44066260 0x7f9a4410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a4a48f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a4a48f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4410e540 con 0x7f9a440fff40 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a4a48f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4410ed40 con 0x7f9a44066260 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47056/0 (socket says 172.21.15.77:47056) 2024-04-26T04:56:25.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 -- 172.21.15.77:0/1860649154 learned_addr learned my addr 172.21.15.77:0/1860649154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a43fff700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a44066260 0x7f9a4410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 -- 172.21.15.77:0/1860649154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a44066260 msgr2=0x7f9a4410da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a44066260 0x7f9a4410da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 -- 172.21.15.77:0/1860649154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38005040 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a437fe700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400eb40 tx=0x7f9a3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a34007cc0 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.887+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a3400f040 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.888+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4410efc0 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.888+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a3400ca10 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.888+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4410f500 con 0x7f9a440fff40 2024-04-26T04:56:25.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.888+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4405f5c0 con 0x7f9a440fff40 2024-04-26T04:56:25.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.889+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9a3400cb70 con 0x7f9a440fff40 2024-04-26T04:56:25.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.889+0000 7f9a40ff9700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 0x7f9a2c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:25.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.889+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9a3408d860 con 0x7f9a440fff40 2024-04-26T04:56:25.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.890+0000 7f9a43fff700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 0x7f9a2c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:25.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.890+0000 7f9a43fff700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 0x7f9a2c070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a38004890 tx=0x7f9a38009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:25.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:25.894+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9a3405a670 con 0x7f9a440fff40 2024-04-26T04:56:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:25 smithi077 ceph-mon[91833]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:25 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:25.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:25 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:25 smithi078 ceph-mon[79689]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:25 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:25 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.098+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a4410f880 con 0x7f9a2c06e260 2024-04-26T04:56:26.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.108+0000 7f9a40ff9700 1 -- 172.21.15.77:0/1860649154 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9a4410f880 con 0x7f9a2c06e260 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (3m) 50s ago 11m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 starting - - - - 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (11m) 37s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 2195a0caa6b5 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 50s ago 11m - - 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (6m) 50s ago 6m 154M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (6m) 50s ago 6m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (6m) 37s ago 6m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (6m) 37s ago 6m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (2m) 50s ago 13m 474M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (77s) 37s ago 10m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:56:26.109 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (63s) 50s ago 13m 27.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (41s) 37s ago 10m 18.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (3m) 50s ago 11m 15.8M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (2m) 37s ago 10m 16.5M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (10m) 50s ago 10m 203M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 87168974e841 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (9m) 50s ago 9m 213M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (9m) 50s ago 9m 178M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (8m) 37s ago 8m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (8m) 37s ago 8m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (7m) 37s ago 7m 192M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:56:26.110 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (83s) 50s ago 11m 52.3M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 msgr2=0x7f9a2c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 0x7f9a2c070710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a38004890 tx=0x7f9a38009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 msgr2=0x7f9a4410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400eb40 tx=0x7f9a3400b040 comp rx=0 tx=0).stop 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 shutdown_connections 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9a2c06e260 0x7f9a2c070710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a44066260 0x7f9a4410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 --2- 172.21.15.77:0/1860649154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a440fff40 0x7f9a4410df40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 >> 172.21.15.77:0/1860649154 conn(0x7f9a440f7430 msgr2=0x7f9a440f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 shutdown_connections 2024-04-26T04:56:26.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.110+0000 7f9a4a48f700 1 -- 172.21.15.77:0/1860649154 wait complete. 2024-04-26T04:56:26.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 -- 172.21.15.77:0/3336128221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fcf70 msgr2=0x7ff9bc0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/3336128221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc0fd3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac004940 tx=0x7ff9ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 -- 172.21.15.77:0/3336128221 shutdown_connections 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/3336128221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc0fd3e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/3336128221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 -- 172.21.15.77:0/3336128221 >> 172.21.15.77:0/3336128221 conn(0x7ff9bc0f7440 msgr2=0x7ff9bc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 -- 172.21.15.77:0/3336128221 shutdown_connections 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.209+0000 7ff9c2adb700 1 -- 172.21.15.77:0/3336128221 wait complete. 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 Processor -- start 2024-04-26T04:56:26.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 -- start start 2024-04-26T04:56:26.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc114ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc10e4a0 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c2adb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc115000 con 0x7ff9bc0fcf70 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c0877700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c0877700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47072/0 (socket says 172.21.15.77:47072) 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c0877700 1 -- 172.21.15.77:0/2245907236 learned_addr learned my addr 172.21.15.77:0/2245907236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9bbfff700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc114ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c0877700 1 -- 172.21.15.77:0/2245907236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fcf70 msgr2=0x7ff9bc114ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.210+0000 7ff9c0877700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc114ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9c0877700 1 -- 172.21.15.77:0/2245907236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac005040 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9c0877700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4001de0 tx=0x7ff9b4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b4007d10 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc115200 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc115770 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9b4007e70 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.211+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b4014640 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.212+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc04c710 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.212+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff9b40147a0 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.213+0000 7ff9b97fa700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 0x7ff9a8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.213+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff9b408d960 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.213+0000 7ff9bbfff700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 0x7ff9a8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.213+0000 7ff9bbfff700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 0x7ff9a8070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b3b0 tx=0x7ff9ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.217+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff9b405a880 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.499+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9bc115cd0 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.500+0000 7ff9b97fa700 1 -- 172.21.15.77:0/2245907236 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7ff9b405a410 con 0x7ff9bc0fbcc0 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:56:26.502 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 msgr2=0x7ff9a8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 0x7ff9a8070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b3b0 tx=0x7ff9ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 msgr2=0x7ff9bc066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4001de0 tx=0x7ff9b4009ee0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 shutdown_connections 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff9a806e270 0x7ff9a8070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9bc0fbcc0 0x7ff9bc066cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 --2- 172.21.15.77:0/2245907236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9bc0fcf70 0x7ff9bc114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 >> 172.21.15.77:0/2245907236 conn(0x7ff9bc0f7440 msgr2=0x7ff9bc0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 shutdown_connections 2024-04-26T04:56:26.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.502+0000 7ff9c2adb700 1 -- 172.21.15.77:0/2245907236 wait complete. 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 -- 172.21.15.77:0/4182741357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c0ffef0 msgr2=0x7fc85c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 --2- 172.21.15.77:0/4182741357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c0ffef0 0x7fc85c102310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc850004940 tx=0x7fc850009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 -- 172.21.15.77:0/4182741357 shutdown_connections 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 --2- 172.21.15.77:0/4182741357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c0ffef0 0x7fc85c102310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 --2- 172.21.15.77:0/4182741357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c066f50 0x7fc85c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 -- 172.21.15.77:0/4182741357 >> 172.21.15.77:0/4182741357 conn(0x7fc85c0f7440 msgr2=0x7fc85c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 -- 172.21.15.77:0/4182741357 shutdown_connections 2024-04-26T04:56:26.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.602+0000 7fc861553700 1 -- 172.21.15.77:0/4182741357 wait complete. 2024-04-26T04:56:26.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 Processor -- start 2024-04-26T04:56:26.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 -- start start 2024-04-26T04:56:26.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c0ffef0 0x7fc85c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c10e490 con 0x7fc85c066f50 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc861553700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c10e600 con 0x7fc85c0ffef0 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc85affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc85affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47100/0 (socket says 172.21.15.77:47100) 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc85affd700 1 -- 172.21.15.77:0/816893478 learned_addr learned my addr 172.21.15.77:0/816893478 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.603+0000 7fc85a7fc700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c0ffef0 0x7fc85c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc85affd700 1 -- 172.21.15.77:0/816893478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c0ffef0 msgr2=0x7fc85c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc85affd700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c0ffef0 0x7fc85c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc85affd700 1 -- 172.21.15.77:0/816893478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850005040 con 0x7fc85c066f50 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc85affd700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00eb40 tx=0x7fc84c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84c007cc0 con 0x7fc85c066f50 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c113000 con 0x7fc85c066f50 2024-04-26T04:56:26.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc84c00f040 con 0x7fc85c066f50 2024-04-26T04:56:26.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84c00cad0 con 0x7fc85c066f50 2024-04-26T04:56:26.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.604+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c113540 con 0x7fc85c066f50 2024-04-26T04:56:26.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.605+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c04c710 con 0x7fc85c066f50 2024-04-26T04:56:26.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.605+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc84c00cc30 con 0x7fc85c066f50 2024-04-26T04:56:26.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.606+0000 7fc8437fe700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 0x7fc844070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.606+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc84c08d8a0 con 0x7fc85c066f50 2024-04-26T04:56:26.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.606+0000 7fc85a7fc700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 0x7fc844070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.606+0000 7fc85a7fc700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 0x7fc844070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc850013870 tx=0x7fc850009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.610+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc84c05a580 con 0x7fc85c066f50 2024-04-26T04:56:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.857+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc85c113940 con 0x7fc85c066f50 2024-04-26T04:56:26.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.858+0000 7fc8437fe700 1 -- 172.21.15.77:0/816893478 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fc84c05a110 con 0x7fc85c066f50 2024-04-26T04:56:26.859 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:56:26.859 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:56:26.859 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:56:26.860 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:56:26.861 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 msgr2=0x7fc844070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 0x7fc844070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc850013870 tx=0x7fc850009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 msgr2=0x7fc85c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00eb40 tx=0x7fc84c00b040 comp rx=0 tx=0).stop 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 shutdown_connections 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc84406e270 0x7fc844070720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc85c066f50 0x7fc85c10d950 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 --2- 172.21.15.77:0/816893478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85c0ffef0 0x7fc85c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 >> 172.21.15.77:0/816893478 conn(0x7fc85c0f7440 msgr2=0x7fc85c0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 shutdown_connections 2024-04-26T04:56:26.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.860+0000 7fc861553700 1 -- 172.21.15.77:0/816893478 wait complete. 2024-04-26T04:56:26.864 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- 172.21.15.77:0/4203646683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 msgr2=0x7fa4d00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 --2- 172.21.15.77:0/4203646683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d00fd3d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc004940 tx=0x7fa4bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- 172.21.15.77:0/4203646683 shutdown_connections 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 --2- 172.21.15.77:0/4203646683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d00fd3d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 --2- 172.21.15.77:0/4203646683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 0x7fa4d00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- 172.21.15.77:0/4203646683 >> 172.21.15.77:0/4203646683 conn(0x7fa4d00f7430 msgr2=0x7fa4d00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- 172.21.15.77:0/4203646683 shutdown_connections 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- 172.21.15.77:0/4203646683 wait complete. 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 Processor -- start 2024-04-26T04:56:26.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.962+0000 7fa4d7152700 1 -- start start 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 0x7fa4d010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d010d470 con 0x7fa4d00fcf60 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d010dcc0 con 0x7fa4d00fbcb0 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47130/0 (socket says 172.21.15.77:47130) 2024-04-26T04:56:26.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 -- 172.21.15.77:0/2428819890 learned_addr learned my addr 172.21.15.77:0/2428819890 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d4eee700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 0x7fa4d010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 -- 172.21.15.77:0/2428819890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 msgr2=0x7fa4d010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 0x7fa4d010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 -- 172.21.15.77:0/2428819890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc005040 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cffff700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009150 tx=0x7fa4bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4bc00e040 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4bc00bbf0 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d010df40 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4bc01c630 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.963+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d010e480 con 0x7fa4d00fcf60 2024-04-26T04:56:26.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.965+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d005f5c0 con 0x7fa4d00fcf60 2024-04-26T04:56:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.965+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa4bc00bd60 con 0x7fa4d00fcf60 2024-04-26T04:56:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.965+0000 7fa4cd7fa700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 0x7fa4c0070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.965+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa4bc08f090 con 0x7fa4d00fcf60 2024-04-26T04:56:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.966+0000 7fa4d4eee700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 0x7fa4c0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:26.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.966+0000 7fa4d4eee700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 0x7fa4c0070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4004f60 tx=0x7fa4c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:26.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:26.970+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa4bc05bdf0 con 0x7fa4d00fcf60 2024-04-26T04:56:26.983 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:26 smithi078 ceph-mon[79689]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:26.984 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:26 smithi078 ceph-mon[79689]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:26.984 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:26 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2245907236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:27.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.187+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4d010e7b0 con 0x7fa4c006e260 2024-04-26T04:56:27.189 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:26 smithi077 ceph-mon[91833]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:27.189 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:26 smithi077 ceph-mon[91833]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:27.189 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:26 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2245907236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.188+0000 7fa4cd7fa700 1 -- 172.21.15.77:0/2428819890 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fa4d010e7b0 con 0x7fa4c006e260 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:56:27.190 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "4/21 daemons upgraded", 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading crash daemons", 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:56:27.191 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:56:27.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.190+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 msgr2=0x7fa4c0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.190+0000 7fa4d7152700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 0x7fa4c0070710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4004f60 tx=0x7fa4c400b040 comp rx=0 tx=0).stop 2024-04-26T04:56:27.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.190+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 msgr2=0x7fa4d010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.190+0000 7fa4d7152700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009150 tx=0x7fa4bc009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 shutdown_connections 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa4c006e260 0x7fa4c0070710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4d00fbcb0 0x7fa4d010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 --2- 172.21.15.77:0/2428819890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d00fcf60 0x7fa4d010ce70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 >> 172.21.15.77:0/2428819890 conn(0x7fa4d00f7430 msgr2=0x7fa4d00f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 shutdown_connections 2024-04-26T04:56:27.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.191+0000 7fa4d7152700 1 -- 172.21.15.77:0/2428819890 wait complete. 2024-04-26T04:56:27.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 -- 172.21.15.77:0/2977342063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fed30 msgr2=0x7f54200fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 --2- 172.21.15.77:0/2977342063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fed30 0x7f54200fae70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5414004940 tx=0x7f5414009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 -- 172.21.15.77:0/2977342063 shutdown_connections 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 --2- 172.21.15.77:0/2977342063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fed30 0x7f54200fae70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 --2- 172.21.15.77:0/2977342063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fe3e0 0x7f54200fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.293+0000 7f5425ab1700 1 -- 172.21.15.77:0/2977342063 >> 172.21.15.77:0/2977342063 conn(0x7f54200f6b80 msgr2=0x7f54200f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 -- 172.21.15.77:0/2977342063 shutdown_connections 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 -- 172.21.15.77:0/2977342063 wait complete. 2024-04-26T04:56:27.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 Processor -- start 2024-04-26T04:56:27.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 -- start start 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fed30 0x7f542010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542010e570 con 0x7f54200fe3e0 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.294+0000 7f5425ab1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542010ed70 con 0x7f54200fed30 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47152/0 (socket says 172.21.15.77:47152) 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 -- 172.21.15.77:0/1617264231 learned_addr learned my addr 172.21.15.77:0/1617264231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fed30 0x7f542010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 -- 172.21.15.77:0/1617264231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fed30 msgr2=0x7f542010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fed30 0x7f542010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 -- 172.21.15.77:0/1617264231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414005040 con 0x7f54200fe3e0 2024-04-26T04:56:27.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f541f7fe700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5410002a50 tx=0x7f54100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5410007cc0 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f541000f040 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541000da30 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542010f050 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.295+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542010f590 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.296+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542005f5c0 con 0x7f54200fe3e0 2024-04-26T04:56:27.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.297+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f541000db90 con 0x7f54200fe3e0 2024-04-26T04:56:27.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.297+0000 7f5407fff700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 0x7f5408070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:27.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.297+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f541008db00 con 0x7f54200fe3e0 2024-04-26T04:56:27.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.297+0000 7f541effd700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 0x7f5408070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:27.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.297+0000 7f541effd700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 0x7f5408070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f541400b3b0 tx=0x7f5414009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:27.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.302+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f541005a7e0 con 0x7f54200fe3e0 2024-04-26T04:56:27.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.582+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5420105890 con 0x7f54200fe3e0 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.582+0000 7f5407fff700 1 -- 172.21.15.77:0/1617264231 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+347 (secure 0 0 0) 0x7f541005a370 con 0x7f54200fe3e0 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:56:27.584 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:56:27.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 msgr2=0x7f5408070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 0x7f5408070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f541400b3b0 tx=0x7f5414009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 msgr2=0x7f542010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5410002a50 tx=0x7f54100046b0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 shutdown_connections 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f540806e260 0x7f5408070710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54200fe3e0 0x7f542010da30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 --2- 172.21.15.77:0/1617264231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54200fed30 0x7f542010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 >> 172.21.15.77:0/1617264231 conn(0x7f54200f6b80 msgr2=0x7f54200f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 shutdown_connections 2024-04-26T04:56:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:27.584+0000 7f5425ab1700 1 -- 172.21.15.77:0/1617264231 wait complete. 2024-04-26T04:56:27.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/816893478' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:27 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1617264231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/816893478' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi078", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:28.094 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:27 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1617264231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:56:28.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:28 smithi077 ceph-mon[91833]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:28.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:28 smithi077 ceph-mon[91833]: Upgrade: Updating crash.smithi078 (2/2) 2024-04-26T04:56:28.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:28 smithi077 ceph-mon[91833]: Deploying daemon crash.smithi078 on smithi078 2024-04-26T04:56:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:28 smithi078 ceph-mon[79689]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:28 smithi078 ceph-mon[79689]: Upgrade: Updating crash.smithi078 (2/2) 2024-04-26T04:56:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:28 smithi078 ceph-mon[79689]: Deploying daemon crash.smithi078 on smithi078 2024-04-26T04:56:30.049 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:29 smithi077 ceph-mon[91833]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:30.186 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:29 smithi078 ceph-mon[79689]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:31.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:31 smithi077 ceph-mon[91833]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:31.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:31.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:31.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:31.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:31 smithi078 ceph-mon[79689]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:33.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:33 smithi077 ceph-mon[91833]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:33 smithi078 ceph-mon[79689]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:34.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:34 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:34.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:34 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:34 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:35.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:34 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:35 smithi078 ceph-mon[79689]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:35 smithi077 ceph-mon[91833]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:36.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:36.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:37.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:37.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:37.735 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:37 smithi077 ceph-mon[91833]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:37.747 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:37.747 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:37.747 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:37 smithi078 ceph-mon[79689]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:39.373 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:39.373 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:39.373 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:39 smithi077 ceph-mon[91833]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:39 smithi078 ceph-mon[79689]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi077"}]: dispatch 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi077"}]': finished 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi078"}]: dispatch 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi078"}]': finished 2024-04-26T04:56:41.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:56:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi077"}]: dispatch 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi077"}]': finished 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi078"}]: dispatch 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi078"}]': finished 2024-04-26T04:56:41.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:56:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all crash 2024-04-26T04:56:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:56:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: Upgrade: osd.0 is safe to restart 2024-04-26T04:56:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:56:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all crash 2024-04-26T04:56:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:56:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: Upgrade: osd.0 is safe to restart 2024-04-26T04:56:42.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:42.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:56:42.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:56:43.193 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:42 smithi077 ceph-mon[91833]: Upgrade: Updating osd.0 2024-04-26T04:56:43.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:42 smithi077 ceph-mon[91833]: Deploying daemon osd.0 on smithi077 2024-04-26T04:56:43.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:42 smithi077 ceph-mon[91833]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:42 smithi078 ceph-mon[79689]: Upgrade: Updating osd.0 2024-04-26T04:56:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:42 smithi078 ceph-mon[79689]: Deploying daemon osd.0 on smithi077 2024-04-26T04:56:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:42 smithi078 ceph-mon[79689]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:45.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:44 smithi077 ceph-mon[91833]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:44 smithi078 ceph-mon[79689]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:45.989 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:45 smithi077 systemd[1]: Stopping Ceph osd.0 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:56:45.990 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:45 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[44715]: 2024-04-26T04:56:45.881+0000 7f272d482700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:56:45.990 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:45 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[44715]: 2024-04-26T04:56:45.881+0000 7f272d482700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-26T04:56:45.990 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:45 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[44715]: 2024-04-26T04:56:45.881+0000 7f272d482700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:56:46.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:45 smithi077 ceph-mon[91833]: osd.0 marked itself down and dead 2024-04-26T04:56:46.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:45 smithi078 ceph-mon[79689]: osd.0 marked itself down and dead 2024-04-26T04:56:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:56:46.876 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:46 smithi077 podman[100653]: 2024-04-26 04:56:46.568188641 +0000 UTC m=+0.781821213 container died 87168974e841bb155ba7951f5030e03f86be19b336ff7fae4119664ef9a35d76 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T04:56:47.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:46 smithi077 ceph-mon[91833]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:47.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:46 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:56:47.218 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:46 smithi077 ceph-mon[91833]: osdmap e50: 6 total, 5 up, 6 in 2024-04-26T04:56:47.219 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:46 smithi077 podman[100653]: 2024-04-26 04:56:46.966981279 +0000 UTC m=+1.180613859 container cleanup 87168974e841bb155ba7951f5030e03f86be19b336ff7fae4119664ef9a35d76 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T04:56:47.219 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:46 smithi077 bash[100653]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0 2024-04-26T04:56:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:46 smithi078 ceph-mon[79689]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:46 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:56:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:46 smithi078 ceph-mon[79689]: osdmap e50: 6 total, 5 up, 6 in 2024-04-26T04:56:47.492 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100671]: 2024-04-26 04:56:47.218284714 +0000 UTC m=+0.653275460 container remove 87168974e841bb155ba7951f5030e03f86be19b336ff7fae4119664ef9a35d76 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T04:56:47.789 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100765]: 2024-04-26 04:56:47.439934018 +0000 UTC m=+0.017474571 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:56:47.789 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100765]: 2024-04-26 04:56:47.561218743 +0000 UTC m=+0.138759289 container create ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:56:47.789 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100765]: 2024-04-26 04:56:47.788802119 +0000 UTC m=+0.366342660 container init ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_CLEAN=True) 2024-04-26T04:56:48.081 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:47 smithi077 ceph-mon[91833]: osdmap e51: 6 total, 5 up, 6 in 2024-04-26T04:56:48.081 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100765]: 2024-04-26 04:56:47.792005572 +0000 UTC m=+0.369546117 container start ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:56:48.081 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:47 smithi077 podman[100765]: 2024-04-26 04:56:47.888812425 +0000 UTC m=+0.466352984 container attach ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb) 2024-04-26T04:56:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:47 smithi078 ceph-mon[79689]: osdmap e51: 6 total, 5 up, 6 in 2024-04-26T04:56:48.362 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 podman[100765]: 2024-04-26 04:56:48.081554052 +0000 UTC m=+0.659094589 container died ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T04:56:48.921 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 podman[100791]: 2024-04-26 04:56:48.692991733 +0000 UTC m=+0.605464820 container remove ff21ceb0fbfc11249cb6a12078530304c379e6d337c66f9e67683bfa43dee3fc (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T04:56:48.921 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-26T04:56:48.921 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 systemd[1]: Stopped Ceph osd.0 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:56:48.921 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.0.service: Consumed 49.049s CPU time. 2024-04-26T04:56:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-mon[91833]: pgmap v47: 65 pgs: 34 peering, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-mon[91833]: Health check failed: Reduced data availability: 34 pgs peering (PG_AVAILABILITY) 2024-04-26T04:56:49.243 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:48 smithi077 systemd[1]: Starting Ceph osd.0 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:56:49.243 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.03950283 +0000 UTC m=+0.017728929 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:56:49.243 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.144250169 +0000 UTC m=+0.122476311 container create 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T04:56:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:49 smithi078 ceph-mon[79689]: pgmap v47: 65 pgs: 34 peering, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:49 smithi078 ceph-mon[79689]: Health check failed: Reduced data availability: 34 pgs peering (PG_AVAILABILITY) 2024-04-26T04:56:49.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.372161334 +0000 UTC m=+0.350387429 container init 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T04:56:49.743 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.376657063 +0000 UTC m=+0.354883148 container start 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T04:56:49.743 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.480080901 +0000 UTC m=+0.458306985 container attach 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:56:50.148 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 bash[100896]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate[100910]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-26T04:56:50.149 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:49 smithi077 podman[100896]: 2024-04-26 04:56:49.843941405 +0000 UTC m=+0.822167480 container died 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T04:56:50.724 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 podman[101049]: 2024-04-26 04:56:50.409717379 +0000 UTC m=+0.558952452 container remove 5264b1970873387be885fe2e972e40003ee7656297e2858d0ac6d41d26930f52 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-activate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T04:56:50.725 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 podman[101114]: 2024-04-26 04:56:50.523030751 +0000 UTC m=+0.017750556 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:56:50.725 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 podman[101114]: 2024-04-26 04:56:50.62734424 +0000 UTC m=+0.122064031 container create ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T04:56:50.992 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 podman[101114]: 2024-04-26 04:56:50.830321056 +0000 UTC m=+0.325040850 container init ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-b22e2eb) 2024-04-26T04:56:50.992 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 podman[101114]: 2024-04-26 04:56:50.833455311 +0000 UTC m=+0.328175104 container start ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb) 2024-04-26T04:56:50.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 ceph-osd[101132]: -- 172.21.15.77:0/3189237412 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd02cf1680 con 0x55fd02cbb000 2024-04-26T04:56:50.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 bash[101114]: ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 2024-04-26T04:56:50.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:50 smithi077 systemd[1]: Started Ceph osd.0 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:56:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:51 smithi078 ceph-mon[79689]: pgmap v48: 65 pgs: 34 peering, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:51.281 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:51 smithi077 ceph-mon[91833]: pgmap v48: 65 pgs: 34 peering, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:56:51.281 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:51.281 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:51.282 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:56:51.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:51 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T04:56:51.371+0000 7f3c05e347c0 -1 Falling back to public interface 2024-04-26T04:56:52.118 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:52 smithi077 ceph-mon[91833]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:52.118 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:52 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-26T04:56:52.118 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:52 smithi077 ceph-mon[91833]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 34 pgs peering) 2024-04-26T04:56:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:52 smithi078 ceph-mon[79689]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:52 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-26T04:56:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:52 smithi078 ceph-mon[79689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 34 pgs peering) 2024-04-26T04:56:55.351 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:55 smithi077 ceph-mon[91833]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:55 smithi078 ceph-mon[79689]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:56.450 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:56 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T04:56:56.122+0000 7f3c05e347c0 -1 osd.0 49 log_to_monitors true 2024-04-26T04:56:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:57 smithi078 ceph-mon[79689]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:57 smithi078 ceph-mon[79689]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:56:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:57.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:57 smithi077 ceph-mon[91833]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:57.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:57 smithi077 ceph-mon[91833]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:56:57.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:57.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:57.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.738+0000 7f86801ed700 1 -- 172.21.15.77:0/1550698793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 msgr2=0x7f8678104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:57.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.738+0000 7f86801ed700 1 --2- 172.21.15.77:0/1550698793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f8678104630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8668004940 tx=0x7f8668009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:57.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 -- 172.21.15.77:0/1550698793 shutdown_connections 2024-04-26T04:56:57.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 --2- 172.21.15.77:0/1550698793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f8678104630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 --2- 172.21.15.77:0/1550698793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 -- 172.21.15.77:0/1550698793 >> 172.21.15.77:0/1550698793 conn(0x7f86780f6b80 msgr2=0x7f86780f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 -- 172.21.15.77:0/1550698793 shutdown_connections 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 -- 172.21.15.77:0/1550698793 wait complete. 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 Processor -- start 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.739+0000 7f86801ed700 1 -- start start 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f86801ed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f86801ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f86801ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780ff150 con 0x7f86780fdce0 2024-04-26T04:56:57.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f86801ed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780ff9a0 con 0x7f86780fd390 2024-04-26T04:56:57.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:57.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45976/0 (socket says 172.21.15.77:45976) 2024-04-26T04:56:57.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 -- 172.21.15.77:0/3923685841 learned_addr learned my addr 172.21.15.77:0/3923685841 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867df89700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 -- 172.21.15.77:0/3923685841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 msgr2=0x7f86780fe5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fe5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867d788700 1 -- 172.21.15.77:0/3923685841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668005040 con 0x7f86780fdce0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.740+0000 7f867df89700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f867d788700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f866800b3b0 tx=0x7f8668009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866800e040 con 0x7f86780fdce0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f866800bbf0 con 0x7f86780fdce0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780ffc20 con 0x7f86780fdce0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866801c630 con 0x7f86780fdce0 2024-04-26T04:56:57.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.741+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86781001e0 con 0x7f86780fdce0 2024-04-26T04:56:57.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.742+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867805f5c0 con 0x7f86780fdce0 2024-04-26T04:56:57.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.744+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f866800bd60 con 0x7f86780fdce0 2024-04-26T04:56:57.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.744+0000 7f866e7fc700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 0x7f8664070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:57.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.744+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f866808efc0 con 0x7f86780fdce0 2024-04-26T04:56:57.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.744+0000 7f867df89700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 0x7f8664070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:57.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.745+0000 7f867df89700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 0x7f8664070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f86780fe350 tx=0x7f867400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:57.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.750+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8668058290 con 0x7f86780fdce0 2024-04-26T04:56:57.992 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 04:56:57 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T04:56:57.896+0000 7f3bf3284700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:56:57.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.998+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86781004c0 con 0x7f866406e260 2024-04-26T04:56:57.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:57.999+0000 7f866e7fc700 1 -- 172.21.15.77:0/3923685841 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f86781004c0 con 0x7f866406e260 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 msgr2=0x7f8664070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 0x7f8664070710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f86780fe350 tx=0x7f867400b040 comp rx=0 tx=0).stop 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 msgr2=0x7f86780feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f866800b3b0 tx=0x7f8668009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 shutdown_connections 2024-04-26T04:56:58.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f866406e260 0x7f8664070710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86780fd390 0x7f86780fe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 --2- 172.21.15.77:0/3923685841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86780fdce0 0x7f86780feb00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 >> 172.21.15.77:0/3923685841 conn(0x7f86780f6b80 msgr2=0x7f86780f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 shutdown_connections 2024-04-26T04:56:58.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.001+0000 7f86801ed700 1 -- 172.21.15.77:0/3923685841 wait complete. 2024-04-26T04:56:58.012 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.110+0000 7f1471fdb700 1 -- 172.21.15.77:0/337091234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fcf60 msgr2=0x7f146c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.110+0000 7f1471fdb700 1 --2- 172.21.15.77:0/337091234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fcf60 0x7f146c0fd3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1460004940 tx=0x7f1460009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- 172.21.15.77:0/337091234 shutdown_connections 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 --2- 172.21.15.77:0/337091234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fcf60 0x7f146c0fd3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 --2- 172.21.15.77:0/337091234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fbcb0 0x7f146c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- 172.21.15.77:0/337091234 >> 172.21.15.77:0/337091234 conn(0x7f146c0f7430 msgr2=0x7f146c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- 172.21.15.77:0/337091234 shutdown_connections 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- 172.21.15.77:0/337091234 wait complete. 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 Processor -- start 2024-04-26T04:56:58.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- start start 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fcf60 0x7f146c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c10d440 con 0x7f146c0fbcb0 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.111+0000 7f1471fdb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c10dc90 con 0x7f146c0fcf60 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45994/0 (socket says 172.21.15.77:45994) 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 -- 172.21.15.77:0/1926758477 learned_addr learned my addr 172.21.15.77:0/1926758477 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146affd700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fcf60 0x7f146c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 -- 172.21.15.77:0/1926758477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fcf60 msgr2=0x7f146c10ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fcf60 0x7f146c10ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 -- 172.21.15.77:0/1926758477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1460005040 con 0x7f146c0fbcb0 2024-04-26T04:56:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f146b7fe700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c830 tx=0x7f145c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145c0106f0 con 0x7f146c0fbcb0 2024-04-26T04:56:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145c010d20 con 0x7f146c0fbcb0 2024-04-26T04:56:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f1471fdb700 1 -- 172.21.15.77:0/1926758477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c10df70 con 0x7f146c0fbcb0 2024-04-26T04:56:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145c0113a0 con 0x7f146c0fbcb0 2024-04-26T04:56:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.112+0000 7f1471fdb700 1 -- 172.21.15.77:0/1926758477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c10e4b0 con 0x7f146c0fbcb0 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.113+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c05f5c0 con 0x7f146c0fbcb0 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.114+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f145c012400 con 0x7f146c0fbcb0 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.114+0000 7f1453fff700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 0x7f1454070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.115+0000 7f146affd700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 0x7f1454070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.115+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f145c08dba0 con 0x7f146c0fbcb0 2024-04-26T04:56:58.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.115+0000 7f146affd700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 0x7f1454070710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f146000b970 tx=0x7f1460009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.119+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f145c05aa10 con 0x7f146c0fbcb0 2024-04-26T04:56:58.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.350+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f146c04bed0 con 0x7f145406e260 2024-04-26T04:56:58.351 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:58 smithi077 ceph-mon[91833]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:56:58.351 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:58 smithi077 ceph-mon[91833]: osdmap e52: 6 total, 5 up, 6 in 2024-04-26T04:56:58.351 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:58 smithi077 ceph-mon[91833]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:56:58.351 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:58 smithi077 ceph-mon[91833]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' 2024-04-26T04:56:58.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.351+0000 7f1453fff700 1 -- 172.21.15.77:0/1926758477 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f146c04bed0 con 0x7f145406e260 2024-04-26T04:56:58.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 msgr2=0x7f1454070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 0x7f1454070710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f146000b970 tx=0x7f1460009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 msgr2=0x7f146c10c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c830 tx=0x7f145c004470 comp rx=0 tx=0).stop 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 shutdown_connections 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f145406e260 0x7f1454070710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0fbcb0 0x7f146c10c900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 --2- 172.21.15.77:0/1926758477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f146c0fcf60 0x7f146c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 >> 172.21.15.77:0/1926758477 conn(0x7f146c0f7430 msgr2=0x7f146c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 shutdown_connections 2024-04-26T04:56:58.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.354+0000 7f1451ffb700 1 -- 172.21.15.77:0/1926758477 wait complete. 2024-04-26T04:56:58.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 -- 172.21.15.77:0/991435777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c0fffd0 msgr2=0x7ff06c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 --2- 172.21.15.77:0/991435777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c0fffd0 0x7ff06c1023b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff05c004940 tx=0x7ff05c009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:58.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 -- 172.21.15.77:0/991435777 shutdown_connections 2024-04-26T04:56:58.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 --2- 172.21.15.77:0/991435777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c0fffd0 0x7ff06c1023b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 --2- 172.21.15.77:0/991435777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c066af0 0x7ff06c0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 -- 172.21.15.77:0/991435777 >> 172.21.15.77:0/991435777 conn(0x7ff06c0f74c0 msgr2=0x7ff06c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 -- 172.21.15.77:0/991435777 shutdown_connections 2024-04-26T04:56:58.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.460+0000 7ff0721b7700 1 -- 172.21.15.77:0/991435777 wait complete. 2024-04-26T04:56:58.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 Processor -- start 2024-04-26T04:56:58.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 -- start start 2024-04-26T04:56:58.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 0x7ff06c0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0fbf40 con 0x7ff06c066af0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff0721b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0fc0b0 con 0x7ff06c0fffd0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46006/0 (socket says 172.21.15.77:46006) 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff06b7fe700 1 -- 172.21.15.77:0/867818704 learned_addr learned my addr 172.21.15.77:0/867818704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.461+0000 7ff06affd700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 0x7ff06c0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff06b7fe700 1 -- 172.21.15.77:0/867818704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 msgr2=0x7ff06c0fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff06b7fe700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 0x7ff06c0fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff06b7fe700 1 -- 172.21.15.77:0/867818704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c005040 con 0x7ff06c066af0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff06affd700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 0x7ff06c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff06b7fe700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff060002a80 tx=0x7ff060004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff060007cc0 con 0x7ff06c066af0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff06000f040 con 0x7ff06c066af0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c0fc390 con 0x7ff06c066af0 2024-04-26T04:56:58.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff06000dad0 con 0x7ff06c066af0 2024-04-26T04:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.462+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c115a60 con 0x7ff06c066af0 2024-04-26T04:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.463+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c05f5c0 con 0x7ff06c066af0 2024-04-26T04:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.463+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff06000dc30 con 0x7ff06c066af0 2024-04-26T04:56:58.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.464+0000 7ff053fff700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 0x7ff054070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.464+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff06008da40 con 0x7ff06c066af0 2024-04-26T04:56:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.464+0000 7ff06affd700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 0x7ff054070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.464+0000 7ff06affd700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 0x7ff054070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00b3b0 tx=0x7ff05c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.469+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff060056c70 con 0x7ff06c066af0 2024-04-26T04:56:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:58 smithi078 ceph-mon[79689]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:56:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:58 smithi078 ceph-mon[79689]: osdmap e52: 6 total, 5 up, 6 in 2024-04-26T04:56:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:58 smithi078 ceph-mon[79689]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:56:58.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:58 smithi078 ceph-mon[79689]: from='osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312]' entity='osd.0' 2024-04-26T04:56:58.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.689+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff06c0f8fb0 con 0x7ff05406e260 2024-04-26T04:56:58.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.701+0000 7ff053fff700 1 -- 172.21.15.77:0/867818704 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff06c0f8fb0 con 0x7ff05406e260 2024-04-26T04:56:58.702 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (3m) 1s ago 12m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (33s) 1s ago 12m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (28s) 24s ago 11m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 1s ago 12m - - 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (6m) 1s ago 6m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (6m) 1s ago 6m 17.0M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (6m) 24s ago 6m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (6m) 24s ago 6m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (2m) 1s ago 13m 500M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (110s) 24s ago 11m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (96s) 1s ago 13m 43.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (74s) 24s ago 11m 31.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (4m) 1s ago 12m 16.0M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:56:58.703 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (3m) 24s ago 11m 16.7M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (8s) 1s ago 10m 13.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (10m) 1s ago 10m 213M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 69f7a839880b 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (9m) 1s ago 9m 176M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (9m) 24s ago 9m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (8m) 24s ago 8m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (8m) 24s ago 8m 193M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (115s) 1s ago 11m 56.6M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:56:58.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 msgr2=0x7ff054070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 0x7ff054070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00b3b0 tx=0x7ff05c009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 msgr2=0x7ff06c0fd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff060002a80 tx=0x7ff060004600 comp rx=0 tx=0).stop 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 shutdown_connections 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff05406e260 0x7ff054070710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff06c066af0 0x7ff06c0fd4c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 --2- 172.21.15.77:0/867818704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c0fffd0 0x7ff06c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 >> 172.21.15.77:0/867818704 conn(0x7ff06c0f74c0 msgr2=0x7ff06c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 shutdown_connections 2024-04-26T04:56:58.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.704+0000 7ff0721b7700 1 -- 172.21.15.77:0/867818704 wait complete. 2024-04-26T04:56:58.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- 172.21.15.77:0/3560106836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 msgr2=0x7f58640fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 --2- 172.21.15.77:0/3560106836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f58640fd270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f585c009ce0 tx=0x7f585c004d10 comp rx=0 tx=0).stop 2024-04-26T04:56:58.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- 172.21.15.77:0/3560106836 shutdown_connections 2024-04-26T04:56:58.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 --2- 172.21.15.77:0/3560106836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 0x7f58640fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 --2- 172.21.15.77:0/3560106836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f58640fd270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- 172.21.15.77:0/3560106836 >> 172.21.15.77:0/3560106836 conn(0x7f58640f8560 msgr2=0x7f58640fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- 172.21.15.77:0/3560106836 shutdown_connections 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- 172.21.15.77:0/3560106836 wait complete. 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 Processor -- start 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.814+0000 7f586be8d700 1 -- start start 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 0x7f586410e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586410e6a0 con 0x7f58640fce60 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586410eef0 con 0x7f58640fdfd0 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46020/0 (socket says 172.21.15.77:46020) 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 -- 172.21.15.77:0/2812337887 learned_addr learned my addr 172.21.15.77:0/2812337887 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:58.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 -- 172.21.15.77:0/2812337887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 msgr2=0x7f586410e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:58.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869428700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 0x7f586410e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 0x7f586410e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:58.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 -- 172.21.15.77:0/2812337887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009940 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f5869c29700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f585c009380 tx=0x7f585c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c00e040 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585c0128e0 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586410f170 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c01ca10 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.815+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586410f6b0 con 0x7f58640fce60 2024-04-26T04:56:58.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.816+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586405f5c0 con 0x7f58640fce60 2024-04-26T04:56:58.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.817+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f585c012a50 con 0x7f58640fce60 2024-04-26T04:56:58.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.817+0000 7f58627fc700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 0x7f584c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:58.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.817+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f585c08f940 con 0x7f58640fce60 2024-04-26T04:56:58.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.817+0000 7f5869428700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 0x7f584c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:58.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.818+0000 7f5869428700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 0x7f584c070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5858004d60 tx=0x7f585800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:58.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:58.822+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f585c05c620 con 0x7f58640fce60 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.127+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58641058c0 con 0x7f58640fce60 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.128+0000 7f58627fc700 1 -- 172.21.15.77:0/2812337887 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f585c012d00 con 0x7f58640fce60 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-26T04:56:59.129 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 9, 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 5 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:56:59.130 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 msgr2=0x7f584c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 0x7f584c070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5858004d60 tx=0x7f585800b040 comp rx=0 tx=0).stop 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 msgr2=0x7f586410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f585c009380 tx=0x7f585c004d10 comp rx=0 tx=0).stop 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 shutdown_connections 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f584c06e260 0x7f584c070710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58640fce60 0x7f586410db60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 --2- 172.21.15.77:0/2812337887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58640fdfd0 0x7f586410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 >> 172.21.15.77:0/2812337887 conn(0x7f58640f8560 msgr2=0x7f58640f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 shutdown_connections 2024-04-26T04:56:59.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.130+0000 7f586be8d700 1 -- 172.21.15.77:0/2812337887 wait complete. 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 -- 172.21.15.77:0/1300892363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fed80 msgr2=0x7fc32c0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1300892363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fed80 0x7fc32c0faec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc320004940 tx=0x7fc320009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 -- 172.21.15.77:0/1300892363 shutdown_connections 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1300892363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fed80 0x7fc32c0faec0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1300892363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fe430 0x7fc32c0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 -- 172.21.15.77:0/1300892363 >> 172.21.15.77:0/1300892363 conn(0x7fc32c0f6bd0 msgr2=0x7fc32c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 -- 172.21.15.77:0/1300892363 shutdown_connections 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.234+0000 7fc3345aa700 1 -- 172.21.15.77:0/1300892363 wait complete. 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 Processor -- start 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 -- start start 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fed80 0x7fc32c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10e440 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc3345aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10e5b0 con 0x7fc32c0fed80 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46050/0 (socket says 172.21.15.77:46050) 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 -- 172.21.15.77:0/1662183663 learned_addr learned my addr 172.21.15.77:0/1662183663 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc331b45700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fed80 0x7fc32c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 -- 172.21.15.77:0/1662183663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fed80 msgr2=0x7fc32c10de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fed80 0x7fc32c10de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.235+0000 7fc332346700 1 -- 172.21.15.77:0/1662183663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320005040 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc332346700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc32400cb10 tx=0x7fc324007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc324004200 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c112fb0 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc32400ac70 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc324003d30 con 0x7fc32c0fe430 2024-04-26T04:56:59.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.236+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c113540 con 0x7fc32c0fe430 2024-04-26T04:56:59.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.237+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c04c710 con 0x7fc32c0fe430 2024-04-26T04:56:59.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.237+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc32400a7a0 con 0x7fc32c0fe430 2024-04-26T04:56:59.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.238+0000 7fc32affd700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 0x7fc314070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.238+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc32408e0b0 con 0x7fc32c0fe430 2024-04-26T04:56:59.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.238+0000 7fc331b45700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 0x7fc314070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.238+0000 7fc331b45700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 0x7fc314070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc32000b970 tx=0x7fc320009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.243+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc32405aea0 con 0x7fc32c0fe430 2024-04-26T04:56:59.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.482+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc32c113b30 con 0x7fc32c0fe430 2024-04-26T04:56:59.483 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312] boot 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: osdmap e53: 6 total, 6 up, 6 in 2024-04-26T04:56:59.484 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:56:59 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.483+0000 7fc32affd700 1 -- 172.21.15.77:0/1662183663 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fc324004360 con 0x7fc32c0fe430 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:56:59.485 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:56:59.486 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:59.487 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:56:59.488 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:56:59.488 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:59.488 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:59.488 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.485+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 msgr2=0x7fc314070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.485+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 0x7fc314070720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc32000b970 tx=0x7fc320009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.485+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 msgr2=0x7fc32c10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.485+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc32400cb10 tx=0x7fc324007f90 comp rx=0 tx=0).stop 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 shutdown_connections 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc31406e270 0x7fc314070720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc32c0fe430 0x7fc32c10d900 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 --2- 172.21.15.77:0/1662183663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc32c0fed80 0x7fc32c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 >> 172.21.15.77:0/1662183663 conn(0x7fc32c0f6bd0 msgr2=0x7fc32c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 shutdown_connections 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.486+0000 7fc3345aa700 1 -- 172.21.15.77:0/1662183663 wait complete. 2024-04-26T04:56:59.489 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: osd.0 [v2:172.21.15.77:6802/2493290312,v1:172.21.15.77:6803/2493290312] boot 2024-04-26T04:56:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: osdmap e53: 6 total, 6 up, 6 in 2024-04-26T04:56:59.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:56:59 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.588+0000 7fbaaeac6700 1 -- 172.21.15.77:0/2922690245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 msgr2=0x7fbaa80fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.588+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/2922690245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa80fe450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba94004940 tx=0x7fba94009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- 172.21.15.77:0/2922690245 shutdown_connections 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/2922690245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa80fe450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/2922690245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- 172.21.15.77:0/2922690245 >> 172.21.15.77:0/2922690245 conn(0x7fbaa80f8550 msgr2=0x7fbaa80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- 172.21.15.77:0/2922690245 shutdown_connections 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- 172.21.15.77:0/2922690245 wait complete. 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 Processor -- start 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- start start 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa810e440 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.589+0000 7fbaaeac6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa810e5b0 con 0x7fbaa80fce70 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaac862700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46054/0 (socket says 172.21.15.77:46054) 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 -- 172.21.15.77:0/487659851 learned_addr learned my addr 172.21.15.77:0/487659851 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 -- 172.21.15.77:0/487659851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 msgr2=0x7fbaa810d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa810d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 -- 172.21.15.77:0/487659851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94005040 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaac862700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa7fff700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b3b0 tx=0x7fba94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba9400e040 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa8112fb0 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba9400bbf0 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba9401c630 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.590+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa8113540 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.591+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa804c710 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.592+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fba9400bd60 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.592+0000 7fbaa57fa700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 0x7fba98070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.592+0000 7fbaac862700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 0x7fba98070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.592+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fba9408e900 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.593+0000 7fbaac862700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 0x7fba98070720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba9c010520 tx=0x7fba9c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.597+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fba9405b660 con 0x7fbaa80fdfe0 2024-04-26T04:56:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.817+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa8102350 con 0x7fba9806e270 2024-04-26T04:56:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.818+0000 7fbaa57fa700 1 -- 172.21.15.77:0/487659851 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbaa8102350 con 0x7fba9806e270 2024-04-26T04:56:59.818 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:56:59.818 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "7/21 daemons upgraded", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:56:59.819 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:56:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 msgr2=0x7fba98070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 0x7fba98070720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba9c010520 tx=0x7fba9c00b040 comp rx=0 tx=0).stop 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 msgr2=0x7fbaa810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b3b0 tx=0x7fba94009e30 comp rx=0 tx=0).stop 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 shutdown_connections 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fba9806e270 0x7fba98070720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbaa80fce70 0x7fbaa810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 --2- 172.21.15.77:0/487659851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80fdfe0 0x7fbaa810de40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 >> 172.21.15.77:0/487659851 conn(0x7fbaa80f8550 msgr2=0x7fbaa80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 shutdown_connections 2024-04-26T04:56:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.820+0000 7fbaaeac6700 1 -- 172.21.15.77:0/487659851 wait complete. 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 -- 172.21.15.77:0/3003876822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc071de0 msgr2=0x7fcecc0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3003876822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc071de0 0x7fcecc0702f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009ce0 tx=0x7fcec4004d10 comp rx=0 tx=0).stop 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 -- 172.21.15.77:0/3003876822 shutdown_connections 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3003876822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc070830 0x7fcecc070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3003876822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc071de0 0x7fcecc0702f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 -- 172.21.15.77:0/3003876822 >> 172.21.15.77:0/3003876822 conn(0x7fcecc0f85c0 msgr2=0x7fcecc0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 -- 172.21.15.77:0/3003876822 shutdown_connections 2024-04-26T04:56:59.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.936+0000 7fced0fd9700 1 -- 172.21.15.77:0/3003876822 wait complete. 2024-04-26T04:56:59.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 Processor -- start 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 -- start start 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc071de0 0x7fcecc110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc1110e0 con 0x7fcecc070830 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fced0fd9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc111250 con 0x7fcecc071de0 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46072/0 (socket says 172.21.15.77:46072) 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 -- 172.21.15.77:0/3408964175 learned_addr learned my addr 172.21.15.77:0/3408964175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fcec9d9b700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc071de0 0x7fcecc110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 -- 172.21.15.77:0/3408964175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc071de0 msgr2=0x7fcecc110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc071de0 0x7fcecc110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.937+0000 7fceca59c700 1 -- 172.21.15.77:0/3408964175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec4009940 con 0x7fcecc070830 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fceca59c700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcec40095d0 tx=0x7fcec4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcec400e040 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcec40128e0 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc1114d0 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcec401c9a0 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.938+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc111a90 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.939+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc1082e0 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.939+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 39) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcec4012410 con 0x7fcecc070830 2024-04-26T04:56:59.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.940+0000 7fcec2ffd700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 0x7fceb8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:56:59.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.940+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcec408f900 con 0x7fcecc070830 2024-04-26T04:56:59.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.940+0000 7fcec9d9b700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 0x7fceb8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:56:59.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.940+0000 7fcec9d9b700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 0x7fceb8070710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcecc10fb90 tx=0x7fceb40043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:56:59.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:56:59.945+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcec4058b30 con 0x7fcecc070830 2024-04-26T04:57:00.099 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:00 smithi077 ceph-mon[91833]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:00.100 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:00 smithi077 ceph-mon[91833]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:00.100 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:00 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2812337887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:00.100 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:00 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1662183663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:57:00.100 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:00 smithi077 ceph-mon[91833]: osdmap e54: 6 total, 6 up, 6 in 2024-04-26T04:57:00.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.237+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcecc05f5c0 con 0x7fcecc070830 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.238+0000 7fcec2ffd700 1 -- 172.21.15.77:0/3408964175 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1574 (secure 0 0 0) 0x7fcec4004e10 con 0x7fcecc070830 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-26T04:57:00.239 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-26T04:57:00.240 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 msgr2=0x7fceb8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 0x7fceb8070710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcecc10fb90 tx=0x7fceb40043c0 comp rx=0 tx=0).stop 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 msgr2=0x7fcecc10fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcec40095d0 tx=0x7fcec4004d10 comp rx=0 tx=0).stop 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 shutdown_connections 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fceb806e260 0x7fceb8070710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:00.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc070830 0x7fcecc10fe00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 --2- 172.21.15.77:0/3408964175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcecc071de0 0x7fcecc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 >> 172.21.15.77:0/3408964175 conn(0x7fcecc0f85c0 msgr2=0x7fcecc0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 shutdown_connections 2024-04-26T04:57:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:00.240+0000 7fced0fd9700 1 -- 172.21.15.77:0/3408964175 wait complete. 2024-04-26T04:57:00.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:00 smithi078 ceph-mon[79689]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:00.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:00 smithi078 ceph-mon[79689]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:00.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:00 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2812337887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:00.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:00 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1662183663' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:57:00.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:00 smithi078 ceph-mon[79689]: osdmap e54: 6 total, 6 up, 6 in 2024-04-26T04:57:01.393 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:01 smithi077 ceph-mon[91833]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:01.393 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:01 smithi077 ceph-mon[91833]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:01.393 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:01 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3408964175' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:57:01.393 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:01.393 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:01 smithi078 ceph-mon[79689]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:01 smithi078 ceph-mon[79689]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:01 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3408964175' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:57:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:03.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:02 smithi078 ceph-mon[79689]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-26T04:57:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:03.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:02 smithi077 ceph-mon[91833]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-26T04:57:05.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:04 smithi078 ceph-mon[79689]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 532 B/s wr, 0 op/s 2024-04-26T04:57:05.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:04 smithi078 ceph-mon[79689]: mgrmap e40: smithi077.hkvsor(active, since 92s), standbys: smithi078.navvuw 2024-04-26T04:57:05.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:04 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-04-26T04:57:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:04 smithi077 ceph-mon[91833]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 532 B/s wr, 0 op/s 2024-04-26T04:57:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:04 smithi077 ceph-mon[91833]: mgrmap e40: smithi077.hkvsor(active, since 92s), standbys: smithi078.navvuw 2024-04-26T04:57:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:04 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-04-26T04:57:07.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:06 smithi078 ceph-mon[79689]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 447 B/s wr, 0 op/s 2024-04-26T04:57:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:06 smithi077 ceph-mon[91833]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 447 B/s wr, 0 op/s 2024-04-26T04:57:09.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:08 smithi078 ceph-mon[79689]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 390 B/s wr, 0 op/s 2024-04-26T04:57:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:08 smithi077 ceph-mon[91833]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 390 B/s wr, 0 op/s 2024-04-26T04:57:11.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:10 smithi077 ceph-mon[91833]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-26T04:57:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:10 smithi078 ceph-mon[79689]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-26T04:57:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:12 smithi077 ceph-mon[91833]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-26T04:57:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:12 smithi078 ceph-mon[79689]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-26T04:57:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:14 smithi077 ceph-mon[91833]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-26T04:57:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:14 smithi078 ceph-mon[79689]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-26T04:57:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:16 smithi077 ceph-mon[91833]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:16 smithi078 ceph-mon[79689]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:17 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:18.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:17 smithi077 ceph-mon[91833]: Upgrade: osd.1 is safe to restart 2024-04-26T04:57:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:17 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:57:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:17 smithi078 ceph-mon[79689]: Upgrade: osd.1 is safe to restart 2024-04-26T04:57:19.210 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:18 smithi077 ceph-mon[91833]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:19.210 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:19.210 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:57:19.210 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:19.249 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:18 smithi078 ceph-mon[79689]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:19.249 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:19.249 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:57:19.249 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:20.017 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:19 smithi077 ceph-mon[91833]: Upgrade: Updating osd.1 2024-04-26T04:57:20.017 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:19 smithi077 ceph-mon[91833]: Deploying daemon osd.1 on smithi077 2024-04-26T04:57:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:19 smithi078 ceph-mon[79689]: Upgrade: Updating osd.1 2024-04-26T04:57:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:19 smithi078 ceph-mon[79689]: Deploying daemon osd.1 on smithi077 2024-04-26T04:57:20.972 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:20 smithi077 ceph-mon[91833]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:20 smithi078 ceph-mon[79689]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:23.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:22 smithi077 ceph-mon[91833]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:22 smithi078 ceph-mon[79689]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:23.492 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:23 smithi077 systemd[1]: Stopping Ceph osd.1 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:57:23.492 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:23 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[52055]: 2024-04-26T04:57:23.436+0000 7fa253772700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:57:23.493 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:23 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[52055]: 2024-04-26T04:57:23.436+0000 7fa253772700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-26T04:57:23.493 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:23 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[52055]: 2024-04-26T04:57:23.436+0000 7fa253772700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:57:24.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:23 smithi077 ceph-mon[91833]: osd.1 marked itself down and dead 2024-04-26T04:57:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:23 smithi078 ceph-mon[79689]: osd.1 marked itself down and dead 2024-04-26T04:57:24.588 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:24 smithi077 podman[106509]: 2024-04-26 04:57:24.30411916 +0000 UTC m=+0.968480156 container died 69f7a839880bac0df317863cf1b1003b71ad3037011af796aabfa061c403e749 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, RELEASE=quincy-c61472c, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T04:57:24.949 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:24 smithi077 podman[106509]: 2024-04-26 04:57:24.670589278 +0000 UTC m=+1.334950277 container cleanup 69f7a839880bac0df317863cf1b1003b71ad3037011af796aabfa061c403e749 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T04:57:24.949 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:24 smithi077 bash[106509]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1 2024-04-26T04:57:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:24 smithi077 ceph-mon[91833]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:24 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:57:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:24 smithi077 ceph-mon[91833]: osdmap e55: 6 total, 5 up, 6 in 2024-04-26T04:57:25.243 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:24 smithi077 podman[106527]: 2024-04-26 04:57:24.947079374 +0000 UTC m=+0.639108176 container remove 69f7a839880bac0df317863cf1b1003b71ad3037011af796aabfa061c403e749 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T04:57:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:24 smithi078 ceph-mon[79689]: pgmap v69: 65 pgs: 65 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:24 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:57:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:24 smithi078 ceph-mon[79689]: osdmap e55: 6 total, 5 up, 6 in 2024-04-26T04:57:25.562 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.191934895 +0000 UTC m=+0.018245684 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:57:25.562 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.340303597 +0000 UTC m=+0.166614372 container create 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T04:57:25.858 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.559820965 +0000 UTC m=+0.386131749 container init 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:57:25.859 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.563794112 +0000 UTC m=+0.390104897 container start 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b22e2eb) 2024-04-26T04:57:25.859 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.65122249 +0000 UTC m=+0.477533288 container attach 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T04:57:26.138 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:25 smithi077 ceph-mon[91833]: osdmap e56: 6 total, 5 up, 6 in 2024-04-26T04:57:26.139 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:25 smithi077 podman[106621]: 2024-04-26 04:57:25.858682715 +0000 UTC m=+0.684993490 container died 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T04:57:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:25 smithi078 ceph-mon[79689]: osdmap e56: 6 total, 5 up, 6 in 2024-04-26T04:57:26.743 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 podman[106647]: 2024-04-26 04:57:26.455363598 +0000 UTC m=+0.591540174 container remove 13b32f77a501653969f133f32f7f5b69ed866292363ebc555192ece83fef48a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, ceph=True) 2024-04-26T04:57:26.743 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-26T04:57:26.743 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 systemd[1]: Stopped Ceph osd.1 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:57:26.743 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.1.service: Consumed 54.030s CPU time. 2024-04-26T04:57:26.743 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 systemd[1]: Starting Ceph osd.1 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:57:27.185 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:26 smithi077 ceph-mon[91833]: pgmap v72: 65 pgs: 12 stale+active+clean, 53 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:27.186 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 podman[106751]: 2024-04-26 04:57:26.794389785 +0000 UTC m=+0.018921408 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:57:27.186 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:26 smithi077 podman[106751]: 2024-04-26 04:57:26.898805247 +0000 UTC m=+0.123336897 container create 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:57:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:26 smithi078 ceph-mon[79689]: pgmap v72: 65 pgs: 12 stale+active+clean, 53 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:27.492 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 podman[106751]: 2024-04-26 04:57:27.185169491 +0000 UTC m=+0.409701098 container init 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T04:57:27.492 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 podman[106751]: 2024-04-26 04:57:27.188724129 +0000 UTC m=+0.413255746 container start 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:57:27.493 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 podman[106751]: 2024-04-26 04:57:27.293506209 +0000 UTC m=+0.518037840 container attach 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb) 2024-04-26T04:57:27.947 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:57:27.947 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:57:27.947 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 bash[106751]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:57:27.948 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate[106767]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-26T04:57:27.949 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:27 smithi077 podman[106751]: 2024-04-26 04:57:27.655583999 +0000 UTC m=+0.880115610 container died 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=squid-b22e2eb, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:57:28.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:28 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:57:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:28 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:57:28.605 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 podman[106905]: 2024-04-26 04:57:28.256473693 +0000 UTC m=+0.593942196 container remove 8f53fd952f62cf27a0ac753b931a59f002224afa0987f01d9ba9328564c71ff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T04:57:28.605 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 podman[106971]: 2024-04-26 04:57:28.37293682 +0000 UTC m=+0.018434996 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:57:28.605 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 podman[106971]: 2024-04-26 04:57:28.482533902 +0000 UTC m=+0.128032054 container create 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:57:28.992 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 podman[106971]: 2024-04-26 04:57:28.710251754 +0000 UTC m=+0.355749915 container init 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T04:57:28.993 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 podman[106971]: 2024-04-26 04:57:28.71329549 +0000 UTC m=+0.358793645 container start 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T04:57:28.993 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 bash[106971]: 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 2024-04-26T04:57:28.993 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:28 smithi077 systemd[1]: Started Ceph osd.1 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:57:29.317 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:29 smithi078 ceph-mon[79689]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:29.317 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:29 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:29.317 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:29 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:29.317 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:29 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:29.492 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:29 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T04:57:29.253+0000 7f4f65a247c0 -1 Falling back to public interface 2024-04-26T04:57:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:29 smithi077 ceph-mon[91833]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:29 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:29 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:29.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:29 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 -- 172.21.15.77:0/937974771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fce60 msgr2=0x7f0ed40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 --2- 172.21.15.77:0/937974771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fce60 0x7f0ed40fd270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0004940 tx=0x7f0ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 -- 172.21.15.77:0/937974771 shutdown_connections 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 --2- 172.21.15.77:0/937974771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 --2- 172.21.15.77:0/937974771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fce60 0x7f0ed40fd270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 -- 172.21.15.77:0/937974771 >> 172.21.15.77:0/937974771 conn(0x7f0ed40f8560 msgr2=0x7f0ed40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 -- 172.21.15.77:0/937974771 shutdown_connections 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.347+0000 7f0edc69c700 1 -- 172.21.15.77:0/937974771 wait complete. 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 Processor -- start 2024-04-26T04:57:30.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 -- start start 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fce60 0x7f0ed410da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed410e560 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0edc69c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed410edb0 con 0x7f0ed40fce60 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0ed9c37700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0ed9c37700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39146/0 (socket says 172.21.15.77:39146) 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0ed9c37700 1 -- 172.21.15.77:0/1017462728 learned_addr learned my addr 172.21.15.77:0/1017462728 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.348+0000 7f0eda438700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fce60 0x7f0ed410da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ed9c37700 1 -- 172.21.15.77:0/1017462728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fce60 msgr2=0x7f0ed410da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ed9c37700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fce60 0x7f0ed410da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ed9c37700 1 -- 172.21.15.77:0/1017462728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed0005040 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ed9c37700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400c830 tx=0x7f0ec4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:30.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec40106f0 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed410f090 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ec4010d20 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec40113a0 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.349+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed410f5d0 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.350+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed405f5c0 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.351+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0ec4012400 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.351+0000 7f0ecaffd700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.351+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0ec408da20 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.351+0000 7f0eda438700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.352+0000 7f0eda438700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 0x7f0ec00705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000b970 tx=0x7f0ed0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:30.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.357+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0ec405a9a0 con 0x7f0ed40fdfd0 2024-04-26T04:57:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.589+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ed410f950 con 0x7f0ec006e140 2024-04-26T04:57:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.590+0000 7f0ecaffd700 1 -- 172.21.15.77:0/1017462728 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0ed410f950 con 0x7f0ec006e140 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 msgr2=0x7f0ec00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 0x7f0ec00705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000b970 tx=0x7f0ed0009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 msgr2=0x7f0ed410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400c830 tx=0x7f0ec4004470 comp rx=0 tx=0).stop 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 shutdown_connections 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f0ec006e140 0x7f0ec00705f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40fce60 0x7f0ed410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 --2- 172.21.15.77:0/1017462728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed40fdfd0 0x7f0ed410df60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 >> 172.21.15.77:0/1017462728 conn(0x7f0ed40f8560 msgr2=0x7f0ed40f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 shutdown_connections 2024-04-26T04:57:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.592+0000 7f0edc69c700 1 -- 172.21.15.77:0/1017462728 wait complete. 2024-04-26T04:57:30.604 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 -- 172.21.15.77:0/4289841477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fcf70 msgr2=0x7fb9d80fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 --2- 172.21.15.77:0/4289841477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fcf70 0x7fb9d80fd3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc004940 tx=0x7fb9cc009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 -- 172.21.15.77:0/4289841477 shutdown_connections 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 --2- 172.21.15.77:0/4289841477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fcf70 0x7fb9d80fd3e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 --2- 172.21.15.77:0/4289841477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d80fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 -- 172.21.15.77:0/4289841477 >> 172.21.15.77:0/4289841477 conn(0x7fb9d80f7440 msgr2=0x7fb9d80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 -- 172.21.15.77:0/4289841477 shutdown_connections 2024-04-26T04:57:30.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.698+0000 7fb9dd683700 1 -- 172.21.15.77:0/4289841477 wait complete. 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 Processor -- start 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 -- start start 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fcf70 0x7fb9d8112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8112b90 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9dd683700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8112cd0 con 0x7fb9d80fcf70 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9d6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9d6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39158/0 (socket says 172.21.15.77:39158) 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9d6ffd700 1 -- 172.21.15.77:0/1239392278 learned_addr learned my addr 172.21.15.77:0/1239392278 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9d6ffd700 1 -- 172.21.15.77:0/1239392278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fcf70 msgr2=0x7fb9d8112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.699+0000 7fb9d67fc700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fcf70 0x7fb9d8112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9d6ffd700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fcf70 0x7fb9d8112650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9d6ffd700 1 -- 172.21.15.77:0/1239392278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc005040 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9d6ffd700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8002a80 tx=0x7fb9c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c8007cc0 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9c800f040 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d8112fb0 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c80096f0 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.700+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d81134e0 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.701+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d804c710 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.701+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb9c800d690 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.702+0000 7fb9bf7fe700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 0x7fb9c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:30.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.702+0000 7fb9d67fc700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 0x7fb9c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:30.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.702+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb9c808d2f0 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.702+0000 7fb9d67fc700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 0x7fb9c00705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:30.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.706+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb9c805a190 con 0x7fb9d80fbcc0 2024-04-26T04:57:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.929+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9d8101210 con 0x7fb9c006e140 2024-04-26T04:57:30.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.931+0000 7fb9bf7fe700 1 -- 172.21.15.77:0/1239392278 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb9d8101210 con 0x7fb9c006e140 2024-04-26T04:57:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 msgr2=0x7fb9c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 0x7fb9c00705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc000c00 tx=0x7fb9cc009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 msgr2=0x7fb9d8066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8002a80 tx=0x7fb9c80046b0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 shutdown_connections 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9c006e140 0x7fb9c00705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9d80fbcc0 0x7fb9d8066d10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 --2- 172.21.15.77:0/1239392278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9d80fcf70 0x7fb9d8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 >> 172.21.15.77:0/1239392278 conn(0x7fb9d80f7440 msgr2=0x7fb9d80f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 shutdown_connections 2024-04-26T04:57:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:30.934+0000 7fb9dd683700 1 -- 172.21.15.77:0/1239392278 wait complete. 2024-04-26T04:57:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 -- 172.21.15.77:0/63978887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fce60 msgr2=0x7f78580fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 --2- 172.21.15.77:0/63978887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fce60 0x7f78580fd270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7848004940 tx=0x7f7848009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 -- 172.21.15.77:0/63978887 shutdown_connections 2024-04-26T04:57:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 --2- 172.21.15.77:0/63978887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fdfd0 0x7f78580fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 --2- 172.21.15.77:0/63978887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fce60 0x7f78580fd270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 -- 172.21.15.77:0/63978887 >> 172.21.15.77:0/63978887 conn(0x7f78580f8540 msgr2=0x7f78580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 -- 172.21.15.77:0/63978887 shutdown_connections 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.053+0000 7f785e295700 1 -- 172.21.15.77:0/63978887 wait complete. 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 Processor -- start 2024-04-26T04:57:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 -- start start 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 0x7f785810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785810e5f0 con 0x7f78580fdfd0 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f785e295700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785810ede0 con 0x7f78580fce60 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f78577fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f78577fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39178/0 (socket says 172.21.15.77:39178) 2024-04-26T04:57:31.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.054+0000 7f78577fe700 1 -- 172.21.15.77:0/3704123096 learned_addr learned my addr 172.21.15.77:0/3704123096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f78577fe700 1 -- 172.21.15.77:0/3704123096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 msgr2=0x7f785810da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f7857fff700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 0x7f785810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f78577fe700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 0x7f785810da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f78577fe700 1 -- 172.21.15.77:0/3704123096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848005040 con 0x7f78580fdfd0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f7857fff700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 0x7f785810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f78577fe700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b790 tx=0x7f784c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c00f770 con 0x7f78580fdfd0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785810f0c0 con 0x7f78580fdfd0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c00fda0 con 0x7f78580fdfd0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c00d310 con 0x7f78580fdfd0 2024-04-26T04:57:31.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.055+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785810f680 con 0x7f78580fdfd0 2024-04-26T04:57:31.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.056+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785805f5c0 con 0x7f78580fdfd0 2024-04-26T04:57:31.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.057+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f784c010420 con 0x7f78580fdfd0 2024-04-26T04:57:31.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.057+0000 7f7854ff9700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 0x7f78400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.058+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f784c08daf0 con 0x7f78580fdfd0 2024-04-26T04:57:31.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.058+0000 7f7857fff700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 0x7f78400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.058+0000 7f7857fff700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 0x7f78400705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f784800b970 tx=0x7f7848009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.062+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f784c05aa70 con 0x7f78580fdfd0 2024-04-26T04:57:31.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:31 smithi077 ceph-mon[91833]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:31.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:31.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:31.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.281+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f785810f970 con 0x7f784006e140 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.291+0000 7f7854ff9700 1 -- 172.21.15.77:0/3704123096 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f785810f970 con 0x7f784006e140 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (4m) 34s ago 12m 26.3M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (66s) 34s ago 13m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (60s) 57s ago 12m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 34s ago 12m - - 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (7m) 34s ago 7m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (7m) 34s ago 7m 17.0M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (7m) 57s ago 7m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:57:31.293 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (7m) 57s ago 7m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (3m) 34s ago 14m 500M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (2m) 57s ago 12m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (2m) 34s ago 14m 43.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (106s) 57s ago 11m 31.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (4m) 34s ago 13m 16.0M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (3m) 57s ago 12m 16.7M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (40s) 34s ago 11m 13.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 starting - - - 1557M 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (10m) 34s ago 10m 176M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (9m) 57s ago 9m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (9m) 57s ago 9m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:57:31.294 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (9m) 57s ago 9m 193M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:57:31.295 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (2m) 34s ago 12m 56.6M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:57:31.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 msgr2=0x7f78400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 0x7f78400705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f784800b970 tx=0x7f7848009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:31.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 msgr2=0x7f785810df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b790 tx=0x7f784c009250 comp rx=0 tx=0).stop 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 shutdown_connections 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f784006e140 0x7f78400705f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78580fce60 0x7f785810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 --2- 172.21.15.77:0/3704123096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78580fdfd0 0x7f785810df60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 >> 172.21.15.77:0/3704123096 conn(0x7f78580f8540 msgr2=0x7f78580f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 shutdown_connections 2024-04-26T04:57:31.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.294+0000 7f785e295700 1 -- 172.21.15.77:0/3704123096 wait complete. 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1738408781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fbcf0 msgr2=0x7f18c80fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1738408781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fbcf0 0x7f18c80fc100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18b8004940 tx=0x7f18b8009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1738408781 shutdown_connections 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1738408781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fcfa0 0x7f18c80fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1738408781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fbcf0 0x7f18c80fc100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1738408781 >> 172.21.15.77:0/1738408781 conn(0x7f18c80f7450 msgr2=0x7f18c80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1738408781 shutdown_connections 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1738408781 wait complete. 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 Processor -- start 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- start start 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fbcf0 0x7f18c8066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8112c80 con 0x7f18c80fcfa0 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.395+0000 7f18cdbdb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8112dc0 con 0x7f18c80fbcf0 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fbcf0 0x7f18c8066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39198/0 (socket says 172.21.15.77:39198) 2024-04-26T04:57:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 -- 172.21.15.77:0/1733800979 learned_addr learned my addr 172.21.15.77:0/1733800979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 -- 172.21.15.77:0/1733800979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fbcf0 msgr2=0x7f18c8066cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fbcf0 0x7f18c8066cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 -- 172.21.15.77:0/1733800979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b8005040 con 0x7f18c80fcfa0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18bffff700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18b0002a50 tx=0x7f18b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b0007cc0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18b000f040 con 0x7f18c80fcfa0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c8113020 con 0x7f18c80fcfa0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b000da30 con 0x7f18c80fcfa0 2024-04-26T04:57:31.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.396+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81135b0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.397+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c804c710 con 0x7f18c80fcfa0 2024-04-26T04:57:31.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.398+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f18b000dbb0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.398+0000 7f18c4ff9700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 0x7f18b40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.398+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f18b008d8a0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.398+0000 7f18c77fe700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 0x7f18b40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.398+0000 7f18c77fe700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 0x7f18b40705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b970 tx=0x7f18b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.403+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f18b005a810 con 0x7f18c80fcfa0 2024-04-26T04:57:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:31 smithi078 ceph-mon[79689]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.697+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18c81047e0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.698+0000 7f18c4ff9700 1 -- 172.21.15.77:0/1733800979 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f18b005a3a0 con 0x7f18c80fcfa0 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:57:31.700 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 5 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:57:31.701 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 msgr2=0x7f18b40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 0x7f18b40705f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b970 tx=0x7f18b8009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 msgr2=0x7f18c8112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18b0002a50 tx=0x7f18b00046b0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 shutdown_connections 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f18b406e140 0x7f18b40705f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18c80fbcf0 0x7f18c8066cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 --2- 172.21.15.77:0/1733800979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18c80fcfa0 0x7f18c8112740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 >> 172.21.15.77:0/1733800979 conn(0x7f18c80f7450 msgr2=0x7f18c80f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 shutdown_connections 2024-04-26T04:57:31.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.701+0000 7f18cdbdb700 1 -- 172.21.15.77:0/1733800979 wait complete. 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1507019367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fbd40 msgr2=0x7fa9480fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1507019367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fbd40 0x7fa9480fc150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa938004940 tx=0x7fa938009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1507019367 shutdown_connections 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1507019367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fcff0 0x7fa9480fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1507019367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fbd40 0x7fa9480fc150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1507019367 >> 172.21.15.77:0/1507019367 conn(0x7fa9480f74c0 msgr2=0x7fa9480f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1507019367 shutdown_connections 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.832+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1507019367 wait complete. 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 Processor -- start 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 -- start start 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480fe8e0 con 0x7fa9480fcff0 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94ccc0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480fea50 con 0x7fa9480fbd40 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:60866/0 (socket says 172.21.15.77:60866) 2024-04-26T04:57:31.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa94659c700 1 -- 172.21.15.77:0/1145297779 learned_addr learned my addr 172.21.15.77:0/1145297779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:31.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.833+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39224/0 (socket says 172.21.15.77:39224) 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94659c700 1 -- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 msgr2=0x7fa9480fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94659c700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94659c700 1 -- 172.21.15.77:0/1145297779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa938005040 con 0x7fa9480fbd40 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa93ffff700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94659c700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b0d0 tx=0x7fa938009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93800e030 con 0x7fa9480fbd40 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9480fecd0 con 0x7fa9480fbd40 2024-04-26T04:57:31.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93801c400 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9481159b0 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.834+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93800b630 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.835+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94805f5c0 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.836+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa938013af0 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.836+0000 7fa93effd700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 0x7fa9340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.836+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa93808f350 con 0x7fa9480fbd40 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.836+0000 7fa93ffff700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 0x7fa9340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:31.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.837+0000 7fa93ffff700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 0x7fa9340705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa930009b80 tx=0x7fa93000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:31.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:31.842+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa9380587c0 con 0x7fa9480fbd40 2024-04-26T04:57:32.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.129+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa948104900 con 0x7fa9480fbd40 2024-04-26T04:57:32.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.130+0000 7fa93effd700 1 -- 172.21.15.77:0/1145297779 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fa93805be80 con 0x7fa9480fbd40 2024-04-26T04:57:32.132 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:57:32.132 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:57:32.132 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:57:32.133 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:57:32.134 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:57:32.135 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 msgr2=0x7fa9340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 0x7fa9340705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa930009b80 tx=0x7fa93000b040 comp rx=0 tx=0).stop 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 msgr2=0x7fa9480ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b0d0 tx=0x7fa938009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 shutdown_connections 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa93406e140 0x7fa9340705f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9480fbd40 0x7fa9480ffe60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.132+0000 7fa94ccc0700 1 --2- 172.21.15.77:0/1145297779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9480fcff0 0x7fa9480fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.133+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 >> 172.21.15.77:0/1145297779 conn(0x7fa9480f74c0 msgr2=0x7fa948066380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:32.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.133+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 shutdown_connections 2024-04-26T04:57:32.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.133+0000 7fa94ccc0700 1 -- 172.21.15.77:0/1145297779 wait complete. 2024-04-26T04:57:32.137 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- 172.21.15.77:0/4148125887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4070830 msgr2=0x7f40b4070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 --2- 172.21.15.77:0/4148125887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4070830 0x7f40b4070ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40a4004940 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- 172.21.15.77:0/4148125887 shutdown_connections 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 --2- 172.21.15.77:0/4148125887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4070830 0x7f40b4070ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 --2- 172.21.15.77:0/4148125887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4071de0 0x7f40b40702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- 172.21.15.77:0/4148125887 >> 172.21.15.77:0/4148125887 conn(0x7f40b40f8520 msgr2=0x7f40b40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- 172.21.15.77:0/4148125887 shutdown_connections 2024-04-26T04:57:32.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- 172.21.15.77:0/4148125887 wait complete. 2024-04-26T04:57:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:32 smithi077 ceph-mon[91833]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:32 smithi077 ceph-mon[91833]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:32 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1733800979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:32.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 Processor -- start 2024-04-26T04:57:32.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.239+0000 7f40bc47f700 1 -- start start 2024-04-26T04:57:32.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40bc47f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4070830 0x7f40b410fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40bc47f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40bc47f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4110650 con 0x7f40b4071de0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40bc47f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41107c0 con 0x7f40b4070830 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39228/0 (socket says 172.21.15.77:39228) 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 -- 172.21.15.77:0/1805995854 learned_addr learned my addr 172.21.15.77:0/1805995854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40ba21b700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4070830 0x7f40b410fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 -- 172.21.15.77:0/1805995854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4070830 msgr2=0x7f40b410fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4070830 0x7f40b410fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 -- 172.21.15.77:0/1805995854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4005040 con 0x7f40b4071de0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40b9a1a700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b3b0 tx=0x7f40a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a400e040 con 0x7f40b4071de0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.240+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b41151c0 con 0x7f40b4071de0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.241+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40a400bbf0 con 0x7f40b4071de0 2024-04-26T04:57:32.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.241+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b4115780 con 0x7f40b4071de0 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.241+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a401c630 con 0x7f40b4071de0 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.242+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b404c710 con 0x7f40b4071de0 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.242+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40a4013420 con 0x7f40b4071de0 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.243+0000 7f40aaffd700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 0x7f40a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.243+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f40a408e5d0 con 0x7f40b4071de0 2024-04-26T04:57:32.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.243+0000 7f40ba21b700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 0x7f40a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.244+0000 7f40ba21b700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 0x7f40a00705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40b4071b20 tx=0x7f40b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:32.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.248+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f40a405b5f0 con 0x7f40b4071de0 2024-04-26T04:57:32.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.479+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40b4103eb0 con 0x7f40a006e140 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.480+0000 7f40aaffd700 1 -- 172.21.15.77:0/1805995854 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f40b4103eb0 con 0x7f40a006e140 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "7/21 daemons upgraded", 2024-04-26T04:57:32.482 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:57:32.483 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:57:32.483 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:57:32.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 msgr2=0x7f40a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 0x7f40a00705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40b4071b20 tx=0x7f40b000b040 comp rx=0 tx=0).stop 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 msgr2=0x7f40b4110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40a400b3b0 tx=0x7f40a4009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 shutdown_connections 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40a006e140 0x7f40a00705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b4070830 0x7f40b410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 --2- 172.21.15.77:0/1805995854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b4071de0 0x7f40b4110050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 >> 172.21.15.77:0/1805995854 conn(0x7f40b40f8520 msgr2=0x7f40b40f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 shutdown_connections 2024-04-26T04:57:32.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.483+0000 7f40bc47f700 1 -- 172.21.15.77:0/1805995854 wait complete. 2024-04-26T04:57:32.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:32 smithi078 ceph-mon[79689]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:32.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:32 smithi078 ceph-mon[79689]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:32.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:32 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1733800979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:32.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 -- 172.21.15.77:0/1921792319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 msgr2=0x7f71b00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 --2- 172.21.15.77:0/1921792319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b00fc0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71ac004940 tx=0x7f71ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:32.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 -- 172.21.15.77:0/1921792319 shutdown_connections 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 --2- 172.21.15.77:0/1921792319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 0x7f71b00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 --2- 172.21.15.77:0/1921792319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b00fc0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 -- 172.21.15.77:0/1921792319 >> 172.21.15.77:0/1921792319 conn(0x7f71b00f7430 msgr2=0x7f71b00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 -- 172.21.15.77:0/1921792319 shutdown_connections 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.593+0000 7f71b8120700 1 -- 172.21.15.77:0/1921792319 wait complete. 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 Processor -- start 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 -- start start 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 0x7f71b0112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0066350 con 0x7f71b00fbcb0 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b8120700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0066490 con 0x7f71b00fcf60 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39244/0 (socket says 172.21.15.77:39244) 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 -- 172.21.15.77:0/1790747660 learned_addr learned my addr 172.21.15.77:0/1790747660 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b56bb700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 0x7f71b0112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 -- 172.21.15.77:0/1790747660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 msgr2=0x7f71b0112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 0x7f71b0112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 -- 172.21.15.77:0/1790747660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac005040 con 0x7f71b00fbcb0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71b5ebc700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b3b0 tx=0x7f71ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.594+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ac00e040 con 0x7f71b00fbcb0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.595+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b01131f0 con 0x7f71b00fbcb0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.595+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0113780 con 0x7f71b00fbcb0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.595+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71ac004290 con 0x7f71b00fbcb0 2024-04-26T04:57:32.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.595+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ac01c3f0 con 0x7f71b00fbcb0 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.596+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b005f5c0 con 0x7f71b00fbcb0 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.596+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f71ac01c660 con 0x7f71b00fbcb0 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.596+0000 7f71a67fc700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 0x7f719c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.596+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f71ac08ece0 con 0x7f71b00fbcb0 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.596+0000 7f71b56bb700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 0x7f719c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:57:32.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.597+0000 7f71b56bb700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 0x7f719c070600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71a0007fd0 tx=0x7f71a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:57:32.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.601+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f71ac058240 con 0x7f71b00fbcb0 2024-04-26T04:57:32.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.885+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f71b004bed0 con 0x7f71b00fbcb0 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.885+0000 7f71a67fc700 1 -- 172.21.15.77:0/1790747660 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f71ac05b900 con 0x7f71b00fbcb0 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: osd.1 (root=default,host=smithi077) is down 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/264 objects degraded (12.879%), 13 pgs degraded 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-26T04:57:32.888 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-26T04:57:32.889 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 msgr2=0x7f719c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 0x7f719c070600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71a0007fd0 tx=0x7f71a0009040 comp rx=0 tx=0).stop 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 msgr2=0x7f71b0110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b3b0 tx=0x7f71ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 shutdown_connections 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f719c06e150 0x7f719c070600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71b00fbcb0 0x7f71b0110700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 --2- 172.21.15.77:0/1790747660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71b00fcf60 0x7f71b0112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 >> 172.21.15.77:0/1790747660 conn(0x7f71b00f7430 msgr2=0x7f71b00f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:57:32.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 shutdown_connections 2024-04-26T04:57:32.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:57:32.888+0000 7f71b8120700 1 -- 172.21.15.77:0/1790747660 wait complete. 2024-04-26T04:57:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:33 smithi077 ceph-mon[91833]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:33 smithi077 ceph-mon[91833]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:33 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1145297779' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:57:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:33 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1790747660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:57:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:33 smithi078 ceph-mon[79689]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:33 smithi078 ceph-mon[79689]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:33 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1145297779' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:57:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:33 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1790747660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:57:34.327 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:34 smithi077 ceph-mon[91833]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:34.327 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:34 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T04:57:34.147+0000 7f4f65a247c0 -1 osd.1 54 log_to_monitors true 2024-04-26T04:57:34.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:34 smithi078 ceph-mon[79689]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:57:35.134 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 04:57:35 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T04:57:35.057+0000 7f4f56e7c700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:57:35.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:35 smithi077 ceph-mon[91833]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:35.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:35 smithi077 ceph-mon[91833]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:57:35.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:35.135 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:35 smithi078 ceph-mon[79689]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:35 smithi078 ceph-mon[79689]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:57:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:36 smithi077 ceph-mon[91833]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:57:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:36 smithi077 ceph-mon[91833]: osdmap e57: 6 total, 5 up, 6 in 2024-04-26T04:57:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:36 smithi077 ceph-mon[91833]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:57:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:36 smithi078 ceph-mon[79689]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:57:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:36 smithi078 ceph-mon[79689]: osdmap e57: 6 total, 5 up, 6 in 2024-04-26T04:57:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:36 smithi078 ceph-mon[79689]: from='osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435] boot 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: osdmap e58: 6 total, 6 up, 6 in 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:57:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:57:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: osd.1 [v2:172.21.15.77:6810/3696890435,v1:172.21.15.77:6811/3696890435] boot 2024-04-26T04:57:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: osdmap e58: 6 total, 6 up, 6 in 2024-04-26T04:57:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:57:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:38 smithi077 ceph-mon[91833]: osdmap e59: 6 total, 6 up, 6 in 2024-04-26T04:57:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:38 smithi078 ceph-mon[79689]: osdmap e59: 6 total, 6 up, 6 in 2024-04-26T04:57:39.400 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:39 smithi077 ceph-mon[91833]: pgmap v81: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-26T04:57:39.400 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:39 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-04-26T04:57:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:39 smithi078 ceph-mon[79689]: pgmap v81: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-26T04:57:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:39 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: pgmap v82: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:40.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:40.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:40 smithi077 ceph-mon[91833]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: pgmap v82: 65 pgs: 22 peering, 7 active+undersized+degraded, 5 active+undersized, 31 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 18/264 objects degraded (6.818%) 2024-04-26T04:57:40.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:40.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:40 smithi078 ceph-mon[79689]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-26T04:57:41.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:41 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded) 2024-04-26T04:57:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:41 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 7 pgs degraded) 2024-04-26T04:57:42.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:42 smithi077 ceph-mon[91833]: pgmap v83: 65 pgs: 22 peering, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:42 smithi078 ceph-mon[79689]: pgmap v83: 65 pgs: 22 peering, 43 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:44.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:44 smithi077 ceph-mon[91833]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:45.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:44 smithi078 ceph-mon[79689]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:46 smithi077 ceph-mon[91833]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:46 smithi078 ceph-mon[79689]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:57:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:48 smithi077 ceph-mon[91833]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:48 smithi078 ceph-mon[79689]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:50 smithi077 ceph-mon[91833]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:50 smithi078 ceph-mon[79689]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:52 smithi077 ceph-mon[91833]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:52 smithi078 ceph-mon[79689]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:55.226 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:54 smithi077 ceph-mon[91833]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:55.226 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:54 smithi078 ceph-mon[79689]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:56.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:55 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:55 smithi077 ceph-mon[91833]: Upgrade: osd.2 is safe to restart 2024-04-26T04:57:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:55 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:57:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:55 smithi078 ceph-mon[79689]: Upgrade: osd.2 is safe to restart 2024-04-26T04:57:57.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:56 smithi077 ceph-mon[91833]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:57.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:57.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:57:57.134 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:56 smithi078 ceph-mon[79689]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:57:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:57:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:57:58.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:57 smithi077 ceph-mon[91833]: Upgrade: Updating osd.2 2024-04-26T04:57:58.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:57 smithi077 ceph-mon[91833]: Deploying daemon osd.2 on smithi077 2024-04-26T04:57:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:57 smithi078 ceph-mon[79689]: Upgrade: Updating osd.2 2024-04-26T04:57:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:57 smithi078 ceph-mon[79689]: Deploying daemon osd.2 on smithi077 2024-04-26T04:57:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:57:58 smithi077 ceph-mon[91833]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:57:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:57:58 smithi078 ceph-mon[79689]: pgmap v91: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:58:00.743 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:00 smithi077 systemd[1]: Stopping Ceph osd.2 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:58:00.743 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[59848]: 2024-04-26T04:58:00.710+0000 7f4ac0a1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:58:00.743 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[59848]: 2024-04-26T04:58:00.710+0000 7f4ac0a1f700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-26T04:58:00.743 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[59848]: 2024-04-26T04:58:00.710+0000 7f4ac0a1f700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:58:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-mon[91833]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:58:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-mon[91833]: osd.2 marked itself down and dead 2024-04-26T04:58:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:00 smithi078 ceph-mon[79689]: pgmap v92: 65 pgs: 65 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:58:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:00 smithi078 ceph-mon[79689]: osd.2 marked itself down and dead 2024-04-26T04:58:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:01.742 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:01 smithi077 podman[112465]: 2024-04-26 04:58:01.43470352 +0000 UTC m=+0.821052953 container died e2fe4f5ff169f12d19a0f77308ea988cbb3db182b8e86065d81f21ef480c7a46 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T04:58:02.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:01 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:58:02.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:01 smithi077 ceph-mon[91833]: osdmap e60: 6 total, 5 up, 6 in 2024-04-26T04:58:02.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112465]: 2024-04-26 04:58:02.094847024 +0000 UTC m=+1.481196499 container cleanup e2fe4f5ff169f12d19a0f77308ea988cbb3db182b8e86065d81f21ef480c7a46 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T04:58:02.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 bash[112465]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2 2024-04-26T04:58:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:01 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:58:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:01 smithi078 ceph-mon[79689]: osdmap e60: 6 total, 5 up, 6 in 2024-04-26T04:58:02.691 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112482]: 2024-04-26 04:58:02.387941475 +0000 UTC m=+0.952583086 container remove e2fe4f5ff169f12d19a0f77308ea988cbb3db182b8e86065d81f21ef480c7a46 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T04:58:02.976 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112577]: 2024-04-26 04:58:02.593628498 +0000 UTC m=+0.018274220 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:02.976 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112577]: 2024-04-26 04:58:02.736123317 +0000 UTC m=+0.160769021 container create 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T04:58:02.976 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112577]: 2024-04-26 04:58:02.975504791 +0000 UTC m=+0.400150499 container init 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T04:58:03.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- 172.21.15.77:0/2244717077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fcf90 msgr2=0x7f16f40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/2244717077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fcf90 0x7f16f40fd400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16e8004940 tx=0x7f16e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- 172.21.15.77:0/2244717077 shutdown_connections 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/2244717077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fcf90 0x7f16f40fd400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/2244717077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fbce0 0x7f16f40fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- 172.21.15.77:0/2244717077 >> 172.21.15.77:0/2244717077 conn(0x7f16f40f7420 msgr2=0x7f16f40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- 172.21.15.77:0/2244717077 shutdown_connections 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- 172.21.15.77:0/2244717077 wait complete. 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 Processor -- start 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.000+0000 7f16f8a5d700 1 -- start start 2024-04-26T04:58:03.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f8a5d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f8a5d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 0x7f16f410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f8a5d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f410e460 con 0x7f16f40fbce0 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f8a5d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f410e5d0 con 0x7f16f40fcf90 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47778/0 (socket says 172.21.15.77:47778) 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 -- 172.21.15.77:0/4110371657 learned_addr learned my addr 172.21.15.77:0/4110371657 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f1d9b700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 0x7f16f410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 -- 172.21.15.77:0/4110371657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 msgr2=0x7f16f410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 0x7f16f410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 -- 172.21.15.77:0/4110371657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8005040 con 0x7f16f40fbce0 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f1d9b700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 0x7f16f410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:58:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f259c700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16e4002a50 tx=0x7f16e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e4007cc0 con 0x7f16f40fbce0 2024-04-26T04:58:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f4112fd0 con 0x7f16f40fbce0 2024-04-26T04:58:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e400f040 con 0x7f16f40fbce0 2024-04-26T04:58:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.001+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16e4009640 con 0x7f16f40fbce0 2024-04-26T04:58:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.002+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f4113560 con 0x7f16f40fbce0 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.003+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4101b20 con 0x7f16f40fbce0 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.003+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f16e40097a0 con 0x7f16f40fbce0 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.004+0000 7f16e2ffd700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 0x7f16dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.004+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f16e408d8d0 con 0x7f16f40fbce0 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.004+0000 7f16f1d9b700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 0x7f16dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.004+0000 7f16f1d9b700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 0x7f16dc0705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16e800b7a0 tx=0x7f16e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.008+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f16e405a7d0 con 0x7f16f40fbce0 2024-04-26T04:58:03.235 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:02 smithi077 ceph-mon[91833]: pgmap v94: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:58:03.235 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:02 smithi077 ceph-mon[91833]: osdmap e61: 6 total, 5 up, 6 in 2024-04-26T04:58:03.236 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:02 smithi077 podman[112577]: 2024-04-26 04:58:02.97913542 +0000 UTC m=+0.403781129 container start 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:58:03.236 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 podman[112577]: 2024-04-26 04:58:03.067271572 +0000 UTC m=+0.491917285 container attach 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T04:58:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.234+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f402b450 con 0x7f16dc06e140 2024-04-26T04:58:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.235+0000 7f16e2ffd700 1 -- 172.21.15.77:0/4110371657 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f16f402b450 con 0x7f16dc06e140 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 msgr2=0x7f16dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 0x7f16dc0705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16e800b7a0 tx=0x7f16e8009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 msgr2=0x7f16f410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16e4002a50 tx=0x7f16e40046b0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 shutdown_connections 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f16dc06e140 0x7f16dc0705f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16f40fbce0 0x7f16f410d920 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 --2- 172.21.15.77:0/4110371657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16f40fcf90 0x7f16f410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 >> 172.21.15.77:0/4110371657 conn(0x7f16f40f7420 msgr2=0x7f16f40f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 shutdown_connections 2024-04-26T04:58:03.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.238+0000 7f16f8a5d700 1 -- 172.21.15.77:0/4110371657 wait complete. 2024-04-26T04:58:03.250 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:58:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:02 smithi078 ceph-mon[79689]: pgmap v94: 65 pgs: 9 stale+active+clean, 56 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-26T04:58:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:02 smithi078 ceph-mon[79689]: osdmap e61: 6 total, 5 up, 6 in 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 -- 172.21.15.77:0/2939606512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 msgr2=0x7f01440fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 --2- 172.21.15.77:0/2939606512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fd3d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0138004940 tx=0x7f0138009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 -- 172.21.15.77:0/2939606512 shutdown_connections 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 --2- 172.21.15.77:0/2939606512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fd3d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 --2- 172.21.15.77:0/2939606512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 0x7f01440fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 -- 172.21.15.77:0/2939606512 >> 172.21.15.77:0/2939606512 conn(0x7f01440f7430 msgr2=0x7f01440f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 -- 172.21.15.77:0/2939606512 shutdown_connections 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.346+0000 7f01492c1700 1 -- 172.21.15.77:0/2939606512 wait complete. 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 Processor -- start 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 -- start start 2024-04-26T04:58:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 0x7f01440ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440fe8e0 con 0x7f01440fcf60 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01492c1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440fea50 con 0x7f01440fbcb0 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47802/0 (socket says 172.21.15.77:47802) 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 -- 172.21.15.77:0/2250355203 learned_addr learned my addr 172.21.15.77:0/2250355203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f0142ffd700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 0x7f01440ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 -- 172.21.15.77:0/2250355203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 msgr2=0x7f01440ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 0x7f01440ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.347+0000 7f01427fc700 1 -- 172.21.15.77:0/2250355203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138005040 con 0x7f01440fcf60 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f01427fc700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f013800b3b0 tx=0x7f0138009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013800e040 con 0x7f01440fcf60 2024-04-26T04:58:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01440fecd0 con 0x7f01440fcf60 2024-04-26T04:58:03.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f013800bcc0 con 0x7f01440fcf60 2024-04-26T04:58:03.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013801c630 con 0x7f01440fcf60 2024-04-26T04:58:03.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.348+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144115a10 con 0x7f01440fcf60 2024-04-26T04:58:03.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.349+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144101af0 con 0x7f01440fcf60 2024-04-26T04:58:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.349+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0138013980 con 0x7f01440fcf60 2024-04-26T04:58:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.350+0000 7f012b7fe700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 0x7f012c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.350+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f013808e720 con 0x7f01440fcf60 2024-04-26T04:58:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.350+0000 7f0142ffd700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 0x7f012c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.350+0000 7f0142ffd700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 0x7f012c0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0134006fd0 tx=0x7f0134008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.355+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0138057b70 con 0x7f01440fcf60 2024-04-26T04:58:03.492 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 podman[112577]: 2024-04-26 04:58:03.279734552 +0000 UTC m=+0.704380250 container died 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T04:58:03.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.578+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f014405e3a0 con 0x7f012c06e140 2024-04-26T04:58:03.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.579+0000 7f012b7fe700 1 -- 172.21.15.77:0/2250355203 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f014405e3a0 con 0x7f012c06e140 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 msgr2=0x7f012c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 0x7f012c0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0134006fd0 tx=0x7f0134008040 comp rx=0 tx=0).stop 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 msgr2=0x7f01440fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f013800b3b0 tx=0x7f0138009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 shutdown_connections 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f012c06e140 0x7f012c0705f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01440fbcb0 0x7f01440ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 --2- 172.21.15.77:0/2250355203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01440fcf60 0x7f01440fe2a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 >> 172.21.15.77:0/2250355203 conn(0x7f01440f7430 msgr2=0x7f01440f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 shutdown_connections 2024-04-26T04:58:03.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.581+0000 7f01492c1700 1 -- 172.21.15.77:0/2250355203 wait complete. 2024-04-26T04:58:03.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- 172.21.15.77:0/1280580310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 msgr2=0x7fe7880fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 --2- 172.21.15.77:0/1280580310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe7880fd270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe778004940 tx=0x7fe778009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- 172.21.15.77:0/1280580310 shutdown_connections 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 --2- 172.21.15.77:0/1280580310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 0x7fe7880fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 --2- 172.21.15.77:0/1280580310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe7880fd270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- 172.21.15.77:0/1280580310 >> 172.21.15.77:0/1280580310 conn(0x7fe7880f8540 msgr2=0x7fe7880fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- 172.21.15.77:0/1280580310 shutdown_connections 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- 172.21.15.77:0/1280580310 wait complete. 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 Processor -- start 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.684+0000 7fe7906ec700 1 -- start start 2024-04-26T04:58:03.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 0x7fe7880702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788072200 con 0x7fe7880fce60 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880707f0 con 0x7fe7880fdfd0 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47824/0 (socket says 172.21.15.77:47824) 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 -- 172.21.15.77:0/2099347350 learned_addr learned my addr 172.21.15.77:0/2099347350 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78dc87700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 0x7fe7880702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 -- 172.21.15.77:0/2099347350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 msgr2=0x7fe7880702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 0x7fe7880702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 -- 172.21.15.77:0/2099347350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778005040 con 0x7fe7880fce60 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe78e488700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b3b0 tx=0x7fe778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77800e040 con 0x7fe7880fce60 2024-04-26T04:58:03.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe778004290 con 0x7fe7880fce60 2024-04-26T04:58:03.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe788070a40 con 0x7fe7880fce60 2024-04-26T04:58:03.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77801c630 con 0x7fe7880fce60 2024-04-26T04:58:03.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.685+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788071000 con 0x7fe7880fce60 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.686+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78804c710 con 0x7fe7880fce60 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.687+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe77800bbf0 con 0x7fe7880fce60 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.687+0000 7fe77effd700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 0x7fe7740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.687+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe77808ee70 con 0x7fe7880fce60 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.688+0000 7fe78dc87700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 0x7fe7740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:03.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.688+0000 7fe78dc87700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 0x7fe7740705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe784004990 tx=0x7fe784009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.692+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe77805be90 con 0x7fe7880fce60 2024-04-26T04:58:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.900+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe788102360 con 0x7fe77406e140 2024-04-26T04:58:03.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.910+0000 7fe77effd700 1 -- 172.21.15.77:0/2099347350 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe788102360 con 0x7fe77406e140 2024-04-26T04:58:03.911 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:58:03.911 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (4m) 28s ago 13m 25.2M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:58:03.911 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (98s) 28s ago 13m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (93s) 90s ago 12m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 28s ago 13m - - 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (8m) 28s ago 8m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (7m) 28s ago 7m 17.1M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (7m) 90s ago 7m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (8m) 90s ago 8m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (3m) 28s ago 14m 502M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (2m) 90s ago 12m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (2m) 28s ago 15m 45.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (2m) 90s ago 12m 31.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (5m) 28s ago 13m 16.2M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (4m) 90s ago 12m 16.7M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (73s) 28s ago 11m 86.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:58:03.912 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (35s) 28s ago 11m 12.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T04:58:03.913 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (10m) 28s ago 10m 177M 1557M 17.2.7-902-gc61472c8 0de407c1b3af e2fe4f5ff169 2024-04-26T04:58:03.913 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (10m) 90s ago 10m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:58:03.913 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (10m) 90s ago 10m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:58:03.913 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (9m) 90s ago 9m 193M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:58:03.913 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (3m) 28s ago 13m 58.9M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 msgr2=0x7fe7740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 0x7fe7740705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe784004990 tx=0x7fe784009ee0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 msgr2=0x7fe788071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b3b0 tx=0x7fe778009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 shutdown_connections 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe77406e140 0x7fe7740705f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7880fce60 0x7fe788071c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 --2- 172.21.15.77:0/2099347350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7880fdfd0 0x7fe7880702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 >> 172.21.15.77:0/2099347350 conn(0x7fe7880f8540 msgr2=0x7fe7880f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 shutdown_connections 2024-04-26T04:58:03.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:03.913+0000 7fe7906ec700 1 -- 172.21.15.77:0/2099347350 wait complete. 2024-04-26T04:58:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 -- 172.21.15.77:0/1679356382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 msgr2=0x7f40280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 --2- 172.21.15.77:0/1679356382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f40280fd3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f401c004940 tx=0x7f401c009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 -- 172.21.15.77:0/1679356382 shutdown_connections 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 --2- 172.21.15.77:0/1679356382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f40280fd3d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 --2- 172.21.15.77:0/1679356382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f40280fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 -- 172.21.15.77:0/1679356382 >> 172.21.15.77:0/1679356382 conn(0x7f40280f7430 msgr2=0x7f40280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 -- 172.21.15.77:0/1679356382 shutdown_connections 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 -- 172.21.15.77:0/1679356382 wait complete. 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.024+0000 7f402e56d700 1 Processor -- start 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f402e56d700 1 -- start start 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f402e56d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f402810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f402e56d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f402e56d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402810e430 con 0x7f40280fcf60 2024-04-26T04:58:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f402e56d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402810e5a0 con 0x7f40280fbcb0 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47844/0 (socket says 172.21.15.77:47844) 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 -- 172.21.15.77:0/3414895547 learned_addr learned my addr 172.21.15.77:0/3414895547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f4027fff700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f402810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 -- 172.21.15.77:0/3414895547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 msgr2=0x7f402810d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f402810d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 -- 172.21.15.77:0/3414895547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c005040 con 0x7f40280fcf60 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f4027fff700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f402810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.025+0000 7f40277fe700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f401c00b3b0 tx=0x7f401c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401c00e040 con 0x7f40280fcf60 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f401c00bcc0 con 0x7f40280fcf60 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4028112fa0 con 0x7f40280fcf60 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401c01c630 con 0x7f40280fcf60 2024-04-26T04:58:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4028113530 con 0x7f40280fcf60 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.026+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4028101af0 con 0x7f40280fcf60 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.027+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f401c013980 con 0x7f40280fcf60 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.027+0000 7f4024ff9700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 0x7f40100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.028+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f401c08e7a0 con 0x7f40280fcf60 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.028+0000 7f4027fff700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 0x7f40100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.028+0000 7f4027fff700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 0x7f40100705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4018009fd0 tx=0x7f4018009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.032+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f401c05b6a0 con 0x7f40280fcf60 2024-04-26T04:58:04.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:04 smithi077 ceph-mon[91833]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:58:04.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:04 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:04.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 podman[112631]: 2024-04-26 04:58:03.904799878 +0000 UTC m=+0.620013851 container remove 8b44833c5e66d83b975399fc939894bb41c1e4e1b271cb7a0fc359b32de80321 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T04:58:04.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-26T04:58:04.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 systemd[1]: Stopped Ceph osd.2 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:58:04.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:03 smithi077 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.2.service: Consumed 45.380s CPU time. 2024-04-26T04:58:04.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 systemd[1]: Starting Ceph osd.2 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:58:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.346+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40281047a0 con 0x7f40280fcf60 2024-04-26T04:58:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.346+0000 7f4024ff9700 1 -- 172.21.15.77:0/3414895547 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f401c05b230 con 0x7f40280fcf60 2024-04-26T04:58:04.347 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:58:04.347 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:58:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 msgr2=0x7f40100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 0x7f40100705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4018009fd0 tx=0x7f4018009ee0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 msgr2=0x7f402810de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f401c00b3b0 tx=0x7f401c009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 shutdown_connections 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f401006e140 0x7f40100705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40280fbcb0 0x7f402810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 --2- 172.21.15.77:0/3414895547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40280fcf60 0x7f402810de30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 >> 172.21.15.77:0/3414895547 conn(0x7f40280f7430 msgr2=0x7f4028066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 shutdown_connections 2024-04-26T04:58:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.349+0000 7f402e56d700 1 -- 172.21.15.77:0/3414895547 wait complete. 2024-04-26T04:58:04.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 -- 172.21.15.77:0/190585757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1340fdd10 msgr2=0x7fb134100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/190585757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1340fdd10 0x7fb134100190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb120004940 tx=0x7fb120009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 -- 172.21.15.77:0/190585757 shutdown_connections 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/190585757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1340fdd10 0x7fb134100190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/190585757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb134066f30 0x7fb1340fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 -- 172.21.15.77:0/190585757 >> 172.21.15.77:0/190585757 conn(0x7fb1340f7450 msgr2=0x7fb1340f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 -- 172.21.15.77:0/190585757 shutdown_connections 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.452+0000 7fb13b1f3700 1 -- 172.21.15.77:0/190585757 wait complete. 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 Processor -- start 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 -- start start 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1340fdd10 0x7fb134114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134114a20 con 0x7fb134066f30 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb13b1f3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134114b90 con 0x7fb1340fdd10 2024-04-26T04:58:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb138f8f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb138f8f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47858/0 (socket says 172.21.15.77:47858) 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.453+0000 7fb138f8f700 1 -- 172.21.15.77:0/1692766159 learned_addr learned my addr 172.21.15.77:0/1692766159 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb133fff700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1340fdd10 0x7fb134114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb138f8f700 1 -- 172.21.15.77:0/1692766159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1340fdd10 msgr2=0x7fb134114420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb138f8f700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1340fdd10 0x7fb134114420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb138f8f700 1 -- 172.21.15.77:0/1692766159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120005040 con 0x7fb134066f30 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb138f8f700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eb40 tx=0x7fb12800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb128007cc0 con 0x7fb134066f30 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12800f040 con 0x7fb134066f30 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb134119520 con 0x7fb134066f30 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12800cad0 con 0x7fb134066f30 2024-04-26T04:58:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.454+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb134119a60 con 0x7fb134066f30 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.455+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13404c710 con 0x7fb134066f30 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.456+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb12800cc30 con 0x7fb134066f30 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.456+0000 7fb1317fa700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 0x7fb1240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.456+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb12808d5b0 con 0x7fb134066f30 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.456+0000 7fb133fff700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 0x7fb1240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.457+0000 7fb133fff700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 0x7fb1240705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb12000b0d0 tx=0x7fb120009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.461+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb12805a530 con 0x7fb134066f30 2024-04-26T04:58:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:04 smithi078 ceph-mon[79689]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:58:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:04 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:04.601 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 podman[112800]: 2024-04-26 04:58:04.259529096 +0000 UTC m=+0.018591259 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:04.601 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 podman[112800]: 2024-04-26 04:58:04.372705429 +0000 UTC m=+0.131767588 container create 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T04:58:04.601 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 podman[112800]: 2024-04-26 04:58:04.600772659 +0000 UTC m=+0.359834813 container init 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T04:58:04.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.717+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb134119eb0 con 0x7fb134066f30 2024-04-26T04:58:04.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.718+0000 7fb1317fa700 1 -- 172.21.15.77:0/1692766159 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fb12805a0c0 con 0x7fb134066f30 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:58:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:58:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:04.721 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:04.722 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:04.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 msgr2=0x7fb1240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 0x7fb1240705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb12000b0d0 tx=0x7fb120009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 msgr2=0x7fb134113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eb40 tx=0x7fb12800b040 comp rx=0 tx=0).stop 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 shutdown_connections 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb12406e140 0x7fb1240705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb134066f30 0x7fb134113ee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 --2- 172.21.15.77:0/1692766159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1340fdd10 0x7fb134114420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 >> 172.21.15.77:0/1692766159 conn(0x7fb1340f7450 msgr2=0x7fb1340f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 shutdown_connections 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.720+0000 7fb13b1f3700 1 -- 172.21.15.77:0/1692766159 wait complete. 2024-04-26T04:58:04.723 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 -- 172.21.15.77:0/2749971179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fcf60 msgr2=0x7f666c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 --2- 172.21.15.77:0/2749971179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fcf60 0x7f666c0fd3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6658004940 tx=0x7f6658009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 -- 172.21.15.77:0/2749971179 shutdown_connections 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 --2- 172.21.15.77:0/2749971179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fcf60 0x7f666c0fd3d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 --2- 172.21.15.77:0/2749971179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fbcb0 0x7f666c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 -- 172.21.15.77:0/2749971179 >> 172.21.15.77:0/2749971179 conn(0x7f666c0f7430 msgr2=0x7f666c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 -- 172.21.15.77:0/2749971179 shutdown_connections 2024-04-26T04:58:04.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.825+0000 7f6672d82700 1 -- 172.21.15.77:0/2749971179 wait complete. 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 Processor -- start 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 -- start start 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fcf60 0x7f666c112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c066350 con 0x7f666c0fbcb0 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.826+0000 7f6672d82700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c066490 con 0x7f666c0fcf60 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47876/0 (socket says 172.21.15.77:47876) 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 -- 172.21.15.77:0/3678487251 learned_addr learned my addr 172.21.15.77:0/3678487251 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f666bfff700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fcf60 0x7f666c112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 -- 172.21.15.77:0/3678487251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fcf60 msgr2=0x7f666c112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fcf60 0x7f666c112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 -- 172.21.15.77:0/3678487251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658005040 con 0x7f666c0fbcb0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6670b1e700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6660002a50 tx=0x7f66600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6660007cc0 con 0x7f666c0fbcb0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f666000f040 con 0x7f666c0fbcb0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c113280 con 0x7f666c0fbcb0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666000da70 con 0x7f666c0fbcb0 2024-04-26T04:58:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.827+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c113810 con 0x7f666c0fbcb0 2024-04-26T04:58:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.828+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c05f5c0 con 0x7f666c0fbcb0 2024-04-26T04:58:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.829+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f666000dc10 con 0x7f666c0fbcb0 2024-04-26T04:58:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.829+0000 7f66697fa700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 0x7f665c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.829+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f666008d0b0 con 0x7f666c0fbcb0 2024-04-26T04:58:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.829+0000 7f666bfff700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 0x7f665c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.830+0000 7f666bfff700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 0x7f665c070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6658009fd0 tx=0x7f6658009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:04.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:04.834+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6660056500 con 0x7f666c0fbcb0 2024-04-26T04:58:04.992 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 podman[112800]: 2024-04-26 04:58:04.604151296 +0000 UTC m=+0.363213458 container start 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b22e2eb) 2024-04-26T04:58:04.993 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:04 smithi077 podman[112800]: 2024-04-26 04:58:04.709215493 +0000 UTC m=+0.468277657 container attach 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T04:58:05.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.061+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f666c1012d0 con 0x7f665c06e150 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.062+0000 7f66697fa700 1 -- 172.21.15.77:0/3678487251 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f666c1012d0 con 0x7f665c06e150 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:58:05.063 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:58:05.064 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "8/21 daemons upgraded", 2024-04-26T04:58:05.064 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:58:05.064 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:58:05.064 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:58:05.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.064+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 msgr2=0x7f665c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.064+0000 7f6672d82700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 0x7f665c070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6658009fd0 tx=0x7f6658009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.064+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 msgr2=0x7f666c110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.064+0000 7f6672d82700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6660002a50 tx=0x7f66600046b0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 shutdown_connections 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f665c06e150 0x7f665c070600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f666c0fbcb0 0x7f666c110730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 --2- 172.21.15.77:0/3678487251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f666c0fcf60 0x7f666c112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 >> 172.21.15.77:0/3678487251 conn(0x7f666c0f7430 msgr2=0x7f666c0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 shutdown_connections 2024-04-26T04:58:05.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.065+0000 7f6672d82700 1 -- 172.21.15.77:0/3678487251 wait complete. 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- 172.21.15.77:0/3348801501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fdfd0 msgr2=0x7f2ad80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/3348801501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad80fe440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2acc004940 tx=0x7f2acc009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- 172.21.15.77:0/3348801501 shutdown_connections 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/3348801501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad80fe440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/3348801501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fce60 0x7f2ad80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- 172.21.15.77:0/3348801501 >> 172.21.15.77:0/3348801501 conn(0x7f2ad80f8540 msgr2=0x7f2ad80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- 172.21.15.77:0/3348801501 shutdown_connections 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- 172.21.15.77:0/3348801501 wait complete. 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 Processor -- start 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- start start 2024-04-26T04:58:05.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad8110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad8110660 con 0x7f2ad80fce60 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.170+0000 7f2adcd6c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad81107d0 con 0x7f2ad80fdfd0 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47888/0 (socket says 172.21.15.77:47888) 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 -- 172.21.15.77:0/1854351962 learned_addr learned my addr 172.21.15.77:0/1854351962 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad5d9b700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad8110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 -- 172.21.15.77:0/1854351962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fdfd0 msgr2=0x7f2ad8110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad8110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 -- 172.21.15.77:0/1854351962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc005040 con 0x7f2ad80fce60 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ad659c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c830 tx=0x7f2ac8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac800a760 con 0x7f2ad80fce60 2024-04-26T04:58:05.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ac800ad90 con 0x7f2ad80fce60 2024-04-26T04:58:05.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac80113a0 con 0x7f2ad80fce60 2024-04-26T04:58:05.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad8115230 con 0x7f2ad80fce60 2024-04-26T04:58:05.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.171+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad81157f0 con 0x7f2ad80fce60 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.172+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad8101960 con 0x7f2ad80fce60 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.173+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ac8012400 con 0x7f2ad80fce60 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.173+0000 7f2ac6ffd700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 0x7f2ac00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.173+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2ac808da80 con 0x7f2ad80fce60 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.173+0000 7f2ad5d9b700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 0x7f2ac00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:05.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.174+0000 7f2ad5d9b700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 0x7f2ac00705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b3b0 tx=0x7f2acc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.178+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2ac805aaa0 con 0x7f2ad80fce60 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: pgmap v96: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10/264 objects degraded (3.788%) 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3414895547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:05.396 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1692766159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 bash[112800]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:58:05.397 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-26T04:58:05.398 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:58:05.398 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate[112837]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-26T04:58:05.398 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 podman[112800]: 2024-04-26 04:58:05.089372556 +0000 UTC m=+0.848434704 container died 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T04:58:05.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.466+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2ad804c710 con 0x7f2ad80fce60 2024-04-26T04:58:05.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.467+0000 7f2ac6ffd700 1 -- 172.21.15.77:0/1854351962 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1090 (secure 0 0 0) 0x7f2ac805a630 con 0x7f2ad80fce60 2024-04-26T04:58:05.467 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering; Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: osd.2 (root=default,host=smithi077) is down 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.c is stuck peering for 3m, current state peering, last acting [3,0] 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 3.1e is stuck peering for 3m, current state peering, last acting [3,1] 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-04-26T04:58:05.468 INFO:teuthology.orchestra.run.smithi077.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-04-26T04:58:05.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 msgr2=0x7f2ac00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 0x7f2ac00705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b3b0 tx=0x7f2acc009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:05.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 msgr2=0x7f2ad810fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c830 tx=0x7f2ac8004470 comp rx=0 tx=0).stop 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 shutdown_connections 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2ac006e140 0x7f2ac00705f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ad80fce60 0x7f2ad810fb20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 --2- 172.21.15.77:0/1854351962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ad80fdfd0 0x7f2ad8110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 >> 172.21.15.77:0/1854351962 conn(0x7f2ad80f8540 msgr2=0x7f2ad80f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 shutdown_connections 2024-04-26T04:58:05.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:05.469+0000 7f2adcd6c700 1 -- 172.21.15.77:0/1854351962 wait complete. 2024-04-26T04:58:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: pgmap v96: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10/264 objects degraded (3.788%) 2024-04-26T04:58:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3414895547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:05 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1692766159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:58:06.055 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 podman[113001]: 2024-04-26 04:58:05.746970312 +0000 UTC m=+0.649100716 container remove 422ab74325e643d2633e44d2f4ea5b3acd465d709d506462177796dbe451a69c (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T04:58:06.055 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 podman[113089]: 2024-04-26 04:58:05.859428619 +0000 UTC m=+0.017720157 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:06.055 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:05 smithi077 podman[113089]: 2024-04-26 04:58:05.98992369 +0000 UTC m=+0.148215196 container create 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:58:06.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:06 smithi077 ceph-mon[91833]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:06.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:06 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1854351962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:58:06.493 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:06 smithi077 podman[113089]: 2024-04-26 04:58:06.201437872 +0000 UTC m=+0.359729388 container init 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T04:58:06.493 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:06 smithi077 podman[113089]: 2024-04-26 04:58:06.205091083 +0000 UTC m=+0.363382590 container start 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T04:58:06.493 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:06 smithi077 bash[113089]: 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd 2024-04-26T04:58:06.493 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:06 smithi077 systemd[1]: Started Ceph osd.2 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:58:06.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:06 smithi078 ceph-mon[79689]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:06.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:06 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1854351962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:58:07.056 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:06 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T04:58:06.751+0000 7f4ed056e7c0 -1 Falling back to public interface 2024-04-26T04:58:07.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:07 smithi077 ceph-mon[91833]: pgmap v97: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/264 objects degraded (9.091%) 2024-04-26T04:58:07.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:07.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:07.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:07 smithi078 ceph-mon[79689]: pgmap v97: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/264 objects degraded (9.091%) 2024-04-26T04:58:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:08.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:08 smithi077 ceph-mon[91833]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:08.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:08 smithi077 ceph-mon[91833]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:58:08.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:08 smithi078 ceph-mon[79689]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:08.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:08 smithi078 ceph-mon[79689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:58:10.966 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:10 smithi077 ceph-mon[91833]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:11.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:10 smithi078 ceph-mon[79689]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:11.242 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:10 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T04:58:10.964+0000 7f4ed056e7c0 -1 osd.2 59 log_to_monitors true 2024-04-26T04:58:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:11 smithi077 ceph-mon[91833]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:58:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:11 smithi078 ceph-mon[79689]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:58:12.992 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T04:58:12.722+0000 7f4ebd9be700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:58:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: osdmap e62: 6 total, 5 up, 6 in 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:12.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:12 smithi077 ceph-mon[91833]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' 2024-04-26T04:58:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: pgmap v100: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:58:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: osdmap e62: 6 total, 5 up, 6 in 2024-04-26T04:58:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-04-26T04:58:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:13.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:13.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:13.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:12 smithi078 ceph-mon[79689]: from='osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377]' entity='osd.2' 2024-04-26T04:58:14.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:13 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:58:14.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:13 smithi077 ceph-mon[91833]: osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377] boot 2024-04-26T04:58:14.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:13 smithi077 ceph-mon[91833]: osdmap e63: 6 total, 6 up, 6 in 2024-04-26T04:58:14.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:13 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:58:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:13 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:58:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:13 smithi078 ceph-mon[79689]: osd.2 [v2:172.21.15.77:6818/3580780377,v1:172.21.15.77:6819/3580780377] boot 2024-04-26T04:58:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:13 smithi078 ceph-mon[79689]: osdmap e63: 6 total, 6 up, 6 in 2024-04-26T04:58:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:13 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:58:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:14 smithi077 ceph-mon[91833]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:14 smithi077 ceph-mon[91833]: osdmap e64: 6 total, 6 up, 6 in 2024-04-26T04:58:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:14 smithi078 ceph-mon[79689]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/264 objects degraded (12.879%) 2024-04-26T04:58:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:14 smithi078 ceph-mon[79689]: osdmap e64: 6 total, 6 up, 6 in 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: pgmap v105: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: pgmap v106: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: pgmap v107: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: pgmap v105: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: pgmap v106: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: pgmap v107: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 258 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%) 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:17.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:18 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:18 smithi078 ceph-mon[79689]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-26T04:58:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:18 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 30/264 objects degraded (11.364%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:18.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:18 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:18.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:18 smithi077 ceph-mon[91833]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-26T04:58:18.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:18 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 30/264 objects degraded (11.364%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:19.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:19 smithi077 ceph-mon[91833]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:19.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:19 smithi078 ceph-mon[79689]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:20.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:20 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/264 objects degraded (11.364%), 11 pgs degraded) 2024-04-26T04:58:20.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:20 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/264 objects degraded (11.364%), 11 pgs degraded) 2024-04-26T04:58:21.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:21 smithi077 ceph-mon[91833]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:21.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:21 smithi078 ceph-mon[79689]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:23 smithi077 ceph-mon[91833]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:23 smithi078 ceph-mon[79689]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:25.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:25 smithi077 ceph-mon[91833]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:25.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:25 smithi078 ceph-mon[79689]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:26 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:26 smithi077 ceph-mon[91833]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:26 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:26 smithi078 ceph-mon[79689]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:28.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:28 smithi077 ceph-mon[91833]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:28.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:28 smithi078 ceph-mon[79689]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:30.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:30 smithi077 ceph-mon[91833]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:30.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:30 smithi078 ceph-mon[79689]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:32.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:32.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: Upgrade: osd.3 is safe to restart 2024-04-26T04:58:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:58:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:32 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:33.252 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:58:33.252 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: Upgrade: osd.3 is safe to restart 2024-04-26T04:58:33.252 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: pgmap v115: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:33.253 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:33.253 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:58:33.253 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:32 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:34.166 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:33 smithi078 ceph-mon[79689]: Upgrade: Updating osd.3 2024-04-26T04:58:34.166 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:33 smithi078 ceph-mon[79689]: Deploying daemon osd.3 on smithi078 2024-04-26T04:58:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:33 smithi077 ceph-mon[91833]: Upgrade: Updating osd.3 2024-04-26T04:58:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:33 smithi077 ceph-mon[91833]: Deploying daemon osd.3 on smithi078 2024-04-26T04:58:35.167 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:34 smithi077 ceph-mon[91833]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:34 smithi078 ceph-mon[79689]: pgmap v116: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- 172.21.15.77:0/1289701214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 msgr2=0x7feaec0fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 --2- 172.21.15.77:0/1289701214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec0fc6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feae8004940 tx=0x7feae8009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- 172.21.15.77:0/1289701214 shutdown_connections 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 --2- 172.21.15.77:0/1289701214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 0x7feaec0fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 --2- 172.21.15.77:0/1289701214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec0fc6c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- 172.21.15.77:0/1289701214 >> 172.21.15.77:0/1289701214 conn(0x7feaec0f6b80 msgr2=0x7feaec0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- 172.21.15.77:0/1289701214 shutdown_connections 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- 172.21.15.77:0/1289701214 wait complete. 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 Processor -- start 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.579+0000 7feaf427a700 1 -- start start 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf427a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf427a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 0x7feaec10ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf427a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec10d480 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf427a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec10dcd0 con 0x7feaec0fcc00 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34242/0 (socket says 172.21.15.77:34242) 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 -- 172.21.15.77:0/1634924380 learned_addr learned my addr 172.21.15.77:0/1634924380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf1815700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 0x7feaec10ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 -- 172.21.15.77:0/1634924380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 msgr2=0x7feaec10ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 0x7feaec10ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 -- 172.21.15.77:0/1634924380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8005040 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf2016700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feae800b3b0 tx=0x7feae8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae800e040 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec10df50 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feae800bbf0 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.580+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feae801c630 con 0x7feaec0fc2b0 2024-04-26T04:58:35.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.581+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec10e490 con 0x7feaec0fc2b0 2024-04-26T04:58:35.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.582+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec05f5c0 con 0x7feaec0fc2b0 2024-04-26T04:58:35.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.582+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feae800bd60 con 0x7feaec0fc2b0 2024-04-26T04:58:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.582+0000 7feae2ffd700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 0x7fead80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.582+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7feae808e690 con 0x7feaec0fc2b0 2024-04-26T04:58:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.582+0000 7feaf1815700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 0x7fead80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.583+0000 7feaf1815700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 0x7fead80705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feadc0041d0 tx=0x7feadc0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:35.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.587+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feae805b610 con 0x7feaec0fc2b0 2024-04-26T04:58:35.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.825+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feaec10e810 con 0x7fead806e140 2024-04-26T04:58:35.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.826+0000 7feae2ffd700 1 -- 172.21.15.77:0/1634924380 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7feaec10e810 con 0x7fead806e140 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 msgr2=0x7fead80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 0x7fead80705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feadc0041d0 tx=0x7feadc0043c0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 msgr2=0x7feaec10c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feae800b3b0 tx=0x7feae8009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 shutdown_connections 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead806e140 0x7fead80705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaec0fc2b0 0x7feaec10c940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 --2- 172.21.15.77:0/1634924380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaec0fcc00 0x7feaec10ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 >> 172.21.15.77:0/1634924380 conn(0x7feaec0f6b80 msgr2=0x7feaec0f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 shutdown_connections 2024-04-26T04:58:35.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.829+0000 7feaf427a700 1 -- 172.21.15.77:0/1634924380 wait complete. 2024-04-26T04:58:35.840 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 -- 172.21.15.77:0/3251985603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 msgr2=0x7f59580fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3251985603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f59580fe450 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5948004940 tx=0x7f5948009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 -- 172.21.15.77:0/3251985603 shutdown_connections 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3251985603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f59580fe450 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3251985603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 0x7f59580fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 -- 172.21.15.77:0/3251985603 >> 172.21.15.77:0/3251985603 conn(0x7f59580f8510 msgr2=0x7f59580fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 -- 172.21.15.77:0/3251985603 shutdown_connections 2024-04-26T04:58:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.934+0000 7f595fd1f700 1 -- 172.21.15.77:0/3251985603 wait complete. 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 Processor -- start 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 -- start start 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 0x7f595810d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810e3d0 con 0x7f59580fdfe0 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595fd1f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810e540 con 0x7f59580fce70 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595d2ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595d2ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34260/0 (socket says 172.21.15.77:34260) 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595d2ba700 1 -- 172.21.15.77:0/3757988809 learned_addr learned my addr 172.21.15.77:0/3757988809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.935+0000 7f595dabb700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 0x7f595810d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f595d2ba700 1 -- 172.21.15.77:0/3757988809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 msgr2=0x7f595810d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f595d2ba700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 0x7f595810d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f595d2ba700 1 -- 172.21.15.77:0/3757988809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948005040 con 0x7f59580fdfe0 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f595d2ba700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f594800b3b0 tx=0x7f5948009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594800e040 con 0x7f59580fdfe0 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5948004290 con 0x7f59580fdfe0 2024-04-26T04:58:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5958112f40 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.936+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594801c3f0 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.937+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581134d0 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.937+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5948013420 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.938+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595804c710 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.938+0000 7f594e7fc700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 0x7f59440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.938+0000 7f595dabb700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 0x7f59440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.938+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f594808e590 con 0x7f59580fdfe0 2024-04-26T04:58:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.939+0000 7f595dabb700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 0x7f59440705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5954009fd0 tx=0x7f595400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:35.943+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5948093060 con 0x7f59580fdfe0 2024-04-26T04:58:36.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:35 smithi078 systemd[1]: Stopping Ceph osd.3 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:58:36.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.177+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5958113710 con 0x7f594406e140 2024-04-26T04:58:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.179+0000 7f594e7fc700 1 -- 172.21.15.77:0/3757988809 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5958113710 con 0x7f594406e140 2024-04-26T04:58:36.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 msgr2=0x7f59440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 0x7f59440705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5954009fd0 tx=0x7f595400b040 comp rx=0 tx=0).stop 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 msgr2=0x7f595810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f594800b3b0 tx=0x7f5948009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 shutdown_connections 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f594406e140 0x7f59440705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59580fce70 0x7f595810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 --2- 172.21.15.77:0/3757988809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59580fdfe0 0x7f595810ddd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 >> 172.21.15.77:0/3757988809 conn(0x7f59580f8510 msgr2=0x7f59580f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 shutdown_connections 2024-04-26T04:58:36.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.182+0000 7f595fd1f700 1 -- 172.21.15.77:0/3757988809 wait complete. 2024-04-26T04:58:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 -- 172.21.15.77:0/2217079021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fe080 msgr2=0x7f534c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 --2- 172.21.15.77:0/2217079021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fe080 0x7f534c0fe4f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5340004940 tx=0x7f5340009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 -- 172.21.15.77:0/2217079021 shutdown_connections 2024-04-26T04:58:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 --2- 172.21.15.77:0/2217079021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fe080 0x7f534c0fe4f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 --2- 172.21.15.77:0/2217079021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c0fce80 0x7f534c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 -- 172.21.15.77:0/2217079021 >> 172.21.15.77:0/2217079021 conn(0x7f534c0f8590 msgr2=0x7f534c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 -- 172.21.15.77:0/2217079021 shutdown_connections 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.301+0000 7f5351296700 1 -- 172.21.15.77:0/2217079021 wait complete. 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.302+0000 7f5351296700 1 Processor -- start 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.302+0000 7f5351296700 1 -- start start 2024-04-26T04:58:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.302+0000 7f5351296700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f5351296700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c0fe080 0x7f534c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f5351296700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c10e4c0 con 0x7f534c0fce80 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f5351296700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c10ed10 con 0x7f534c0fe080 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34272/0 (socket says 172.21.15.77:34272) 2024-04-26T04:58:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 -- 172.21.15.77:0/3185455262 learned_addr learned my addr 172.21.15.77:0/3185455262 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 -- 172.21.15.77:0/3185455262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c0fe080 msgr2=0x7f534c10dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c0fe080 0x7f534c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.303+0000 7f534affd700 1 -- 172.21.15.77:0/3185455262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340005040 con 0x7f534c0fce80 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f534affd700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f533c00eb70 tx=0x7f533c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c007cc0 con 0x7f534c0fce80 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c10ef90 con 0x7f534c0fce80 2024-04-26T04:58:36.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f533c00f040 con 0x7f534c0fce80 2024-04-26T04:58:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c00cbd0 con 0x7f534c0fce80 2024-04-26T04:58:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.304+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c10f4d0 con 0x7f534c0fce80 2024-04-26T04:58:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.305+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c05f5c0 con 0x7f534c0fce80 2024-04-26T04:58:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.306+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f533c00cd30 con 0x7f534c0fce80 2024-04-26T04:58:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.306+0000 7f53337fe700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 0x7f53340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.306+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f533c08d5c0 con 0x7f534c0fce80 2024-04-26T04:58:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.306+0000 7f534a7fc700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 0x7f53340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.307+0000 7f534a7fc700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 0x7f53340705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5340009b30 tx=0x7f5340009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:36.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.311+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f533c05a570 con 0x7f534c0fce80 2024-04-26T04:58:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:36 smithi077 ceph-mon[91833]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:36 smithi077 ceph-mon[91833]: pgmap v117: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:36.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:36 smithi077 ceph-mon[91833]: osd.3 marked itself down and dead 2024-04-26T04:58:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-mon[79689]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-mon[79689]: pgmap v117: 65 pgs: 65 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-mon[79689]: osd.3 marked itself down and dead 2024-04-26T04:58:36.517 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[39314]: 2024-04-26T04:58:36.109+0000 7f28a23ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:58:36.517 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[39314]: 2024-04-26T04:58:36.109+0000 7f28a23ef700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-26T04:58:36.517 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:36 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[39314]: 2024-04-26T04:58:36.109+0000 7f28a23ef700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:58:36.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.528+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f534c10f850 con 0x7f533406e140 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.538+0000 7f53337fe700 1 -- 172.21.15.77:0/3185455262 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f534c10f850 con 0x7f533406e140 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (5m) 25s ago 13m 25.2M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (2m) 25s ago 14m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (2m) 2m ago 13m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:58:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 25s ago 13m - - 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (8m) 25s ago 8m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (8m) 25s ago 8m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (8m) 2m ago 8m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (8m) 2m ago 8m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (4m) 25s ago 15m 503M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (3m) 2m ago 13m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (3m) 25s ago 15m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (2m) 2m ago 13m 31.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (5m) 25s ago 14m 15.9M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (4m) 2m ago 13m 16.7M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (106s) 25s ago 12m 86.4M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (68s) 25s ago 11m 83.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T04:58:36.540 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (30s) 25s ago 11m 14.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T04:58:36.541 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (10m) 2m ago 10m 233M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2de4b6889df1 2024-04-26T04:58:36.541 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (10m) 2m ago 10m 210M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:58:36.541 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (10m) 2m ago 10m 193M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:58:36.541 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (3m) 25s ago 13m 59.0M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 msgr2=0x7f53340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 0x7f53340705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5340009b30 tx=0x7f5340009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 msgr2=0x7f534c10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f533c00eb70 tx=0x7f533c00b040 comp rx=0 tx=0).stop 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 shutdown_connections 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f533406e140 0x7f53340705f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534c0fce80 0x7f534c10d980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 --2- 172.21.15.77:0/3185455262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f534c0fe080 0x7f534c10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 >> 172.21.15.77:0/3185455262 conn(0x7f534c0f8590 msgr2=0x7f534c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 shutdown_connections 2024-04-26T04:58:36.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.541+0000 7f5351296700 1 -- 172.21.15.77:0/3185455262 wait complete. 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 -- 172.21.15.77:0/502250863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fcf10 msgr2=0x7fb85c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 --2- 172.21.15.77:0/502250863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fcf10 0x7fb85c0fd320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb84c004940 tx=0x7fb84c009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 -- 172.21.15.77:0/502250863 shutdown_connections 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 --2- 172.21.15.77:0/502250863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fe080 0x7fb85c0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 --2- 172.21.15.77:0/502250863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fcf10 0x7fb85c0fd320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 -- 172.21.15.77:0/502250863 >> 172.21.15.77:0/502250863 conn(0x7fb85c0f8560 msgr2=0x7fb85c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 -- 172.21.15.77:0/502250863 shutdown_connections 2024-04-26T04:58:36.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 -- 172.21.15.77:0/502250863 wait complete. 2024-04-26T04:58:36.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.643+0000 7fb862765700 1 Processor -- start 2024-04-26T04:58:36.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb862765700 1 -- start start 2024-04-26T04:58:36.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb862765700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb862765700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fe080 0x7fb85c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb862765700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c10e4d0 con 0x7fb85c0fcf10 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb862765700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c10ed20 con 0x7fb85c0fe080 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34290/0 (socket says 172.21.15.77:34290) 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 -- 172.21.15.77:0/2871141194 learned_addr learned my addr 172.21.15.77:0/2871141194 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85b7fe700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fe080 0x7fb85c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 -- 172.21.15.77:0/2871141194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fe080 msgr2=0x7fb85c10ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fe080 0x7fb85c10ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 -- 172.21.15.77:0/2871141194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850005040 con 0x7fb85c0fcf10 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb85bfff700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b0d0 tx=0x7fb84c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.644+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb84c00e030 con 0x7fb85c0fcf10 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.645+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb84c01c400 con 0x7fb85c0fcf10 2024-04-26T04:58:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.645+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c005040 con 0x7fb85c0fcf10 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.645+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb84c00b630 con 0x7fb85c0fcf10 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.645+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c10f350 con 0x7fb85c0fcf10 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.645+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c05f5c0 con 0x7fb85c0fcf10 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.646+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb84c01c570 con 0x7fb85c0fcf10 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.646+0000 7fb858ff9700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 0x7fb8440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:36.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.647+0000 7fb85b7fe700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 0x7fb8440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:36.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.647+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb84c08f3d0 con 0x7fb85c0fcf10 2024-04-26T04:58:36.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.647+0000 7fb85b7fe700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 0x7fb8440705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb850009fd0 tx=0x7fb85000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:36.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.651+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb84c05c350 con 0x7fb85c0fcf10 2024-04-26T04:58:36.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.934+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb85c10f630 con 0x7fb85c0fcf10 2024-04-26T04:58:36.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.935+0000 7fb858ff9700 1 -- 172.21.15.77:0/2871141194 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fb84c01c820 con 0x7fb85c0fcf10 2024-04-26T04:58:36.935 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:58:36.935 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 3 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-26T04:58:36.936 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 7 2024-04-26T04:58:36.937 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:58:36.937 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:58:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 msgr2=0x7fb8440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 0x7fb8440705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb850009fd0 tx=0x7fb85000b040 comp rx=0 tx=0).stop 2024-04-26T04:58:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 msgr2=0x7fb85c10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b0d0 tx=0x7fb84c009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:36.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 shutdown_connections 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb84406e140 0x7fb8440705f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb85c0fcf10 0x7fb85c10d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 --2- 172.21.15.77:0/2871141194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb85c0fe080 0x7fb85c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 >> 172.21.15.77:0/2871141194 conn(0x7fb85c0f8560 msgr2=0x7fb85c0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 shutdown_connections 2024-04-26T04:58:36.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:36.937+0000 7fb862765700 1 -- 172.21.15.77:0/2871141194 wait complete. 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 -- 172.21.15.77:0/1527982597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fed60 msgr2=0x7f81c00faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 --2- 172.21.15.77:0/1527982597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fed60 0x7f81c00faea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f81b4004940 tx=0x7f81b4009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 -- 172.21.15.77:0/1527982597 shutdown_connections 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 --2- 172.21.15.77:0/1527982597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fed60 0x7f81c00faea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 --2- 172.21.15.77:0/1527982597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fe410 0x7f81c00fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 -- 172.21.15.77:0/1527982597 >> 172.21.15.77:0/1527982597 conn(0x7f81c00f6bd0 msgr2=0x7f81c00f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.039+0000 7f81c55be700 1 -- 172.21.15.77:0/1527982597 shutdown_connections 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 -- 172.21.15.77:0/1527982597 wait complete. 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 Processor -- start 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 -- start start 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 0x7f81c010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01105b0 con 0x7f81c00fe410 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81c55be700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0110720 con 0x7f81c00fed60 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81beffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81beffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34300/0 (socket says 172.21.15.77:34300) 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.040+0000 7f81beffd700 1 -- 172.21.15.77:0/7254134 learned_addr learned my addr 172.21.15.77:0/7254134 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81be7fc700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 0x7f81c010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81be7fc700 1 -- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 msgr2=0x7f81c010fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81be7fc700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81be7fc700 1 -- 172.21.15.77:0/7254134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4005040 con 0x7f81c00fed60 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81beffd700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81be7fc700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 0x7f81c010ffb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81b4000c00 tx=0x7f81b4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b400e030 con 0x7f81c00fed60 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c0115120 con 0x7f81c00fed60 2024-04-26T04:58:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b4013de0 con 0x7f81c00fed60 2024-04-26T04:58:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b401ce50 con 0x7f81c00fed60 2024-04-26T04:58:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.041+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c0115660 con 0x7f81c00fed60 2024-04-26T04:58:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.042+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c0101b50 con 0x7f81c00fed60 2024-04-26T04:58:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.043+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f81b401c400 con 0x7f81c00fed60 2024-04-26T04:58:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.043+0000 7f81a77fe700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 0x7f81a8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.043+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f81b408f400 con 0x7f81c00fed60 2024-04-26T04:58:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.043+0000 7f81beffd700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 0x7f81a8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.043+0000 7f81beffd700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 0x7f81a8070590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b00041d0 tx=0x7f81b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.048+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f81b405c260 con 0x7f81c00fed60 2024-04-26T04:58:37.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:37 smithi077 ceph-mon[91833]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:37.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:37 smithi077 ceph-mon[91833]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:37.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:37 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2871141194' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:37 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:58:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:37 smithi077 ceph-mon[91833]: osdmap e65: 6 total, 5 up, 6 in 2024-04-26T04:58:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:37 smithi078 ceph-mon[79689]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:37 smithi078 ceph-mon[79689]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:37 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2871141194' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:37 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:58:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:37 smithi078 ceph-mon[79689]: osdmap e65: 6 total, 5 up, 6 in 2024-04-26T04:58:37.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.298+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f81c004c710 con 0x7f81c00fed60 2024-04-26T04:58:37.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.299+0000 7f81a77fe700 1 -- 172.21.15.77:0/7254134 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f81b405bdf0 con 0x7f81c00fed60 2024-04-26T04:58:37.299 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:58:37.299 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:58:37.300 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:58:37.301 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:37.302 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 msgr2=0x7f81a8070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 0x7f81a8070590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b00041d0 tx=0x7f81b000b040 comp rx=0 tx=0).stop 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 msgr2=0x7f81c010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 0x7f81c010ffb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81b4000c00 tx=0x7f81b4009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 shutdown_connections 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f81a806e0e0 0x7f81a8070590 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81c00fe410 0x7f81c010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 --2- 172.21.15.77:0/7254134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c00fed60 0x7f81c010ffb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 >> 172.21.15.77:0/7254134 conn(0x7f81c00f6bd0 msgr2=0x7f81c00f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 shutdown_connections 2024-04-26T04:58:37.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.302+0000 7f81c55be700 1 -- 172.21.15.77:0/7254134 wait complete. 2024-04-26T04:58:37.304 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.403+0000 7f0563893700 1 -- 172.21.15.77:0/1750447427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 msgr2=0x7f055c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.403+0000 7f0563893700 1 --2- 172.21.15.77:0/1750447427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c0fd270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0558004940 tx=0x7f0558009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- 172.21.15.77:0/1750447427 shutdown_connections 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 --2- 172.21.15.77:0/1750447427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 0x7f055c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 --2- 172.21.15.77:0/1750447427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c0fd270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- 172.21.15.77:0/1750447427 >> 172.21.15.77:0/1750447427 conn(0x7f055c0f8520 msgr2=0x7f055c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- 172.21.15.77:0/1750447427 shutdown_connections 2024-04-26T04:58:37.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- 172.21.15.77:0/1750447427 wait complete. 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 Processor -- start 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- start start 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 0x7f055c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c072200 con 0x7f055c0fce60 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.404+0000 7f0563893700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0707f0 con 0x7f055c0fdfd0 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34318/0 (socket says 172.21.15.77:34318) 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 -- 172.21.15.77:0/1042232648 learned_addr learned my addr 172.21.15.77:0/1042232648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0560e2e700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 0x7f055c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 -- 172.21.15.77:0/1042232648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 msgr2=0x7f055c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 0x7f055c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 -- 172.21.15.77:0/1042232648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558005040 con 0x7f055c0fce60 2024-04-26T04:58:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f056162f700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0558009150 tx=0x7f0558009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055800e040 con 0x7f055c0fce60 2024-04-26T04:58:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c070a40 con 0x7f055c0fce60 2024-04-26T04:58:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0558004290 con 0x7f055c0fce60 2024-04-26T04:58:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055801c630 con 0x7f055c0fce60 2024-04-26T04:58:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.405+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c071000 con 0x7f055c0fce60 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.406+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c04c710 con 0x7f055c0fce60 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.407+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f055800bbf0 con 0x7f055c0fce60 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.407+0000 7f0551ffb700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 0x7f05480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.407+0000 7f0560e2e700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 0x7f05480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.407+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f055808e6b0 con 0x7f055c0fce60 2024-04-26T04:58:37.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.408+0000 7f0560e2e700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 0x7f05480705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.412+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f055805b630 con 0x7f055c0fce60 2024-04-26T04:58:37.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.627+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f055c1022c0 con 0x7f054806e140 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.628+0000 7f0551ffb700 1 -- 172.21.15.77:0/1042232648 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f055c1022c0 con 0x7f054806e140 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:58:37.629 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "9/21 daemons upgraded", 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:58:37.630 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:58:37.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 msgr2=0x7f05480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 0x7f05480705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c00b040 comp rx=0 tx=0).stop 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 msgr2=0x7f055c071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0558009150 tx=0x7f0558009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 shutdown_connections 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f054806e140 0x7f05480705f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f055c0fce60 0x7f055c071c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 --2- 172.21.15.77:0/1042232648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c0fdfd0 0x7f055c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 >> 172.21.15.77:0/1042232648 conn(0x7f055c0f8520 msgr2=0x7f055c0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 shutdown_connections 2024-04-26T04:58:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.631+0000 7f0563893700 1 -- 172.21.15.77:0/1042232648 wait complete. 2024-04-26T04:58:37.718 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:37 smithi078 podman[88615]: 2024-04-26 04:58:37.422290972 +0000 UTC m=+1.435506652 container died 2de4b6889df153226e031695050b2fbdb466c20fc41bde78d03cd898cfc91f57 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T04:58:37.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.733+0000 7f6c5edb4700 1 -- 172.21.15.77:0/1242246133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fdfd0 msgr2=0x7f6c580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.733+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/1242246133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fdfd0 0x7f6c580fe440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c44004940 tx=0x7f6c44009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.733+0000 7f6c5edb4700 1 -- 172.21.15.77:0/1242246133 shutdown_connections 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.733+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/1242246133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fdfd0 0x7f6c580fe440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.733+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/1242246133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fce60 0x7f6c580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- 172.21.15.77:0/1242246133 >> 172.21.15.77:0/1242246133 conn(0x7f6c580f8540 msgr2=0x7f6c580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- 172.21.15.77:0/1242246133 shutdown_connections 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- 172.21.15.77:0/1242246133 wait complete. 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 Processor -- start 2024-04-26T04:58:37.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- start start 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fdfd0 0x7f6c58110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58110660 con 0x7f6c580fce60 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.734+0000 7f6c5edb4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581107d0 con 0x7f6c580fdfd0 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52680/0 (socket says 172.21.15.77:52680) 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 -- 172.21.15.77:0/2131430323 learned_addr learned my addr 172.21.15.77:0/2131430323 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c57fff700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fdfd0 0x7f6c58110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 -- 172.21.15.77:0/2131430323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fdfd0 msgr2=0x7f6c58110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fdfd0 0x7f6c58110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 -- 172.21.15.77:0/2131430323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44005040 con 0x7f6c580fce60 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5cb50700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c830 tx=0x7f6c4c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4c00a760 con 0x7f6c580fce60 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c4c00ad90 con 0x7f6c580fce60 2024-04-26T04:58:37.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c58115230 con 0x7f6c580fce60 2024-04-26T04:58:37.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4c0113a0 con 0x7f6c580fce60 2024-04-26T04:58:37.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.735+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581157f0 con 0x7f6c580fce60 2024-04-26T04:58:37.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.736+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c58101960 con 0x7f6c580fce60 2024-04-26T04:58:37.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.737+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c4c012400 con 0x7f6c580fce60 2024-04-26T04:58:37.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.737+0000 7f6c557fa700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 0x7f6c480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:58:37.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.737+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6c4c08da20 con 0x7f6c580fce60 2024-04-26T04:58:37.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.737+0000 7f6c57fff700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 0x7f6c480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:58:37.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.737+0000 7f6c57fff700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 0x7f6c480705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:58:37.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:37.742+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6c4c05aa40 con 0x7f6c580fce60 2024-04-26T04:58:38.011 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:37 smithi078 podman[88615]: 2024-04-26 04:58:37.817984208 +0000 UTC m=+1.831199895 container cleanup 2de4b6889df153226e031695050b2fbdb466c20fc41bde78d03cd898cfc91f57 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T04:58:38.011 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:37 smithi078 bash[88615]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3 2024-04-26T04:58:38.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.023+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6c5804c710 con 0x7f6c580fce60 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.023+0000 7f6c557fa700 1 -- 172.21.15.77:0/2131430323 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+436 (secure 0 0 0) 0x7f6c4c05a5d0 con 0x7f6c580fce60 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; 1 osds down 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:58:38.024 INFO:teuthology.orchestra.run.smithi077.stdout: osd.3 (root=default,host=smithi078) is down 2024-04-26T04:58:38.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 msgr2=0x7f6c480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:38.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 0x7f6c480705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44009e30 comp rx=0 tx=0).stop 2024-04-26T04:58:38.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 msgr2=0x7f6c5810fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:58:38.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c830 tx=0x7f6c4c004470 comp rx=0 tx=0).stop 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 shutdown_connections 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6c4806e140 0x7f6c480705f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c580fce60 0x7f6c5810fb20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 --2- 172.21.15.77:0/2131430323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c580fdfd0 0x7f6c58110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 >> 172.21.15.77:0/2131430323 conn(0x7f6c580f8540 msgr2=0x7f6c580f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 shutdown_connections 2024-04-26T04:58:38.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:58:38.025+0000 7f6c5edb4700 1 -- 172.21.15.77:0/2131430323 wait complete. 2024-04-26T04:58:38.267 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88643]: 2024-04-26 04:58:38.01083745 +0000 UTC m=+0.587777903 container remove 2de4b6889df153226e031695050b2fbdb466c20fc41bde78d03cd898cfc91f57 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T04:58:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:38 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/7254134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:58:38.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:38 smithi078 ceph-mon[79689]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:38.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:38 smithi078 ceph-mon[79689]: pgmap v119: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:38.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:38 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2131430323' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:58:38.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:38 smithi078 ceph-mon[79689]: osdmap e66: 6 total, 5 up, 6 in 2024-04-26T04:58:38.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:38 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/7254134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:58:38.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:38 smithi077 ceph-mon[91833]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:58:38.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:38 smithi077 ceph-mon[91833]: pgmap v119: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:38.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:38 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2131430323' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:58:38.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:38 smithi077 ceph-mon[91833]: osdmap e66: 6 total, 5 up, 6 in 2024-04-26T04:58:38.673 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.239879512 +0000 UTC m=+0.018247782 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:38.673 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.345231598 +0000 UTC m=+0.123599843 container create 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T04:58:38.673 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.555740331 +0000 UTC m=+0.334108576 container init 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T04:58:38.673 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.55881264 +0000 UTC m=+0.337180894 container start 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T04:58:38.673 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.67246472 +0000 UTC m=+0.450832965 container attach 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T04:58:39.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:38 smithi078 podman[88732]: 2024-04-26 04:58:38.853466849 +0000 UTC m=+0.631835110 container died 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T04:58:39.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 podman[88758]: 2024-04-26 04:58:39.484536501 +0000 UTC m=+0.625874114 container remove 7f90904b48c47222ec7d324ead77e82f44c8c8835ca0d6be7af813f586ac904d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:58:39.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-26T04:58:39.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 systemd[1]: Stopped Ceph osd.3 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:58:39.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.3.service: Consumed 59.220s CPU time. 2024-04-26T04:58:39.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 systemd[1]: Starting Ceph osd.3 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:58:40.229 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-mon[79689]: pgmap v121: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:40.230 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 podman[88861]: 2024-04-26 04:58:39.832185429 +0000 UTC m=+0.018187674 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:40.230 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:39 smithi078 podman[88861]: 2024-04-26 04:58:39.935743632 +0000 UTC m=+0.121745922 container create 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T04:58:40.230 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 podman[88861]: 2024-04-26 04:58:40.154490743 +0000 UTC m=+0.340492988 container init 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb) 2024-04-26T04:58:40.230 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 podman[88861]: 2024-04-26 04:58:40.157825242 +0000 UTC m=+0.343827495 container start 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_CLEAN=True) 2024-04-26T04:58:40.230 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 podman[88861]: 2024-04-26 04:58:40.229536424 +0000 UTC m=+0.415538673 container attach 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T04:58:40.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:40 smithi077 ceph-mon[91833]: pgmap v121: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:40.908 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:58:40.908 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:58:40.908 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 bash[88861]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate[88876]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-26T04:58:40.909 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:40 smithi078 podman[88861]: 2024-04-26 04:58:40.625894821 +0000 UTC m=+0.811897060 container died 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T04:58:41.614 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 podman[89011]: 2024-04-26 04:58:41.274853593 +0000 UTC m=+0.643402956 container remove 59702cc89f89ace0f74003af1ae3bab95da6b32c7cba651b4ed49371c81ea1d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:58:41.615 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 podman[89075]: 2024-04-26 04:58:41.391805553 +0000 UTC m=+0.017848534 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:58:41.615 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 podman[89075]: 2024-04-26 04:58:41.509057608 +0000 UTC m=+0.135100573 container create a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb) 2024-04-26T04:58:42.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 podman[89075]: 2024-04-26 04:58:41.703263852 +0000 UTC m=+0.329306833 container init a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T04:58:42.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 podman[89075]: 2024-04-26 04:58:41.706277152 +0000 UTC m=+0.332320117 container start a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-b22e2eb, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T04:58:42.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 ceph-osd[89095]: -- 172.21.15.78:0/2641241364 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b6a207680 con 0x556b6a1d0c00 2024-04-26T04:58:42.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 bash[89075]: a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 2024-04-26T04:58:42.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:41 smithi078 systemd[1]: Started Ceph osd.3 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:58:42.517 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:42 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T04:58:42.252+0000 7fc2a9b177c0 -1 Falling back to public interface 2024-04-26T04:58:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:42 smithi078 ceph-mon[79689]: pgmap v122: 65 pgs: 15 active+undersized+degraded, 3 stale+active+clean, 16 active+undersized, 31 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:58:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:42 smithi077 ceph-mon[91833]: pgmap v122: 65 pgs: 15 active+undersized+degraded, 3 stale+active+clean, 16 active+undersized, 31 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-26T04:58:44.103 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:43 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 15 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:43 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 15 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:44 smithi077 ceph-mon[91833]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:44 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:44 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:44 smithi078 ceph-mon[79689]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:44 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:44 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.059 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.060 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.060 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.060 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:47.092 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.093 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.093 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:47.093 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:58:47.495 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:47 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T04:58:47.144+0000 7fc2a9b177c0 -1 osd.3 64 log_to_monitors true 2024-04-26T04:58:48.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:47 smithi078 ceph-mon[79689]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:48.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:47 smithi078 ceph-mon[79689]: from='osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:58:48.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:47 smithi078 ceph-mon[79689]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:58:48.109 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:47 smithi077 ceph-mon[91833]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:48.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:47 smithi077 ceph-mon[91833]: from='osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:58:48.110 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:47 smithi077 ceph-mon[91833]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:58:49.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T04:58:48.837+0000 7fc296f67700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:58:49.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-mon[79689]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:58:49.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-mon[79689]: from='osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:58:49.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-mon[79689]: osdmap e67: 6 total, 5 up, 6 in 2024-04-26T04:58:49.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-mon[79689]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:58:49.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:48 smithi078 ceph-mon[79689]: from='osd.3 ' entity='osd.3' 2024-04-26T04:58:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:48 smithi077 ceph-mon[91833]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:58:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:48 smithi077 ceph-mon[91833]: from='osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:58:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:48 smithi077 ceph-mon[91833]: osdmap e67: 6 total, 5 up, 6 in 2024-04-26T04:58:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:48 smithi077 ceph-mon[91833]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:58:49.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:48 smithi077 ceph-mon[91833]: from='osd.3 ' entity='osd.3' 2024-04-26T04:58:50.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:58:50.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677] boot 2024-04-26T04:58:50.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: osdmap e68: 6 total, 6 up, 6 in 2024-04-26T04:58:50.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:58:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: osd.3 [v2:172.21.15.78:6800/2853701677,v1:172.21.15.78:6801/2853701677] boot 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: osdmap e68: 6 total, 6 up, 6 in 2024-04-26T04:58:50.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:58:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:50 smithi077 ceph-mon[91833]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:50 smithi077 ceph-mon[91833]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:50 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:58:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:50 smithi077 ceph-mon[91833]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-26T04:58:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:50 smithi077 ceph-mon[91833]: osdmap e69: 6 total, 6 up, 6 in 2024-04-26T04:58:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:50 smithi078 ceph-mon[79689]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:50 smithi078 ceph-mon[79689]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-04-26T04:58:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:50 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:58:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:50 smithi078 ceph-mon[79689]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-26T04:58:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:50 smithi078 ceph-mon[79689]: osdmap e69: 6 total, 6 up, 6 in 2024-04-26T04:58:52.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:52 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:52.336 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:52 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:58:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:53 smithi078 ceph-mon[79689]: pgmap v131: 65 pgs: 7 peering, 14 active+undersized+degraded, 17 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-26T04:58:53.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:53 smithi077 ceph-mon[91833]: pgmap v131: 65 pgs: 7 peering, 14 active+undersized+degraded, 17 active+undersized, 27 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-26T04:58:54.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:54 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded) 2024-04-26T04:58:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:54 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded) 2024-04-26T04:58:55.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:55 smithi077 ceph-mon[91833]: pgmap v132: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:55 smithi078 ceph-mon[79689]: pgmap v132: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:57.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:57 smithi077 ceph-mon[91833]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:57.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:57 smithi078 ceph-mon[79689]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:58:59 smithi077 ceph-mon[91833]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:58:59.499 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:58:59 smithi078 ceph-mon[79689]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:01.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:01 smithi077 ceph-mon[91833]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:01.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:01.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:01 smithi078 ceph-mon[79689]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:02.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:02 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:02.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:02 smithi077 ceph-mon[91833]: pgmap v136: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:02 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:02.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:02 smithi078 ceph-mon[79689]: pgmap v136: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:04.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:04 smithi077 ceph-mon[91833]: pgmap v137: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:04.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:04 smithi078 ceph-mon[79689]: pgmap v137: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:59:05.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:05 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:59:05.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:05 smithi078 ceph-mon[79689]: Upgrade: osd.4 is safe to restart 2024-04-26T04:59:05.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:59:05.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:05 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:59:05.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:05 smithi077 ceph-mon[91833]: Upgrade: osd.4 is safe to restart 2024-04-26T04:59:06.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:06 smithi078 ceph-mon[79689]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:06.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:06 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:06.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:06 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:59:06.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:06 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:06 smithi077 ceph-mon[91833]: pgmap v138: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:06 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:06 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:59:06.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:06 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:07.631 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:07 smithi078 ceph-mon[79689]: Upgrade: Updating osd.4 2024-04-26T04:59:07.632 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:07 smithi078 ceph-mon[79689]: Deploying daemon osd.4 on smithi078 2024-04-26T04:59:07.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:07 smithi077 ceph-mon[91833]: Upgrade: Updating osd.4 2024-04-26T04:59:07.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:07 smithi077 ceph-mon[91833]: Deploying daemon osd.4 on smithi078 2024-04-26T04:59:08.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- 172.21.15.77:0/1547053686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fdfe0 msgr2=0x7fb9f40fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 --2- 172.21.15.77:0/1547053686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f40fe450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4004940 tx=0x7fb9e4009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- 172.21.15.77:0/1547053686 shutdown_connections 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 --2- 172.21.15.77:0/1547053686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f40fe450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 --2- 172.21.15.77:0/1547053686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fce70 0x7fb9f40fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- 172.21.15.77:0/1547053686 >> 172.21.15.77:0/1547053686 conn(0x7fb9f40f8550 msgr2=0x7fb9f40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- 172.21.15.77:0/1547053686 shutdown_connections 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- 172.21.15.77:0/1547053686 wait complete. 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 Processor -- start 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.140+0000 7fb9fb288700 1 -- start start 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9fb288700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9fb288700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9fb288700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f410e400 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9fb288700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f410e570 con 0x7fb9f40fdfe0 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39358/0 (socket says 172.21.15.77:39358) 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 -- 172.21.15.77:0/134423603 learned_addr learned my addr 172.21.15.77:0/134423603 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f8823700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 -- 172.21.15.77:0/134423603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fdfe0 msgr2=0x7fb9f410de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f410de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 -- 172.21.15.77:0/134423603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4005040 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9f9024700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000eb40 tx=0x7fb9f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f0007cc0 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9f000f040 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f4112f70 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.141+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f000ca10 con 0x7fb9f40fce70 2024-04-26T04:59:08.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.142+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f4113420 con 0x7fb9f40fce70 2024-04-26T04:59:08.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.143+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f404c710 con 0x7fb9f40fce70 2024-04-26T04:59:08.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.143+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb9f000cb70 con 0x7fb9f40fce70 2024-04-26T04:59:08.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.144+0000 7fb9e9ffb700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 0x7fb9e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.144+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb9f008d5c0 con 0x7fb9f40fce70 2024-04-26T04:59:08.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.144+0000 7fb9f8823700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 0x7fb9e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.144+0000 7fb9f8823700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 0x7fb9e00705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009b30 tx=0x7fb9e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.149+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb9f005a5f0 con 0x7fb9f40fce70 2024-04-26T04:59:08.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.364+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9f4113660 con 0x7fb9e006e140 2024-04-26T04:59:08.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.365+0000 7fb9e9ffb700 1 -- 172.21.15.77:0/134423603 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb9f4113660 con 0x7fb9e006e140 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 msgr2=0x7fb9e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 0x7fb9e00705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009b30 tx=0x7fb9e4009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 msgr2=0x7fb9f410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000eb40 tx=0x7fb9f000b040 comp rx=0 tx=0).stop 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 shutdown_connections 2024-04-26T04:59:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fb9e006e140 0x7fb9e00705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9f40fce70 0x7fb9f410d8c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 --2- 172.21.15.77:0/134423603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9f40fdfe0 0x7fb9f410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 >> 172.21.15.77:0/134423603 conn(0x7fb9f40f8550 msgr2=0x7fb9f40f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 shutdown_connections 2024-04-26T04:59:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.367+0000 7fb9fb288700 1 -- 172.21.15.77:0/134423603 wait complete. 2024-04-26T04:59:08.379 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:59:08.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 -- 172.21.15.77:0/2612456855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fce60 msgr2=0x7f60e80fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 --2- 172.21.15.77:0/2612456855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fce60 0x7f60e80fd270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f60d8004940 tx=0x7f60d8009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:08.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 -- 172.21.15.77:0/2612456855 shutdown_connections 2024-04-26T04:59:08.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 --2- 172.21.15.77:0/2612456855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fdfd0 0x7f60e80fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 --2- 172.21.15.77:0/2612456855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fce60 0x7f60e80fd270 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 -- 172.21.15.77:0/2612456855 >> 172.21.15.77:0/2612456855 conn(0x7f60e80f8540 msgr2=0x7f60e80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 -- 172.21.15.77:0/2612456855 shutdown_connections 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 -- 172.21.15.77:0/2612456855 wait complete. 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.470+0000 7f60ee930700 1 Processor -- start 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60ee930700 1 -- start start 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60ee930700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fce60 0x7f60e810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60ee930700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60ee930700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e810e480 con 0x7f60e80fdfd0 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60ee930700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e810e5f0 con 0x7f60e80fce60 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39378/0 (socket says 172.21.15.77:39378) 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 -- 172.21.15.77:0/663182402 learned_addr learned my addr 172.21.15.77:0/663182402 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:08.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e7fff700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fce60 0x7f60e810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 -- 172.21.15.77:0/663182402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fce60 msgr2=0x7f60e810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fce60 0x7f60e810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 -- 172.21.15.77:0/663182402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8005040 con 0x7f60e80fdfd0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e77fe700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60dc002a50 tx=0x7f60dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.471+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc007cc0 con 0x7f60e80fdfd0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.472+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60dc00f040 con 0x7f60e80fdfd0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.472+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8113050 con 0x7f60e80fdfd0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.472+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60dc00da30 con 0x7f60e80fdfd0 2024-04-26T04:59:08.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.472+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e81135e0 con 0x7f60e80fdfd0 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.473+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e804c710 con 0x7f60e80fdfd0 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.473+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f60dc00db90 con 0x7f60e80fdfd0 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.473+0000 7f60e4ff9700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 0x7f60d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.474+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f60dc08d7a0 con 0x7f60e80fdfd0 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.474+0000 7f60e7fff700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 0x7f60d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.474+0000 7f60e7fff700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 0x7f60d00705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b970 tx=0x7f60d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.478+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f60dc05a720 con 0x7f60e80fdfd0 2024-04-26T04:59:08.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.695+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60e8102330 con 0x7f60d006e140 2024-04-26T04:59:08.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.697+0000 7f60e4ff9700 1 -- 172.21.15.77:0/663182402 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f60e8102330 con 0x7f60d006e140 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 msgr2=0x7f60d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 0x7f60d00705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b970 tx=0x7f60d8009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 msgr2=0x7f60e810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60dc002a50 tx=0x7f60dc0046b0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 shutdown_connections 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f60d006e140 0x7f60d00705f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60e80fce60 0x7f60e810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 --2- 172.21.15.77:0/663182402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f60e80fdfd0 0x7f60e810de80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 >> 172.21.15.77:0/663182402 conn(0x7f60e80f8540 msgr2=0x7f60e80f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 shutdown_connections 2024-04-26T04:59:08.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.699+0000 7f60ee930700 1 -- 172.21.15.77:0/663182402 wait complete. 2024-04-26T04:59:08.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:08 smithi077 ceph-mon[91833]: pgmap v139: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:08.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- 172.21.15.77:0/1441293858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 msgr2=0x7fe840102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 --2- 172.21.15.77:0/1441293858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe840102310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe828004940 tx=0x7fe828009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- 172.21.15.77:0/1441293858 shutdown_connections 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 --2- 172.21.15.77:0/1441293858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe840102310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 --2- 172.21.15.77:0/1441293858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe8400ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- 172.21.15.77:0/1441293858 >> 172.21.15.77:0/1441293858 conn(0x7fe8400f7440 msgr2=0x7fe8400f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- 172.21.15.77:0/1441293858 shutdown_connections 2024-04-26T04:59:08.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- 172.21.15.77:0/1441293858 wait complete. 2024-04-26T04:59:08.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 Processor -- start 2024-04-26T04:59:08.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 -- start start 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe84010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.801+0000 7fe844b6f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe844b6f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84010e470 con 0x7fe8400ffef0 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe844b6f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84010e5e0 con 0x7fe840066f50 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe835d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe835d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39390/0 (socket says 172.21.15.77:39390) 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe835d9b700 1 -- 172.21.15.77:0/849886511 learned_addr learned my addr 172.21.15.77:0/849886511 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe83e59c700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe84010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe83e59c700 1 -- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 msgr2=0x7fe84010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe83e59c700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe83e59c700 1 -- 172.21.15.77:0/849886511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828005040 con 0x7fe840066f50 2024-04-26T04:59:08.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe835d9b700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe83e59c700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe84010d930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe840066c90 tx=0x7fe83000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830004e90 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe83000f040 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe840112fe0 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83000ca20 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.802+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe840113490 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.803+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84004c710 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.804+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe830007cc0 con 0x7fe840066f50 2024-04-26T04:59:08.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.804+0000 7fe8377fe700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 0x7fe82c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:08.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.804+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe83008d530 con 0x7fe840066f50 2024-04-26T04:59:08.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.804+0000 7fe835d9b700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 0x7fe82c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:08.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.805+0000 7fe835d9b700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 0x7fe82c0705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe828004890 tx=0x7fe828009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:08.809+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe83005a440 con 0x7fe840066f50 2024-04-26T04:59:08.897 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:08 smithi078 ceph-mon[79689]: pgmap v139: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:09.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.014+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8401136b0 con 0x7fe82c06e140 2024-04-26T04:59:09.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.024+0000 7fe8377fe700 1 -- 172.21.15.77:0/849886511 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe8401136b0 con 0x7fe82c06e140 2024-04-26T04:59:09.026 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:59:09.026 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (5m) 58s ago 14m 25.2M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:59:09.026 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (2m) 58s ago 14m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (2m) 24s ago 13m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 58s ago 14m - - 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (9m) 58s ago 9m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (9m) 58s ago 9m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (9m) 24s ago 9m 17.1M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (9m) 24s ago 9m 17.1M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (4m) 58s ago 15m 503M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (4m) 24s ago 13m 451M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (3m) 58s ago 16m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (3m) 24s ago 13m 36.1M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (6m) 58s ago 14m 15.9M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (5m) 24s ago 13m 16.4M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:59:09.027 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (2m) 58s ago 12m 86.4M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (100s) 58s ago 12m 83.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (63s) 58s ago 11m 14.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (27s) 24s ago 11m 15.0M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (11m) 24s ago 11m 213M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55f6b2230962 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (10m) 24s ago 10m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (4m) 58s ago 14m 59.0M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:59:09.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 msgr2=0x7fe82c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 0x7fe82c0705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe828004890 tx=0x7fe828009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 msgr2=0x7fe84010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe84010d930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe840066c90 tx=0x7fe83000b040 comp rx=0 tx=0).stop 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 shutdown_connections 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.026+0000 7fe844b6f700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe82c06e140 0x7fe82c0705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.027+0000 7fe844b6f700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe840066f50 0x7fe84010d930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.027+0000 7fe844b6f700 1 --2- 172.21.15.77:0/849886511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8400ffef0 0x7fe84010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.027+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 >> 172.21.15.77:0/849886511 conn(0x7fe8400f7440 msgr2=0x7fe8400f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.027+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 shutdown_connections 2024-04-26T04:59:09.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.027+0000 7fe844b6f700 1 -- 172.21.15.77:0/849886511 wait complete. 2024-04-26T04:59:09.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.126+0000 7f6520c4b700 1 -- 172.21.15.77:0/777047702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fcf60 msgr2=0x7f651c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.126+0000 7f6520c4b700 1 --2- 172.21.15.77:0/777047702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fcf60 0x7f651c0fd3d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6510004940 tx=0x7f6510009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- 172.21.15.77:0/777047702 shutdown_connections 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 --2- 172.21.15.77:0/777047702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fcf60 0x7f651c0fd3d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 --2- 172.21.15.77:0/777047702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fbcb0 0x7f651c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- 172.21.15.77:0/777047702 >> 172.21.15.77:0/777047702 conn(0x7f651c0f7430 msgr2=0x7f651c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- 172.21.15.77:0/777047702 shutdown_connections 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- 172.21.15.77:0/777047702 wait complete. 2024-04-26T04:59:09.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 Processor -- start 2024-04-26T04:59:09.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- start start 2024-04-26T04:59:09.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fcf60 0x7f651c1101b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c110840 con 0x7f651c0fbcb0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f6520c4b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c110fe0 con 0x7f651c0fcf60 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f651a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f651a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39404/0 (socket says 172.21.15.77:39404) 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.127+0000 7f651a59c700 1 -- 172.21.15.77:0/3644254487 learned_addr learned my addr 172.21.15.77:0/3644254487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f6519d9b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fcf60 0x7f651c1101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f651a59c700 1 -- 172.21.15.77:0/3644254487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fcf60 msgr2=0x7f651c1101b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f651a59c700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fcf60 0x7f651c1101b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f651a59c700 1 -- 172.21.15.77:0/3644254487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510005040 con 0x7f651c0fbcb0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f651a59c700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f650c00b790 tx=0x7f650c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c00f6f0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650c00fd20 con 0x7f651c0fbcb0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c0103a0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c1112c0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.128+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c111880 con 0x7f651c0fbcb0 2024-04-26T04:59:09.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.129+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c1018f0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.130+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f650c011400 con 0x7f651c0fbcb0 2024-04-26T04:59:09.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.130+0000 7f650affd700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 0x7f65040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.130+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f650c08d800 con 0x7f651c0fbcb0 2024-04-26T04:59:09.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.130+0000 7f6519d9b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 0x7f65040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.131+0000 7f6519d9b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 0x7f65040705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f651000b970 tx=0x7f6510009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.135+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f650c05a820 con 0x7f651c0fbcb0 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.415+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f651c1048c0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.415+0000 7f650affd700 1 -- 172.21.15.77:0/3644254487 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f650c05a3b0 con 0x7f651c0fbcb0 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:59:09.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6, 2024-04-26T04:59:09.418 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 8 2024-04-26T04:59:09.419 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:59:09.419 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:59:09.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 msgr2=0x7f65040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 0x7f65040705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f651000b970 tx=0x7f6510009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 msgr2=0x7f651c10fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f650c00b790 tx=0x7f650c009250 comp rx=0 tx=0).stop 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 shutdown_connections 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f650406e140 0x7f65040705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f651c0fbcb0 0x7f651c10fc70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 --2- 172.21.15.77:0/3644254487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f651c0fcf60 0x7f651c1101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 >> 172.21.15.77:0/3644254487 conn(0x7f651c0f7430 msgr2=0x7f651c065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 shutdown_connections 2024-04-26T04:59:09.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.418+0000 7f6520c4b700 1 -- 172.21.15.77:0/3644254487 wait complete. 2024-04-26T04:59:09.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- 172.21.15.77:0/665122622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fe090 msgr2=0x7f3af80fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 --2- 172.21.15.77:0/665122622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fe090 0x7f3af80fe500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3aec004940 tx=0x7f3aec009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- 172.21.15.77:0/665122622 shutdown_connections 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 --2- 172.21.15.77:0/665122622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fe090 0x7f3af80fe500 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 --2- 172.21.15.77:0/665122622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fce90 0x7f3af80fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- 172.21.15.77:0/665122622 >> 172.21.15.77:0/665122622 conn(0x7f3af80f85c0 msgr2=0x7f3af80fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- 172.21.15.77:0/665122622 shutdown_connections 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- 172.21.15.77:0/665122622 wait complete. 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 Processor -- start 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.519+0000 7f3afdb46700 1 -- start start 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 0x7f3af810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af810e580 con 0x7f3af80fce90 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af810edd0 con 0x7f3af80fe090 2024-04-26T04:59:09.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39422/0 (socket says 172.21.15.77:39422) 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 -- 172.21.15.77:0/3354118991 learned_addr learned my addr 172.21.15.77:0/3354118991 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af6ffd700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 0x7f3af810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 -- 172.21.15.77:0/3354118991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 msgr2=0x7f3af810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 0x7f3af810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 -- 172.21.15.77:0/3354118991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec005040 con 0x7f3af80fce90 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af6ffd700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 0x7f3af810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3af77fe700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8002a80 tx=0x7f3ae8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae8007cc0 con 0x7f3af80fce90 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af810f0b0 con 0x7f3af80fce90 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ae800f040 con 0x7f3af80fce90 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae800dad0 con 0x7f3af80fce90 2024-04-26T04:59:09.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.520+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af810f5f0 con 0x7f3af80fce90 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.521+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af805f5c0 con 0x7f3af80fce90 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.522+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3ae800dc30 con 0x7f3af80fce90 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.522+0000 7f3adffff700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.522+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3ae808d880 con 0x7f3af80fce90 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.522+0000 7f3af6ffd700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.523+0000 7f3af6ffd700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 0x7f3ae00705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b3b0 tx=0x7f3aec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.527+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3ae805a780 con 0x7f3af80fce90 2024-04-26T04:59:09.607 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:09 smithi078 systemd[1]: Stopping Ceph osd.4 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:59:09.607 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[45748]: 2024-04-26T04:59:09.526+0000 7f9961d35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:59:09.607 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[45748]: 2024-04-26T04:59:09.526+0000 7f9961d35700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-26T04:59:09.607 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[45748]: 2024-04-26T04:59:09.526+0000 7f9961d35700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:59:09.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:09 smithi077 ceph-mon[91833]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:09 smithi077 ceph-mon[91833]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:09 smithi077 ceph-mon[91833]: from='client.44231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:09 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3644254487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:09.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:09 smithi077 ceph-mon[91833]: osd.4 marked itself down and dead 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.783+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3af804bed0 con 0x7f3af80fce90 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.783+0000 7f3adffff700 1 -- 172.21.15.77:0/3354118991 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f3ae805a310 con 0x7f3af80fce90 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:59:09.785 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:59:09.786 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:59:09.787 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.785+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 msgr2=0x7f3ae00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.785+0000 7f3afdb46700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 0x7f3ae00705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b3b0 tx=0x7f3aec009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.785+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 msgr2=0x7f3af810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.785+0000 7f3afdb46700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8002a80 tx=0x7f3ae8004600 comp rx=0 tx=0).stop 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 shutdown_connections 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3af80fce90 0x7f3af810da40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 --2- 172.21.15.77:0/3354118991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af80fe090 0x7f3af810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 >> 172.21.15.77:0/3354118991 conn(0x7f3af80f85c0 msgr2=0x7f3af80f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 shutdown_connections 2024-04-26T04:59:09.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.786+0000 7f3afdb46700 1 -- 172.21.15.77:0/3354118991 wait complete. 2024-04-26T04:59:09.790 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:59:09.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2523981636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 msgr2=0x7f5d300fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2523981636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d300fc0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d18004940 tx=0x7f5d18009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2523981636 shutdown_connections 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2523981636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 0x7f5d300fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2523981636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d300fc0d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2523981636 >> 172.21.15.77:0/2523981636 conn(0x7f5d300f7440 msgr2=0x7f5d300f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2523981636 shutdown_connections 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.887+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2523981636 wait complete. 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 Processor -- start 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 -- start start 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 0x7f5d3010de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010e440 con 0x7f5d300fbcc0 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d35f4e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010e5b0 con 0x7f5d300fcf70 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39438/0 (socket says 172.21.15.77:39438) 2024-04-26T04:59:09.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 -- 172.21.15.77:0/2136509623 learned_addr learned my addr 172.21.15.77:0/2136509623 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2effd700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 0x7f5d3010de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 -- 172.21.15.77:0/2136509623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 msgr2=0x7f5d3010de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 0x7f5d3010de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 -- 172.21.15.77:0/2136509623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18005040 con 0x7f5d300fbcc0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.888+0000 7f5d2f7fe700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d18013870 tx=0x7f5d18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.889+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d18021070 con 0x7f5d300fbcc0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.889+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d1801c400 con 0x7f5d300fbcc0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.889+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d30112fb0 con 0x7f5d300fbcc0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.889+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1800e070 con 0x7f5d300fbcc0 2024-04-26T04:59:09.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.889+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d30113410 con 0x7f5d300fbcc0 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.890+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30101b00 con 0x7f5d300fbcc0 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.890+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d1801ca70 con 0x7f5d300fbcc0 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.890+0000 7f5d27fff700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 0x7f5d1c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.891+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5d1808ec40 con 0x7f5d300fbcc0 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.891+0000 7f5d2effd700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 0x7f5d1c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.891+0000 7f5d2effd700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 0x7f5d1c0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009b80 tx=0x7f5d2000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:09.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:09.895+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d1805bbc0 con 0x7f5d300fbcc0 2024-04-26T04:59:09.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-mon[79689]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-mon[79689]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-mon[79689]: from='client.44231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:09.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3644254487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:09.977 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:09 smithi078 ceph-mon[79689]: osd.4 marked itself down and dead 2024-04-26T04:59:10.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.114+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d3002b460 con 0x7f5d1c06e140 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.115+0000 7f5d27fff700 1 -- 172.21.15.77:0/2136509623 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5d3002b460 con 0x7f5d1c06e140 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:59:10.117 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:59:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:59:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "10/21 daemons upgraded", 2024-04-26T04:59:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:59:10.118 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:59:10.118 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:59:10.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.118+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 msgr2=0x7f5d1c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.118+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 0x7f5d1c0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009b80 tx=0x7f5d2000b040 comp rx=0 tx=0).stop 2024-04-26T04:59:10.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.118+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 msgr2=0x7f5d3010d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.118+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d18013870 tx=0x7f5d18009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:10.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.118+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 shutdown_connections 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5d1c06e140 0x7f5d1c0705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 --2- 172.21.15.77:0/2136509623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d300fcf70 0x7f5d3010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 >> 172.21.15.77:0/2136509623 conn(0x7f5d300f7440 msgr2=0x7f5d300f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 shutdown_connections 2024-04-26T04:59:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.119+0000 7f5d35f4e700 1 -- 172.21.15.77:0/2136509623 wait complete. 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 -- 172.21.15.77:0/1040262090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9100066f50 msgr2=0x7f91000ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1040262090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9100066f50 0x7f91000ff9b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f90f0004940 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 -- 172.21.15.77:0/1040262090 shutdown_connections 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1040262090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91000ffef0 0x7f9100102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1040262090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9100066f50 0x7f91000ff9b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 -- 172.21.15.77:0/1040262090 >> 172.21.15.77:0/1040262090 conn(0x7f91000f7440 msgr2=0x7f91000f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 -- 172.21.15.77:0/1040262090 shutdown_connections 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 -- 172.21.15.77:0/1040262090 wait complete. 2024-04-26T04:59:10.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.221+0000 7f9105c4a700 1 Processor -- start 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f9105c4a700 1 -- start start 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f9105c4a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9100066f50 0x7f91000fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f9105c4a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f9105c4a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000fd800 con 0x7f91000ffef0 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f9105c4a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000fbdf0 con 0x7f9100066f50 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39452/0 (socket says 172.21.15.77:39452) 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 -- 172.21.15.77:0/1988834717 learned_addr learned my addr 172.21.15.77:0/1988834717 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90ff7fe700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9100066f50 0x7f91000fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:10.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 -- 172.21.15.77:0/1988834717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9100066f50 msgr2=0x7f91000fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9100066f50 0x7f91000fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 -- 172.21.15.77:0/1988834717 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0005040 con 0x7f91000ffef0 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90feffd700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90f400eb40 tx=0x7f90f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.222+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f4007cc0 con 0x7f91000ffef0 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.223+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91000fc040 con 0x7f91000ffef0 2024-04-26T04:59:10.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.223+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f400f040 con 0x7f91000ffef0 2024-04-26T04:59:10.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.223+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f400ca10 con 0x7f91000ffef0 2024-04-26T04:59:10.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.223+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91000fc580 con 0x7f91000ffef0 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.224+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910004c710 con 0x7f91000ffef0 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.224+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f90f400cc80 con 0x7f91000ffef0 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.224+0000 7f90e7fff700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 0x7f90e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.225+0000 7f90ff7fe700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 0x7f90e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.225+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f90f408d760 con 0x7f91000ffef0 2024-04-26T04:59:10.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.225+0000 7f90ff7fe700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 0x7f90e80705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009b30 tx=0x7f90f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:10.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.229+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f90f405a670 con 0x7f91000ffef0 2024-04-26T04:59:10.243 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:09 smithi078 podman[93832]: 2024-04-26 04:59:09.976625872 +0000 UTC m=+0.552990313 container died 55f6b2230962a03f2c9f2a1900302133563364d0a0781822eb33b6d159dcf4a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c) 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.509+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91001058b0 con 0x7f91000ffef0 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.509+0000 7f90e7fff700 1 -- 172.21.15.77:0/1988834717 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+436 (secure 0 0 0) 0x7f90f405a200 con 0x7f91000ffef0 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; 1 osds down 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:59:10.511 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:59:10.512 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:59:10.512 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:59:10.512 INFO:teuthology.orchestra.run.smithi077.stdout: osd.4 (root=default,host=smithi078) is down 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 msgr2=0x7f90e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 0x7f90e80705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009b30 tx=0x7f90f0009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 msgr2=0x7f91000fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90f400eb40 tx=0x7f90f400b040 comp rx=0 tx=0).stop 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 shutdown_connections 2024-04-26T04:59:10.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f90e806e140 0x7f90e80705f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9100066f50 0x7f91000fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 --2- 172.21.15.77:0/1988834717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91000ffef0 0x7f91000fb8b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.511+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 >> 172.21.15.77:0/1988834717 conn(0x7f91000f7440 msgr2=0x7f91000f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.512+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 shutdown_connections 2024-04-26T04:59:10.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:10.512+0000 7f9105c4a700 1 -- 172.21.15.77:0/1988834717 wait complete. 2024-04-26T04:59:10.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:10 smithi078 podman[93832]: 2024-04-26 04:59:10.350808276 +0000 UTC m=+0.927172717 container cleanup 55f6b2230962a03f2c9f2a1900302133563364d0a0781822eb33b6d159dcf4a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T04:59:10.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:10 smithi078 bash[93832]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4 2024-04-26T04:59:10.927 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:10 smithi078 podman[93848]: 2024-04-26 04:59:10.602101775 +0000 UTC m=+0.620407153 container remove 55f6b2230962a03f2c9f2a1900302133563364d0a0781822eb33b6d159dcf4a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T04:59:10.928 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:10 smithi078 ceph-mon[79689]: pgmap v140: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:10.928 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:10 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:59:10.928 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:10 smithi078 ceph-mon[79689]: osdmap e70: 6 total, 5 up, 6 in 2024-04-26T04:59:10.928 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:10 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3354118991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:59:10.928 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:10 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1988834717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:59:10.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:10 smithi077 ceph-mon[91833]: pgmap v140: 65 pgs: 65 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:10.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:10 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:59:10.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:10 smithi077 ceph-mon[91833]: osdmap e70: 6 total, 5 up, 6 in 2024-04-26T04:59:10.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:10 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3354118991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:59:10.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:10 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1988834717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:59:11.214 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:10 smithi078 podman[93945]: 2024-04-26 04:59:10.830832163 +0000 UTC m=+0.018729635 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:11.214 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:10 smithi078 podman[93945]: 2024-04-26 04:59:10.928011086 +0000 UTC m=+0.115908549 container create c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:11.215 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:11 smithi078 podman[93945]: 2024-04-26 04:59:11.139031724 +0000 UTC m=+0.326929186 container init c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T04:59:11.215 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:11 smithi078 podman[93945]: 2024-04-26 04:59:11.14213451 +0000 UTC m=+0.330031979 container start c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:11.215 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:11 smithi078 podman[93945]: 2024-04-26 04:59:11.213891513 +0000 UTC m=+0.401788987 container attach c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T04:59:11.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:11 smithi078 podman[93945]: 2024-04-26 04:59:11.436689763 +0000 UTC m=+0.624587220 container died c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:11.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:11 smithi077 ceph-mon[91833]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:11.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:11 smithi077 ceph-mon[91833]: osdmap e71: 6 total, 5 up, 6 in 2024-04-26T04:59:12.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:11 smithi078 ceph-mon[79689]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:12.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:11 smithi078 ceph-mon[79689]: osdmap e71: 6 total, 5 up, 6 in 2024-04-26T04:59:12.502 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[93972]: 2024-04-26 04:59:12.067374612 +0000 UTC m=+0.625828422 container remove c6325f7676054e6e976487379ffd352f7184762b5cdbea14617f84b95b43e971 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T04:59:12.502 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-26T04:59:12.502 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 systemd[1]: Stopped Ceph osd.4 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:59:12.502 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.4.service: Consumed 50.229s CPU time. 2024-04-26T04:59:12.502 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 systemd[1]: Starting Ceph osd.4 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:59:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:12 smithi078 ceph-mon[79689]: pgmap v143: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:12 smithi078 ceph-mon[79689]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T04:59:12.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[94075]: 2024-04-26 04:59:12.403925411 +0000 UTC m=+0.018160290 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:12.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[94075]: 2024-04-26 04:59:12.510232953 +0000 UTC m=+0.124467816 container create 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T04:59:12.768 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[94075]: 2024-04-26 04:59:12.721194451 +0000 UTC m=+0.335429311 container init 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T04:59:12.768 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[94075]: 2024-04-26 04:59:12.724304444 +0000 UTC m=+0.338539306 container start 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T04:59:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:12 smithi077 ceph-mon[91833]: pgmap v143: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:12.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:12 smithi077 ceph-mon[91833]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T04:59:13.169 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:12 smithi078 podman[94075]: 2024-04-26 04:59:12.782898804 +0000 UTC m=+0.397133671 container attach 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T04:59:13.466 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:59:13.466 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 bash[94075]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate[94090]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-26T04:59:13.467 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 podman[94075]: 2024-04-26 04:59:13.192969759 +0000 UTC m=+0.807204611 container died 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T04:59:13.750 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:13 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 39/264 objects degraded (14.773%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:13.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:13 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 39/264 objects degraded (14.773%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:14.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 podman[94225]: 2024-04-26 04:59:13.749519835 +0000 UTC m=+0.550913166 container remove 308c5c2e31172dcea813e41fafd2bfae8057e86bfc1283ba0dae97805507448e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T04:59:14.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 podman[94290]: 2024-04-26 04:59:13.958816569 +0000 UTC m=+0.110705444 container create 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T04:59:14.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:13 smithi078 podman[94290]: 2024-04-26 04:59:13.866122202 +0000 UTC m=+0.018011047 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:14.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 podman[94290]: 2024-04-26 04:59:14.169247036 +0000 UTC m=+0.321135868 container init 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb) 2024-04-26T04:59:14.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 podman[94290]: 2024-04-26 04:59:14.172515242 +0000 UTC m=+0.324404075 container start 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T04:59:14.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-osd[94309]: -- 172.21.15.78:0/3115545767 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e968ad680 con 0x557e96877000 2024-04-26T04:59:14.517 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 bash[94290]: 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a 2024-04-26T04:59:14.518 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 systemd[1]: Started Ceph osd.4 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:59:14.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:14 smithi077 ceph-mon[91833]: pgmap v144: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/264 objects degraded (14.773%) 2024-04-26T04:59:14.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:14 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:14.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:14 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:14.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:14 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:15.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T04:59:14.713+0000 7f95eecc17c0 -1 Falling back to public interface 2024-04-26T04:59:15.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-mon[79689]: pgmap v144: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39/264 objects degraded (14.773%) 2024-04-26T04:59:15.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:15.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:15.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:14 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:15.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:15 smithi077 ceph-mon[91833]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-26T04:59:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:15 smithi078 ceph-mon[79689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-26T04:59:17.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:16 smithi078 ceph-mon[79689]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:17.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:17.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:16 smithi077 ceph-mon[91833]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:18.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:18.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:18.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:18 smithi077 ceph-mon[91833]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:18.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:18.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:18.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:18 smithi078 ceph-mon[79689]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:19.655 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:19.655 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:19.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:19.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:20.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:19 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T04:59:19.653+0000 7f95eecc17c0 -1 osd.4 69 log_to_monitors true 2024-04-26T04:59:20.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:20 smithi077 ceph-mon[91833]: pgmap v147: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:20.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:20 smithi077 ceph-mon[91833]: from='osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:59:20.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:20 smithi077 ceph-mon[91833]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:59:20.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:20 smithi078 ceph-mon[79689]: pgmap v147: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:20.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:20 smithi078 ceph-mon[79689]: from='osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:59:20.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:20 smithi078 ceph-mon[79689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:59:20.768 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 04:59:20 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T04:59:20.447+0000 7f95e0119700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:59:21.718 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:21 smithi078 ceph-mon[79689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:59:21.719 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:21 smithi078 ceph-mon[79689]: osdmap e72: 6 total, 5 up, 6 in 2024-04-26T04:59:21.719 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:21 smithi078 ceph-mon[79689]: from='osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:21.719 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:21 smithi078 ceph-mon[79689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:21.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:21 smithi077 ceph-mon[91833]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:59:21.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:21 smithi077 ceph-mon[91833]: osdmap e72: 6 total, 5 up, 6 in 2024-04-26T04:59:21.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:21 smithi077 ceph-mon[91833]: from='osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:21.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:21 smithi077 ceph-mon[91833]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:22.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175] boot 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: osdmap e73: 6 total, 6 up, 6 in 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: pgmap v150: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: pgmap v152: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:59:22.744 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:22 smithi077 ceph-mon[91833]: osdmap e74: 6 total, 6 up, 6 in 2024-04-26T04:59:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:59:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: osd.4 [v2:172.21.15.78:6808/2142591175,v1:172.21.15.78:6809/2142591175] boot 2024-04-26T04:59:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: osdmap e73: 6 total, 6 up, 6 in 2024-04-26T04:59:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: pgmap v150: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: pgmap v152: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:22.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:59:22.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:22 smithi078 ceph-mon[79689]: osdmap e74: 6 total, 6 up, 6 in 2024-04-26T04:59:23.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:23 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:23.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:23 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:24.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:24 smithi077 ceph-mon[91833]: pgmap v154: 65 pgs: 11 peering, 6 active+undersized+degraded, 7 active+undersized, 41 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/264 objects degraded (4.545%) 2024-04-26T04:59:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:24 smithi078 ceph-mon[79689]: pgmap v154: 65 pgs: 11 peering, 6 active+undersized+degraded, 7 active+undersized, 41 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 12/264 objects degraded (4.545%) 2024-04-26T04:59:26.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:25 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/264 objects degraded (4.545%), 6 pgs degraded) 2024-04-26T04:59:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:25 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/264 objects degraded (4.545%), 6 pgs degraded) 2024-04-26T04:59:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:26 smithi077 ceph-mon[91833]: pgmap v155: 65 pgs: 11 peering, 54 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:26 smithi078 ceph-mon[79689]: pgmap v155: 65 pgs: 11 peering, 54 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:28 smithi077 ceph-mon[91833]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:28 smithi078 ceph-mon[79689]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:30 smithi077 ceph-mon[91833]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:30 smithi078 ceph-mon[79689]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:32.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:32.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:32 smithi077 ceph-mon[91833]: pgmap v158: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:32 smithi078 ceph-mon[79689]: pgmap v158: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:35.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:34 smithi077 ceph-mon[91833]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:34 smithi078 ceph-mon[79689]: pgmap v159: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:37.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:36 smithi077 ceph-mon[91833]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:36 smithi078 ceph-mon[79689]: pgmap v160: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:37 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:37 smithi077 ceph-mon[91833]: Upgrade: osd.5 is safe to restart 2024-04-26T04:59:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:37 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:59:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:37 smithi078 ceph-mon[79689]: Upgrade: osd.5 is safe to restart 2024-04-26T04:59:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:38 smithi077 ceph-mon[91833]: pgmap v161: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:59:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:38 smithi078 ceph-mon[79689]: pgmap v161: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:59:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:40.225 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:39 smithi078 ceph-mon[79689]: Upgrade: Updating osd.5 2024-04-26T04:59:40.225 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:39 smithi078 ceph-mon[79689]: Deploying daemon osd.5 on smithi078 2024-04-26T04:59:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:39 smithi077 ceph-mon[91833]: Upgrade: Updating osd.5 2024-04-26T04:59:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:39 smithi077 ceph-mon[91833]: Deploying daemon osd.5 on smithi078 2024-04-26T04:59:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 -- 172.21.15.77:0/3697158606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 msgr2=0x7fbc380fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/3697158606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc380fd280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbc28004940 tx=0x7fbc28009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 -- 172.21.15.77:0/3697158606 shutdown_connections 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/3697158606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 0x7fbc380fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/3697158606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc380fd280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 -- 172.21.15.77:0/3697158606 >> 172.21.15.77:0/3697158606 conn(0x7fbc380f8550 msgr2=0x7fbc380fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 -- 172.21.15.77:0/3697158606 shutdown_connections 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.628+0000 7fbc3cecf700 1 -- 172.21.15.77:0/3697158606 wait complete. 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 Processor -- start 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 -- start start 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 0x7fbc380702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38072200 con 0x7fbc380fce70 2024-04-26T04:59:40.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3cecf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380707f0 con 0x7fbc380fdfe0 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54470/0 (socket says 172.21.15.77:54470) 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 -- 172.21.15.77:0/245447506 learned_addr learned my addr 172.21.15.77:0/245447506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc35d9b700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 0x7fbc380702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 -- 172.21.15.77:0/245447506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 msgr2=0x7fbc380702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 0x7fbc380702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 -- 172.21.15.77:0/245447506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28005040 con 0x7fbc380fce70 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.629+0000 7fbc3659c700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800b3b0 tx=0x7fbc28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.630+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2800e040 con 0x7fbc380fce70 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.630+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc2800bbf0 con 0x7fbc380fce70 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.630+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc38070a40 con 0x7fbc380fce70 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.630+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2801c630 con 0x7fbc380fce70 2024-04-26T04:59:40.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.630+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc38071000 con 0x7fbc380fce70 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.631+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3804c710 con 0x7fbc380fce70 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.631+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbc2800bd60 con 0x7fbc380fce70 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.632+0000 7fbc26ffd700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 0x7fbc200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.632+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbc2808e5b0 con 0x7fbc380fce70 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.632+0000 7fbc35d9b700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 0x7fbc200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.632+0000 7fbc35d9b700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 0x7fbc200705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c004f40 tx=0x7fbc2c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.636+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbc2805b5d0 con 0x7fbc380fce70 2024-04-26T04:59:40.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.850+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc38102300 con 0x7fbc2006e140 2024-04-26T04:59:40.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.852+0000 7fbc26ffd700 1 -- 172.21.15.77:0/245447506 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbc38102300 con 0x7fbc2006e140 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 msgr2=0x7fbc200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 0x7fbc200705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c004f40 tx=0x7fbc2c00b040 comp rx=0 tx=0).stop 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 msgr2=0x7fbc38071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800b3b0 tx=0x7fbc28009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 shutdown_connections 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbc2006e140 0x7fbc200705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc380fce70 0x7fbc38071c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 --2- 172.21.15.77:0/245447506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc380fdfe0 0x7fbc380702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 >> 172.21.15.77:0/245447506 conn(0x7fbc380f8550 msgr2=0x7fbc380f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 shutdown_connections 2024-04-26T04:59:40.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.854+0000 7fbc3cecf700 1 -- 172.21.15.77:0/245447506 wait complete. 2024-04-26T04:59:40.865 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 -- 172.21.15.77:0/1852892022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fcff0 msgr2=0x7f2b280fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 --2- 172.21.15.77:0/1852892022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fcff0 0x7f2b280fd460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b24004940 tx=0x7f2b24009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 -- 172.21.15.77:0/1852892022 shutdown_connections 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 --2- 172.21.15.77:0/1852892022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fcff0 0x7f2b280fd460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 --2- 172.21.15.77:0/1852892022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fbd40 0x7f2b280fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 -- 172.21.15.77:0/1852892022 >> 172.21.15.77:0/1852892022 conn(0x7f2b280f74c0 msgr2=0x7f2b280f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 -- 172.21.15.77:0/1852892022 shutdown_connections 2024-04-26T04:59:40.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.956+0000 7f2b300f9700 1 -- 172.21.15.77:0/1852892022 wait complete. 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 Processor -- start 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 -- start start 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fcff0 0x7f2b28110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28110720 con 0x7f2b280fbd40 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b300f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28110f70 con 0x7f2b280fcff0 2024-04-26T04:59:40.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b2de95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b2de95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54480/0 (socket says 172.21.15.77:54480) 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.957+0000 7f2b2de95700 1 -- 172.21.15.77:0/2022579054 learned_addr learned my addr 172.21.15.77:0/2022579054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b2d694700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fcff0 0x7f2b28110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b2de95700 1 -- 172.21.15.77:0/2022579054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fcff0 msgr2=0x7f2b28110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b2de95700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fcff0 0x7f2b28110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b2de95700 1 -- 172.21.15.77:0/2022579054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24005040 con 0x7f2b280fbd40 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b2de95700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b18002a50 tx=0x7f2b180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b18007cc0 con 0x7f2b280fbd40 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b28111250 con 0x7f2b280fbd40 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b1800f040 con 0x7f2b280fbd40 2024-04-26T04:59:40.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b18009640 con 0x7f2b280fbd40 2024-04-26T04:59:40.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.958+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28111790 con 0x7f2b280fbd40 2024-04-26T04:59:40.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.959+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28101980 con 0x7f2b280fbd40 2024-04-26T04:59:40.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.959+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b180097a0 con 0x7f2b280fbd40 2024-04-26T04:59:40.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.960+0000 7f2b1e7fc700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 0x7f2b140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:40.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.960+0000 7f2b2d694700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 0x7f2b140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:40.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.960+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2b1808d800 con 0x7f2b280fbd40 2024-04-26T04:59:40.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.961+0000 7f2b2d694700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 0x7f2b140705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b3b0 tx=0x7f2b24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:40.965+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2b1805a780 con 0x7f2b280fbd40 2024-04-26T04:59:41.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.179+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b28111a00 con 0x7f2b1406e140 2024-04-26T04:59:41.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.180+0000 7f2b1e7fc700 1 -- 172.21.15.77:0/2022579054 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2b28111a00 con 0x7f2b1406e140 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 msgr2=0x7f2b140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 0x7f2b140705f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b3b0 tx=0x7f2b24009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 msgr2=0x7f2b2810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b18002a50 tx=0x7f2b180046b0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 shutdown_connections 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2b1406e140 0x7f2b140705f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b280fbd40 0x7f2b2810fbe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 --2- 172.21.15.77:0/2022579054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b280fcff0 0x7f2b28110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 >> 172.21.15.77:0/2022579054 conn(0x7f2b280f74c0 msgr2=0x7f2b280f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:41.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 shutdown_connections 2024-04-26T04:59:41.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.182+0000 7f2b300f9700 1 -- 172.21.15.77:0/2022579054 wait complete. 2024-04-26T04:59:41.221 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:40 smithi078 ceph-mon[79689]: pgmap v162: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:40 smithi077 ceph-mon[91833]: pgmap v162: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:41.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 -- 172.21.15.77:0/264380538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 msgr2=0x7fa3b80fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/264380538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b80fd0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac004940 tx=0x7fa3ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 -- 172.21.15.77:0/264380538 shutdown_connections 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/264380538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b80fd0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/264380538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 -- 172.21.15.77:0/264380538 >> 172.21.15.77:0/264380538 conn(0x7fa3b80f6be0 msgr2=0x7fa3b80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 -- 172.21.15.77:0/264380538 shutdown_connections 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.284+0000 7fa3bccb2700 1 -- 172.21.15.77:0/264380538 wait complete. 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 Processor -- start 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 -- start start 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8112c70 con 0x7fa3b80fcc60 2024-04-26T04:59:41.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3bccb2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8112db0 con 0x7fa3b80fc310 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54500/0 (socket says 172.21.15.77:54500) 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:34850/0 (socket says 172.21.15.77:34850) 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 -- 172.21.15.77:0/3955623876 learned_addr learned my addr 172.21.15.77:0/3955623876 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 -- 172.21.15.77:0/3955623876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 msgr2=0x7fa3b80fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 -- 172.21.15.77:0/3955623876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac005040 con 0x7fa3b80fcc60 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.285+0000 7fa3b5d9b700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b970 tx=0x7fa3ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.286+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ac00e040 con 0x7fa3b80fcc60 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.286+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3ac00bbf0 con 0x7fa3b80fcc60 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.286+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b8112fb0 con 0x7fa3b80fcc60 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.286+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ac01c630 con 0x7fa3b80fcc60 2024-04-26T04:59:41.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.286+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b8113540 con 0x7fa3b80fcc60 2024-04-26T04:59:41.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.287+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b804c710 con 0x7fa3b80fcc60 2024-04-26T04:59:41.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.287+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3ac00bd60 con 0x7fa3b80fcc60 2024-04-26T04:59:41.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.288+0000 7fa3a6ffd700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 0x7fa3a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.288+0000 7fa3b659c700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 0x7fa3a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.288+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa3ac08ecf0 con 0x7fa3b80fcc60 2024-04-26T04:59:41.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.288+0000 7fa3b659c700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 0x7fa3a00705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8004f40 tx=0x7fa3a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:41.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.292+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3ac05bd10 con 0x7fa3b80fcc60 2024-04-26T04:59:41.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.496+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3b804c710 con 0x7fa3a006e140 2024-04-26T04:59:41.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.506+0000 7fa3a6ffd700 1 -- 172.21.15.77:0/3955623876 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa3b804c710 con 0x7fa3a006e140 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (6m) 90s ago 15m 25.2M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (3m) 90s ago 15m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (3m) 24s ago 14m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 90s ago 14m - - 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (9m) 90s ago 9m 152M - 17.2.7-902-gc61472c8 0de407c1b3af dd975265f32a 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (9m) 90s ago 9m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (9m) 24s ago 9m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (9m) 24s ago 9m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T04:59:41.507 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (5m) 90s ago 16m 503M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (4m) 24s ago 14m 452M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (4m) 90s ago 16m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (3m) 24s ago 14m 38.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (6m) 90s ago 15m 15.9M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (5m) 24s ago 14m 16.5M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (2m) 90s ago 13m 86.4M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (2m) 90s ago 12m 83.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (95s) 90s ago 12m 14.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (60s) 24s ago 12m 101M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (27s) 24s ago 11m 12.1M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (11m) 24s ago 11m 194M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 3055dda04e03 2024-04-26T04:59:41.508 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (4m) 90s ago 14m 59.0M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T04:59:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 msgr2=0x7fa3a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 0x7fa3a00705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8004f40 tx=0x7fa3a800b040 comp rx=0 tx=0).stop 2024-04-26T04:59:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 msgr2=0x7fa3b8112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00b970 tx=0x7fa3ac009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 shutdown_connections 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3a006e140 0x7fa3a00705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3b80fc310 0x7fa3b80fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 --2- 172.21.15.77:0/3955623876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3b80fcc60 0x7fa3b8112730 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 >> 172.21.15.77:0/3955623876 conn(0x7fa3b80f6be0 msgr2=0x7fa3b80f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 shutdown_connections 2024-04-26T04:59:41.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.508+0000 7fa3bccb2700 1 -- 172.21.15.77:0/3955623876 wait complete. 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- 172.21.15.77:0/3326457266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fce60 msgr2=0x7f35d40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 --2- 172.21.15.77:0/3326457266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fce60 0x7f35d40fd270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ce0 tx=0x7f35d0004d10 comp rx=0 tx=0).stop 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- 172.21.15.77:0/3326457266 shutdown_connections 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 --2- 172.21.15.77:0/3326457266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fdfd0 0x7f35d40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 --2- 172.21.15.77:0/3326457266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fce60 0x7f35d40fd270 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- 172.21.15.77:0/3326457266 >> 172.21.15.77:0/3326457266 conn(0x7f35d40f8540 msgr2=0x7f35d40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- 172.21.15.77:0/3326457266 shutdown_connections 2024-04-26T04:59:41.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- 172.21.15.77:0/3326457266 wait complete. 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 Processor -- start 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.607+0000 7f35dc424700 1 -- start start 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35dc424700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fce60 0x7f35d4071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35dc424700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35dc424700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40708f0 con 0x7f35d40fdfd0 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35dc424700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4070a60 con 0x7f35d40fce60 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54522/0 (socket says 172.21.15.77:54522) 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35da1c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fce60 0x7f35d4071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 -- 172.21.15.77:0/2478742013 learned_addr learned my addr 172.21.15.77:0/2478742013 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 -- 172.21.15.77:0/2478742013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fce60 msgr2=0x7f35d4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fce60 0x7f35d4071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 -- 172.21.15.77:0/2478742013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009940 con 0x7f35d40fdfd0 2024-04-26T04:59:41.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35d99bf700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b790 tx=0x7f35c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:41.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c400f6f0 con 0x7f35d40fdfd0 2024-04-26T04:59:41.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35c400fd20 con 0x7f35d40fdfd0 2024-04-26T04:59:41.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d4070d40 con 0x7f35d40fdfd0 2024-04-26T04:59:41.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.608+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c40103a0 con 0x7f35d40fdfd0 2024-04-26T04:59:41.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.609+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d4115c30 con 0x7f35d40fdfd0 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.609+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d405f5c0 con 0x7f35d40fdfd0 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.610+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f35c4011400 con 0x7f35d40fdfd0 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.611+0000 7f35caffd700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 0x7f35c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.611+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f35c408e290 con 0x7f35d40fdfd0 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.611+0000 7f35da1c0700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 0x7f35c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:41.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.611+0000 7f35da1c0700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 0x7f35c00705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009470 tx=0x7f35d0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:41.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.618+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f35c4057780 con 0x7f35d40fdfd0 2024-04-26T04:59:41.826 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:41 smithi078 systemd[1]: Stopping Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:59:41.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.898+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f35d404bed0 con 0x7f35d40fdfd0 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.899+0000 7f35caffd700 1 -- 172.21.15.77:0/2478742013 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f35c405ae40 con 0x7f35d40fdfd0 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T04:59:41.900 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 5 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 9 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T04:59:41.901 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:59:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 msgr2=0x7f35c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 0x7f35c00705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009470 tx=0x7f35d0004d10 comp rx=0 tx=0).stop 2024-04-26T04:59:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 msgr2=0x7f35d40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b790 tx=0x7f35c4009250 comp rx=0 tx=0).stop 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 shutdown_connections 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35c006e140 0x7f35c00705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d40fce60 0x7f35d4071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 --2- 172.21.15.77:0/2478742013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d40fdfd0 0x7f35d40702b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 >> 172.21.15.77:0/2478742013 conn(0x7f35d40f8540 msgr2=0x7f35d40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 shutdown_connections 2024-04-26T04:59:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:41.901+0000 7f35dc424700 1 -- 172.21.15.77:0/2478742013 wait complete. 2024-04-26T04:59:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 -- 172.21.15.77:0/2946292002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 msgr2=0x7f48a00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2946292002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a00fd270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4890004940 tx=0x7f4890009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 -- 172.21.15.77:0/2946292002 shutdown_connections 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2946292002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 0x7f48a00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2946292002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a00fd270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 -- 172.21.15.77:0/2946292002 >> 172.21.15.77:0/2946292002 conn(0x7f48a00f8540 msgr2=0x7f48a00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 -- 172.21.15.77:0/2946292002 shutdown_connections 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.007+0000 7f48a4a32700 1 -- 172.21.15.77:0/2946292002 wait complete. 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 Processor -- start 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 -- start start 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 0x7f48a010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a010e470 con 0x7f48a00fce60 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f48a4a32700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a010e5e0 con 0x7f48a00fdfd0 2024-04-26T04:59:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54538/0 (socket says 172.21.15.77:54538) 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 -- 172.21.15.77:0/2544861571 learned_addr learned my addr 172.21.15.77:0/2544861571 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489dd9b700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 0x7f48a010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 -- 172.21.15.77:0/2544861571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 msgr2=0x7f48a010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 0x7f48a010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 -- 172.21.15.77:0/2544861571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890005040 con 0x7f48a00fce60 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.008+0000 7f489e59c700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f489000b3b0 tx=0x7f4890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.009+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489000e040 con 0x7f48a00fce60 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.009+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f489000bbf0 con 0x7f48a00fce60 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.009+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489001c630 con 0x7f48a00fce60 2024-04-26T04:59:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.009+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a0112fe0 con 0x7f48a00fce60 2024-04-26T04:59:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.009+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a0113510 con 0x7f48a00fce60 2024-04-26T04:59:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.010+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a004c710 con 0x7f48a00fce60 2024-04-26T04:59:42.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.010+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f489000bd60 con 0x7f48a00fce60 2024-04-26T04:59:42.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.011+0000 7f488effd700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 0x7f48880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.011+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f489008e690 con 0x7f48a00fce60 2024-04-26T04:59:42.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.011+0000 7f489dd9b700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 0x7f48880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.011+0000 7f489dd9b700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 0x7f48880705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4894004f60 tx=0x7f489400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.015+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f489005b610 con 0x7f48a00fce60 2024-04-26T04:59:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:41 smithi077 ceph-mon[91833]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:41 smithi077 ceph-mon[91833]: osd.5 marked itself down and dead 2024-04-26T04:59:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:41 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2478742013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:42.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.255+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f48a01058e0 con 0x7f48a00fce60 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.255+0000 7f488effd700 1 -- 172.21.15.77:0/2544861571 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f489005b1a0 con 0x7f48a00fce60 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:e14 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T04:59:42.256 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 13 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T04:50:10.502841+0000 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T04:59:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14458} 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T04:59:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.77:6826/132419939,v1:172.21.15.77:6827/132419939] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:14470} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:59:42.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 msgr2=0x7f48880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 0x7f48880705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4894004f60 tx=0x7f489400b040 comp rx=0 tx=0).stop 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 msgr2=0x7f48a010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f489000b3b0 tx=0x7f4890009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 shutdown_connections 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f488806e140 0x7f48880705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48a00fce60 0x7f48a010d930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 --2- 172.21.15.77:0/2544861571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a00fdfd0 0x7f48a010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 >> 172.21.15.77:0/2544861571 conn(0x7f48a00f8540 msgr2=0x7f48a00f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.257+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 shutdown_connections 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.258+0000 7f48a4a32700 1 -- 172.21.15.77:0/2544861571 wait complete. 2024-04-26T04:59:42.261 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T04:59:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-mon[79689]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-mon[79689]: osd.5 marked itself down and dead 2024-04-26T04:59:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2478742013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:42.267 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[52660]: 2024-04-26T04:59:41.825+0000 7f65c1de9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:59:42.267 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[52660]: 2024-04-26T04:59:41.825+0000 7f65c1de9700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-26T04:59:42.268 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:41 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[52660]: 2024-04-26T04:59:41.825+0000 7f65c1de9700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 -- 172.21.15.77:0/85740914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 msgr2=0x7f9d240fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 --2- 172.21.15.77:0/85740914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d240fbf00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004940 tx=0x7f9d18009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 -- 172.21.15.77:0/85740914 shutdown_connections 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 --2- 172.21.15.77:0/85740914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 0x7f9d240fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 --2- 172.21.15.77:0/85740914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d240fbf00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 -- 172.21.15.77:0/85740914 >> 172.21.15.77:0/85740914 conn(0x7f9d240f7250 msgr2=0x7f9d240f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 -- 172.21.15.77:0/85740914 shutdown_connections 2024-04-26T04:59:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 -- 172.21.15.77:0/85740914 wait complete. 2024-04-26T04:59:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.360+0000 7f9d291a0700 1 Processor -- start 2024-04-26T04:59:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 -- start start 2024-04-26T04:59:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 0x7f9d2410dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2410e370 con 0x7f9d240fbaf0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2410eb70 con 0x7f9d240fcda0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54554/0 (socket says 172.21.15.77:54554) 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 -- 172.21.15.77:0/890421252 learned_addr learned my addr 172.21.15.77:0/890421252 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d237fe700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 0x7f9d2410dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 -- 172.21.15.77:0/890421252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 msgr2=0x7f9d2410dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 0x7f9d2410dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 -- 172.21.15.77:0/890421252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18005040 con 0x7f9d240fbaf0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d23fff700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009150 tx=0x7f9d18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1800e040 con 0x7f9d240fbaf0 2024-04-26T04:59:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2410edf0 con 0x7f9d240fbaf0 2024-04-26T04:59:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.361+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d18004290 con 0x7f9d240fbaf0 2024-04-26T04:59:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.362+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2410f330 con 0x7f9d240fbaf0 2024-04-26T04:59:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.362+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1801c5d0 con 0x7f9d240fbaf0 2024-04-26T04:59:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.363+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24101930 con 0x7f9d240fbaf0 2024-04-26T04:59:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.363+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9d1800bbf0 con 0x7f9d240fbaf0 2024-04-26T04:59:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.363+0000 7f9d20ff9700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.363+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9d1808e510 con 0x7f9d240fbaf0 2024-04-26T04:59:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.364+0000 7f9d237fe700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.364+0000 7f9d237fe700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 0x7f9d0c0705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009fd0 tx=0x7f9d1400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.368+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9d1805b540 con 0x7f9d240fbaf0 2024-04-26T04:59:42.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.584+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d2410f5a0 con 0x7f9d0c06e140 2024-04-26T04:59:42.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.586+0000 7f9d20ff9700 1 -- 172.21.15.77:0/890421252 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9d2410f5a0 con 0x7f9d0c06e140 2024-04-26T04:59:42.586 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T04:59:42.586 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:59:42.586 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "11/21 daemons upgraded", 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T04:59:42.587 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 msgr2=0x7f9d0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 0x7f9d0c0705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009fd0 tx=0x7f9d1400b040 comp rx=0 tx=0).stop 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 msgr2=0x7f9d2410d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009150 tx=0x7f9d18009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 shutdown_connections 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d240fbaf0 0x7f9d2410d830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 --2- 172.21.15.77:0/890421252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d240fcda0 0x7f9d2410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 >> 172.21.15.77:0/890421252 conn(0x7f9d240f7250 msgr2=0x7f9d24065b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 shutdown_connections 2024-04-26T04:59:42.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.588+0000 7f9d291a0700 1 -- 172.21.15.77:0/890421252 wait complete. 2024-04-26T04:59:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 -- 172.21.15.77:0/261684063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 msgr2=0x7f57740fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 --2- 172.21.15.77:0/261684063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57740fc0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5768004940 tx=0x7f5768009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 -- 172.21.15.77:0/261684063 shutdown_connections 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 --2- 172.21.15.77:0/261684063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f57740fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 --2- 172.21.15.77:0/261684063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57740fc0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 -- 172.21.15.77:0/261684063 >> 172.21.15.77:0/261684063 conn(0x7f57740f7430 msgr2=0x7f57740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 -- 172.21.15.77:0/261684063 shutdown_connections 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 -- 172.21.15.77:0/261684063 wait complete. 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.688+0000 7f577a289700 1 Processor -- start 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f577a289700 1 -- start start 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f577a289700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57741106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f577a289700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f5774112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f577a289700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774066330 con 0x7f57740fbcb0 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f577a289700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774066470 con 0x7f57740fcf60 2024-04-26T04:59:42.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f5773fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57741106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f5773fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57741106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54576/0 (socket says 172.21.15.77:54576) 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f5773fff700 1 -- 172.21.15.77:0/2534819166 learned_addr learned my addr 172.21.15.77:0/2534819166 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f57737fe700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f5774112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f57737fe700 1 -- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 msgr2=0x7f57741106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f57737fe700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57741106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f57737fe700 1 -- 172.21.15.77:0/2534819166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768005040 con 0x7f57740fcf60 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.689+0000 7f57737fe700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f5774112bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fd0 tx=0x7f576400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.690+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5764004e90 con 0x7f57740fcf60 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.690+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f576400f040 con 0x7f57740fcf60 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.690+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5774113130 con 0x7f57740fcf60 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.690+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576400ca50 con 0x7f57740fcf60 2024-04-26T04:59:42.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.690+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5774113670 con 0x7f57740fcf60 2024-04-26T04:59:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.691+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577405f5c0 con 0x7f57740fcf60 2024-04-26T04:59:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.691+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5764007cc0 con 0x7f57740fcf60 2024-04-26T04:59:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.691+0000 7f5770ff9700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 0x7f575c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:59:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.692+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f576408d5a0 con 0x7f57740fcf60 2024-04-26T04:59:42.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.692+0000 7f5773fff700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 0x7f575c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:59:42.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.692+0000 7f5773fff700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 0x7f575c0705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5768009b30 tx=0x7f5768009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:59:42.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.696+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5764056980 con 0x7f57740fcf60 2024-04-26T04:59:42.751 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:42 smithi078 podman[98861]: 2024-04-26 04:59:42.355333858 +0000 UTC m=+0.631624741 container died 3055dda04e034aaf34465e8f073389eda96a4251c53267c158388519b0c39c43 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-26T04:59:42.985 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-04-26T04:59:42.985 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T04:59:42.985 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: osd.5 (root=default,host=smithi078) is down 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.983+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f577404bed0 con 0x7f57740fcf60 2024-04-26T04:59:42.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.984+0000 7f5770ff9700 1 -- 172.21.15.77:0/2534819166 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+673 (secure 0 0 0) 0x7f576405a040 con 0x7f57740fcf60 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 msgr2=0x7f575c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 0x7f575c0705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5768009b30 tx=0x7f5768009e30 comp rx=0 tx=0).stop 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 msgr2=0x7f5774112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f5774112bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fd0 tx=0x7f576400b040 comp rx=0 tx=0).stop 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 shutdown_connections 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f575c06e140 0x7f575c0705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57740fbcb0 0x7f57741106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 --2- 172.21.15.77:0/2534819166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57740fcf60 0x7f5774112bf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 >> 172.21.15.77:0/2534819166 conn(0x7f57740f7430 msgr2=0x7f57740f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:59:42.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 shutdown_connections 2024-04-26T04:59:42.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T04:59:42.988+0000 7f577a289700 1 -- 172.21.15.77:0/2534819166 wait complete. 2024-04-26T04:59:43.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:42 smithi078 podman[98861]: 2024-04-26 04:59:42.849888642 +0000 UTC m=+1.126179530 container cleanup 3055dda04e034aaf34465e8f073389eda96a4251c53267c158388519b0c39c43 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T04:59:43.018 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:42 smithi078 bash[98861]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: pgmap v163: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: osdmap e75: 6 total, 5 up, 6 in 2024-04-26T04:59:43.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:42 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2544861571' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:59:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: pgmap v163: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:59:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:59:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-26T04:59:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: osdmap e75: 6 total, 5 up, 6 in 2024-04-26T04:59:43.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:42 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2544861571' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:59:43.410 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98887]: 2024-04-26 04:59:43.101096816 +0000 UTC m=+0.744781710 container remove 3055dda04e034aaf34465e8f073389eda96a4251c53267c158388519b0c39c43 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:43.671 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.410189284 +0000 UTC m=+0.107319317 container create 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb) 2024-04-26T04:59:43.671 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.320929804 +0000 UTC m=+0.018059837 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:43.671 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.604604098 +0000 UTC m=+0.301734122 container init 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:43.671 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.607541582 +0000 UTC m=+0.304671606 container start 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T04:59:43.672 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.671050679 +0000 UTC m=+0.368180715 container attach 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:59:43.986 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:43 smithi078 podman[98984]: 2024-04-26 04:59:43.899954373 +0000 UTC m=+0.597084407 container died 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-b22e2eb, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T04:59:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:43 smithi077 ceph-mon[91833]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:43 smithi077 ceph-mon[91833]: osdmap e76: 6 total, 5 up, 6 in 2024-04-26T04:59:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:43 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2534819166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:59:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:43 smithi078 ceph-mon[79689]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:59:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:43 smithi078 ceph-mon[79689]: osdmap e76: 6 total, 5 up, 6 in 2024-04-26T04:59:44.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:43 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2534819166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:59:44.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 podman[99010]: 2024-04-26 04:59:44.441077097 +0000 UTC m=+0.535840451 container remove 1c2ea5721882cfce246c9c4449ea4c5ee81b3b4bac8fedd6a49e6bebd625c0e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:44.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-26T04:59:44.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 systemd[1]: Stopped Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:59:44.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service: Consumed 48.763s CPU time. 2024-04-26T04:59:44.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 systemd[1]: Starting Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T04:59:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:44 smithi077 ceph-mon[91833]: pgmap v166: 65 pgs: 1 activating+undersized+degraded, 9 activating+undersized, 3 peering, 8 stale+active+clean, 44 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/264 objects degraded (9.091%) 2024-04-26T04:59:45.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:44 smithi077 ceph-mon[91833]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:59:45.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:44 smithi077 ceph-mon[91833]: Health check failed: Degraded data redundancy: 24/264 objects degraded (9.091%), 1 pg degraded (PG_DEGRADED) 2024-04-26T04:59:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:44 smithi078 ceph-mon[79689]: pgmap v166: 65 pgs: 1 activating+undersized+degraded, 9 activating+undersized, 3 peering, 8 stale+active+clean, 44 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24/264 objects degraded (9.091%) 2024-04-26T04:59:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:44 smithi078 ceph-mon[79689]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:59:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:44 smithi078 ceph-mon[79689]: Health check failed: Degraded data redundancy: 24/264 objects degraded (9.091%), 1 pg degraded (PG_DEGRADED) 2024-04-26T04:59:45.268 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 podman[99116]: 2024-04-26 04:59:44.774731953 +0000 UTC m=+0.018259200 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:45.268 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:44 smithi078 podman[99116]: 2024-04-26 04:59:44.875774674 +0000 UTC m=+0.119301967 container create 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:59:45.268 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 podman[99116]: 2024-04-26 04:59:45.095285463 +0000 UTC m=+0.338812697 container init 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:59:45.268 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 podman[99116]: 2024-04-26 04:59:45.09852319 +0000 UTC m=+0.342050425 container start 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 podman[99116]: 2024-04-26 04:59:45.289207608 +0000 UTC m=+0.532734853 container attach 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:59:45.560 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 bash[99116]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate[99131]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-26T04:59:45.561 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:45 smithi078 podman[99116]: 2024-04-26 04:59:45.558640574 +0000 UTC m=+0.802167824 container died 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-26T04:59:46.267 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 podman[99250]: 2024-04-26 04:59:46.106751305 +0000 UTC m=+0.539965820 container remove 1f9099191dc87af23f186595a089487ea52156eb6de231f4d3782862d642e166 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:59:46.575 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 podman[99312]: 2024-04-26 04:59:46.223749314 +0000 UTC m=+0.017814879 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:59:46.575 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 podman[99312]: 2024-04-26 04:59:46.324274142 +0000 UTC m=+0.118339725 container create 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2) 2024-04-26T04:59:46.575 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 podman[99312]: 2024-04-26 04:59:46.551502286 +0000 UTC m=+0.345567833 container init 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5) 2024-04-26T04:59:46.575 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 podman[99312]: 2024-04-26 04:59:46.554512722 +0000 UTC m=+0.348578263 container start 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb) 2024-04-26T04:59:46.937 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-osd[99331]: -- 172.21.15.78:0/2594448800 <== mon.1 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56301a71b680 con 0x56301a6e4c00 2024-04-26T04:59:46.937 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 bash[99312]: 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 2024-04-26T04:59:46.937 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:46 smithi078 systemd[1]: Started Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: pgmap v167: 65 pgs: 5 active+undersized, 39 active+clean, 3 stale+active+clean, 3 peering, 5 active+undersized+degraded, 9 activating+undersized, 1 activating+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/264 objects degraded (14.015%) 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:47.267 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:47 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T04:59:47.094+0000 7fdf041cc7c0 -1 Falling back to public interface 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: pgmap v167: 65 pgs: 5 active+undersized, 39 active+clean, 3 stale+active+clean, 3 peering, 5 active+undersized+degraded, 9 activating+undersized, 1 activating+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/264 objects degraded (14.015%) 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:48.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:47 smithi077 ceph-mon[91833]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:59:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:47 smithi078 ceph-mon[79689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:59:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:48 smithi077 ceph-mon[91833]: pgmap v168: 65 pgs: 11 active+undersized, 37 active+clean, 7 active+undersized+degraded, 9 activating+undersized, 1 activating+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:48 smithi078 ceph-mon[79689]: pgmap v168: 65 pgs: 11 active+undersized, 37 active+clean, 7 active+undersized+degraded, 9 activating+undersized, 1 activating+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:50 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:50 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:50 smithi078 ceph-mon[79689]: pgmap v169: 65 pgs: 15 active+undersized, 37 active+clean, 13 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:51.045 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:50 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.045 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:50 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.045 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:50 smithi077 ceph-mon[91833]: pgmap v169: 65 pgs: 15 active+undersized, 37 active+clean, 13 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:51.517 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T04:59:51.252+0000 7fdf041cc7c0 -1 osd.5 74 log_to_monitors true 2024-04-26T04:59:51.874 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:51.874 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.874 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:51.874 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-mon[79689]: from='osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:59:51.874 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-mon[79689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:59:52.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:51 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:52.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:52.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:51 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:52.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:51 smithi077 ceph-mon[91833]: from='osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:59:52.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:51 smithi077 ceph-mon[91833]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:59:52.267 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 04:59:51 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T04:59:51.923+0000 7fdef5624700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:59:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:52 smithi077 ceph-mon[91833]: pgmap v170: 65 pgs: 15 active+undersized, 37 active+clean, 13 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:52 smithi077 ceph-mon[91833]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:59:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:52 smithi077 ceph-mon[91833]: from='osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:52 smithi077 ceph-mon[91833]: osdmap e77: 6 total, 5 up, 6 in 2024-04-26T04:59:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:52 smithi077 ceph-mon[91833]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:52 smithi078 ceph-mon[79689]: pgmap v170: 65 pgs: 15 active+undersized, 37 active+clean, 13 active+undersized+degraded; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/264 objects degraded (15.530%) 2024-04-26T04:59:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:52 smithi078 ceph-mon[79689]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:59:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:52 smithi078 ceph-mon[79689]: from='osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:52 smithi078 ceph-mon[79689]: osdmap e77: 6 total, 5 up, 6 in 2024-04-26T04:59:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:52 smithi078 ceph-mon[79689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-26T04:59:54.003 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:53 smithi078 ceph-mon[79689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:59:54.003 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:53 smithi078 ceph-mon[79689]: osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516] boot 2024-04-26T04:59:54.003 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:53 smithi078 ceph-mon[79689]: osdmap e78: 6 total, 6 up, 6 in 2024-04-26T04:59:54.003 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:53 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:59:54.004 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:53 smithi078 ceph-mon[79689]: osdmap e79: 6 total, 6 up, 6 in 2024-04-26T04:59:54.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:53 smithi077 ceph-mon[91833]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:59:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:53 smithi077 ceph-mon[91833]: osd.5 [v2:172.21.15.78:6816/1951816516,v1:172.21.15.78:6817/1951816516] boot 2024-04-26T04:59:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:53 smithi077 ceph-mon[91833]: osdmap e78: 6 total, 6 up, 6 in 2024-04-26T04:59:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:53 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:59:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:53 smithi077 ceph-mon[91833]: osdmap e79: 6 total, 6 up, 6 in 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: pgmap v173: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-26T04:59:55.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:54 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-26T04:59:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: pgmap v173: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-26T04:59:55.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-26T04:59:55.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:54 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-26T04:59:56.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: pgmap v175: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:56.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: pgmap v176: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all osd 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: Upgrade: Setting require_osd_release to 19 squid 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: osdmap e80: 6 total, 6 up, 6 in 2024-04-26T04:59:56.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:55 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi077.quaiyg"]}]: dispatch 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: pgmap v175: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: pgmap v176: 65 pgs: 7 peering, 13 active+undersized, 37 active+clean, 8 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21/264 objects degraded (7.955%) 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all osd 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: Upgrade: Setting require_osd_release to 19 squid 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: osdmap e80: 6 total, 6 up, 6 in 2024-04-26T04:59:56.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:55 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi077.quaiyg"]}]: dispatch 2024-04-26T04:59:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:56 smithi077 ceph-mon[91833]: Upgrade: It appears safe to stop mds.cephfs.smithi077.quaiyg 2024-04-26T04:59:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:56 smithi078 ceph-mon[79689]: Upgrade: It appears safe to stop mds.cephfs.smithi077.quaiyg 2024-04-26T04:59:58.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:57 smithi077 ceph-mon[91833]: pgmap v178: 65 pgs: 7 peering, 11 active+undersized, 42 active+clean, 5 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15/264 objects degraded (5.682%) 2024-04-26T04:59:58.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:57 smithi077 ceph-mon[91833]: Health check update: Degraded data redundancy: 15/264 objects degraded (5.682%), 5 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:58.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:58.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:59:58.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:57 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:57 smithi078 ceph-mon[79689]: pgmap v178: 65 pgs: 7 peering, 11 active+undersized, 42 active+clean, 5 active+undersized+degraded; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15/264 objects degraded (5.682%) 2024-04-26T04:59:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:57 smithi078 ceph-mon[79689]: Health check update: Degraded data redundancy: 15/264 objects degraded (5.682%), 5 pgs degraded (PG_DEGRADED) 2024-04-26T04:59:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T04:59:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.quaiyg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:59:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:57 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:59:59.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:58 smithi077 ceph-mon[91833]: Upgrade: Updating mds.cephfs.smithi077.quaiyg 2024-04-26T04:59:59.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:58 smithi077 ceph-mon[91833]: Deploying daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:59:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:58 smithi078 ceph-mon[79689]: Upgrade: Updating mds.cephfs.smithi077.quaiyg 2024-04-26T04:59:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:58 smithi078 ceph-mon[79689]: Deploying daemon mds.cephfs.smithi077.quaiyg on smithi077 2024-04-26T04:59:59.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T04:59:59.293+0000 7fe623188700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:00.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:00:00.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/264 objects degraded (5.682%), 5 pgs degraded) 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: osdmap e81: 6 total, 6 up, 6 in 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: Standby daemon mds.cephfs.smithi077.urxtgp assigned to filesystem cephfs as rank 0 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:00.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 04:59:59 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:replay} 2 up:standby 2024-04-26T05:00:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:00:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/264 objects degraded (5.682%), 5 pgs degraded) 2024-04-26T05:00:00.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: osdmap e81: 6 total, 6 up, 6 in 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: Standby daemon mds.cephfs.smithi077.urxtgp assigned to filesystem cephfs as rank 0 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:00.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 04:59:59 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:replay} 2 up:standby 2024-04-26T05:00:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: fs cephfs is degraded 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: fs cephfs is degraded 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:00:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:01.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:02.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:01 smithi077 ceph-mon[91833]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-04-26T05:00:02.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:00:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:01 smithi078 ceph-mon[79689]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-04-26T05:00:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:00:04.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:03 smithi077 ceph-mon[91833]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.2 MiB/s rd, 3 op/s 2024-04-26T05:00:04.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:03 smithi078 ceph-mon[79689]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.2 MiB/s rd, 3 op/s 2024-04-26T05:00:06.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:05 smithi077 ceph-mon[91833]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-26T05:00:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:05 smithi078 ceph-mon[79689]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-26T05:00:07.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:07.459 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:07 smithi077 ceph-mon[91833]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:00:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:07 smithi078 ceph-mon[79689]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:00:09.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:09 smithi077 ceph-mon[91833]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:00:09.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:09 smithi078 ceph-mon[79689]: pgmap v185: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:00:10.519 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:10 smithi077 ceph-mon[91833]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.00099998 2024-04-26T05:00:10.519 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:10 smithi077 ceph-mon[91833]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.00099998 2024-04-26T05:00:10.519 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:10 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:reconnect 2024-04-26T05:00:10.519 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:10 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:reconnect} 2 up:standby 2024-04-26T05:00:10.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:10 smithi078 ceph-mon[79689]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.00099998 2024-04-26T05:00:10.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:10 smithi078 ceph-mon[79689]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.00099998 2024-04-26T05:00:10.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:10 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:reconnect 2024-04-26T05:00:10.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:10 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:reconnect} 2 up:standby 2024-04-26T05:00:11.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:11 smithi077 ceph-mon[91833]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-04-26T05:00:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:11 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:rejoin 2024-04-26T05:00:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:11 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:rejoin} 2 up:standby 2024-04-26T05:00:11.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:11 smithi077 ceph-mon[91833]: daemon mds.cephfs.smithi077.urxtgp is now active in filesystem cephfs as rank 0 2024-04-26T05:00:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:11 smithi078 ceph-mon[79689]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-04-26T05:00:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:11 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:rejoin 2024-04-26T05:00:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:11 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.urxtgp=up:rejoin} 2 up:standby 2024-04-26T05:00:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:11 smithi078 ceph-mon[79689]: daemon mds.cephfs.smithi077.urxtgp is now active in filesystem cephfs as rank 0 2024-04-26T05:00:12.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:active 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:boot 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi077.urxtgp=up:active} 3 up:standby 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:12.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:00:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] up:active 2024-04-26T05:00:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:boot 2024-04-26T05:00:12.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi077.urxtgp=up:active} 3 up:standby 2024-04-26T05:00:12.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:00:12.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:12.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:12.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:13.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.095+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/1093500875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fce60 msgr2=0x7fe1d40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.095+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/1093500875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fce60 0x7fe1d40fd270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4004940 tx=0x7fe1c4009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/1093500875 shutdown_connections 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/1093500875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/1093500875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fce60 0x7fe1d40fd270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/1093500875 >> 172.21.15.77:0/1093500875 conn(0x7fe1d40f8540 msgr2=0x7fe1d40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/1093500875 shutdown_connections 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/1093500875 wait complete. 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 Processor -- start 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- start start 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 0x7fe1d410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d410e500 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d8f1c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4112ea0 con 0x7fe1d40fce60 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.096+0000 7fe1d1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57670/0 (socket says 172.21.15.77:57670) 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 -- 172.21.15.77:0/3668137848 learned_addr learned my addr 172.21.15.77:0/3668137848 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:13.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d259c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 0x7fe1d410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 -- 172.21.15.77:0/3668137848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 msgr2=0x7fe1d410d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 0x7fe1d410d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 -- 172.21.15.77:0/3668137848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4005040 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d259c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 0x7fe1d410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d1d9b700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800b790 tx=0x7fe1c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c800f770 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d4113100 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1c800fda0 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c800d310 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.097+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d4113670 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.098+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d404c710 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.099+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe1c8010420 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.099+0000 7fe1c2ffd700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 0x7fe1bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.099+0000 7fe1d259c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 0x7fe1bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.099+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe1c808db40 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.100+0000 7fe1d259c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 0x7fe1bc0705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b970 tx=0x7fe1c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.104+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe1c805aa70 con 0x7fe1d40fdfd0 2024-04-26T05:00:13.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.340+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1d4102270 con 0x7fe1bc06e140 2024-04-26T05:00:13.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.341+0000 7fe1c2ffd700 1 -- 172.21.15.77:0/3668137848 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe1d4102270 con 0x7fe1bc06e140 2024-04-26T05:00:13.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 msgr2=0x7fe1bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 0x7fe1bc0705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400b970 tx=0x7fe1c4009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:13.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 msgr2=0x7fe1d410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800b790 tx=0x7fe1c8009250 comp rx=0 tx=0).stop 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 shutdown_connections 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe1bc06e140 0x7fe1bc0705f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1d40fce60 0x7fe1d410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 --2- 172.21.15.77:0/3668137848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1d40fdfd0 0x7fe1d410de70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.343+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 >> 172.21.15.77:0/3668137848 conn(0x7fe1d40f8540 msgr2=0x7fe1d40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.344+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 shutdown_connections 2024-04-26T05:00:13.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.344+0000 7fe1d8f1c700 1 -- 172.21.15.77:0/3668137848 wait complete. 2024-04-26T05:00:13.355 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:00:13.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 -- 172.21.15.77:0/3929542698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28070830 msgr2=0x7f8b28070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 --2- 172.21.15.77:0/3929542698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28070830 0x7f8b28070ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c004940 tx=0x7f8b1c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:13.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 -- 172.21.15.77:0/3929542698 shutdown_connections 2024-04-26T05:00:13.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 --2- 172.21.15.77:0/3929542698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28070830 0x7f8b28070ca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 --2- 172.21.15.77:0/3929542698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28071de0 0x7f8b280702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 -- 172.21.15.77:0/3929542698 >> 172.21.15.77:0/3929542698 conn(0x7f8b280f8540 msgr2=0x7f8b280fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:13.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 -- 172.21.15.77:0/3929542698 shutdown_connections 2024-04-26T05:00:13.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.459+0000 7f8b2d051700 1 -- 172.21.15.77:0/3929542698 wait complete. 2024-04-26T05:00:13.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 Processor -- start 2024-04-26T05:00:13.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 -- start start 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28070830 0x7f8b2810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28110650 con 0x7f8b28071de0 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2d051700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281107c0 con 0x7f8b28070830 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57678/0 (socket says 172.21.15.77:57678) 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b2659c700 1 -- 172.21.15.77:0/192783064 learned_addr learned my addr 172.21.15.77:0/192783064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.460+0000 7f8b26d9d700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28070830 0x7f8b2810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2659c700 1 -- 172.21.15.77:0/192783064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28070830 msgr2=0x7f8b2810fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2659c700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28070830 0x7f8b2810fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2659c700 1 -- 172.21.15.77:0/192783064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c005040 con 0x7f8b28071de0 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2659c700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009150 tx=0x7f8b1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c00e040 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1c00bbf0 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b281151c0 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c01c630 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.461+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28115780 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.462+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2804c710 con 0x7f8b28071de0 2024-04-26T05:00:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.462+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8b1c013420 con 0x7f8b28071de0 2024-04-26T05:00:13.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.463+0000 7f8b0f7fe700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 0x7f8b100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.463+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8b1c08edc0 con 0x7f8b28071de0 2024-04-26T05:00:13.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.463+0000 7f8b26d9d700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 0x7f8b100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.463+0000 7f8b26d9d700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 0x7f8b100705f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b28071b20 tx=0x7f8b1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.467+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8b1c05bd10 con 0x7f8b28071de0 2024-04-26T05:00:13.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.687+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b28103eb0 con 0x7f8b1006e140 2024-04-26T05:00:13.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.689+0000 7f8b0f7fe700 1 -- 172.21.15.77:0/192783064 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8b28103eb0 con 0x7f8b1006e140 2024-04-26T05:00:13.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 msgr2=0x7f8b100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 0x7f8b100705f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b28071b20 tx=0x7f8b1800b040 comp rx=0 tx=0).stop 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 msgr2=0x7f8b28110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009150 tx=0x7f8b1c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 shutdown_connections 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8b1006e140 0x7f8b100705f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b28070830 0x7f8b2810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 --2- 172.21.15.77:0/192783064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b28071de0 0x7f8b28110050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 >> 172.21.15.77:0/192783064 conn(0x7f8b280f8540 msgr2=0x7f8b280f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 shutdown_connections 2024-04-26T05:00:13.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.691+0000 7f8b2d051700 1 -- 172.21.15.77:0/192783064 wait complete. 2024-04-26T05:00:13.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- 172.21.15.77:0/2321926573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 msgr2=0x7f3c60070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 --2- 172.21.15.77:0/2321926573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c60070ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3c54004940 tx=0x7f3c54009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:13.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- 172.21.15.77:0/2321926573 shutdown_connections 2024-04-26T05:00:13.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 --2- 172.21.15.77:0/2321926573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c60070ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 --2- 172.21.15.77:0/2321926573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 0x7f3c600702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- 172.21.15.77:0/2321926573 >> 172.21.15.77:0/2321926573 conn(0x7f3c600f8540 msgr2=0x7f3c600fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- 172.21.15.77:0/2321926573 shutdown_connections 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- 172.21.15.77:0/2321926573 wait complete. 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 Processor -- start 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.793+0000 7f3c6642a700 1 -- start start 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 0x7f3c60110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60110650 con 0x7f3c60070830 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601107c0 con 0x7f3c60071de0 2024-04-26T05:00:13.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57692/0 (socket says 172.21.15.77:57692) 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 -- 172.21.15.77:0/81729733 learned_addr learned my addr 172.21.15.77:0/81729733 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5f7fe700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 0x7f3c60110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 -- 172.21.15.77:0/81729733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 msgr2=0x7f3c60110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 0x7f3c60110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 -- 172.21.15.77:0/81729733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54005040 con 0x7f3c60070830 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5ffff700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c8e0 tx=0x7f3c500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c5000a760 con 0x7f3c60070830 2024-04-26T05:00:13.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c5000ad90 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60115220 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c500113a0 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.794+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c601157e0 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.795+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6004c710 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.796+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c50012400 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.796+0000 7f3c5cff9700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 0x7f3c480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.796+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3c5008da50 con 0x7f3c60070830 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.796+0000 7f3c5f7fe700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 0x7f3c480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:13.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.797+0000 7f3c5f7fe700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 0x7f3c480705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400b3b0 tx=0x7f3c54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:13.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:13.801+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3c5005aa20 con 0x7f3c60070830 2024-04-26T05:00:13.953 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:13 smithi077 ceph-mon[91833]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T05:00:14.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.025+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c60103eb0 con 0x7f3c4806e140 2024-04-26T05:00:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.036+0000 7f3c5cff9700 1 -- 172.21.15.77:0/81729733 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3c60103eb0 con 0x7f3c4806e140 2024-04-26T05:00:14.036 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:00:14.036 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (6m) 2m ago 15m 25.2M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (3m) 2m ago 15m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (3m) 24s ago 14m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 unknown 2m ago 15m - - 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 starting - - - - 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (10m) 2m ago 10m 17.3M - 17.2.7-902-gc61472c8 0de407c1b3af fabca956aa5a 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (10m) 24s ago 10m 17.4M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (10m) 24s ago 10m 17.4M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (5m) 2m ago 16m 503M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (5m) 24s ago 14m 452M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (4m) 2m ago 17m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (4m) 24s ago 14m 38.9M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (7m) 2m ago 15m 15.9M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T05:00:14.037 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (6m) 24s ago 14m 16.6M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (3m) 2m ago 13m 86.4M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (2m) 2m ago 13m 83.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (2m) 2m ago 12m 14.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (92s) 24s ago 12m 102M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (60s) 24s ago 12m 84.5M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (27s) 24s ago 11m 11.7M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:00:14.038 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (5m) 2m ago 15m 59.0M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 msgr2=0x7f3c480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 0x7f3c480705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400b3b0 tx=0x7f3c54009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 msgr2=0x7f3c6010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c8e0 tx=0x7f3c500044c0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 shutdown_connections 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3c4806e140 0x7f3c480705f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c60070830 0x7f3c6010fb10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 --2- 172.21.15.77:0/81729733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60071de0 0x7f3c60110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 >> 172.21.15.77:0/81729733 conn(0x7f3c600f8540 msgr2=0x7f3c600f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 shutdown_connections 2024-04-26T05:00:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.038+0000 7f3c6642a700 1 -- 172.21.15.77:0/81729733 wait complete. 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 -- 172.21.15.77:0/3067650710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fe080 msgr2=0x7fc28c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 --2- 172.21.15.77:0/3067650710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fe080 0x7fc28c0fe4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc27c004940 tx=0x7fc27c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 -- 172.21.15.77:0/3067650710 shutdown_connections 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 --2- 172.21.15.77:0/3067650710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fe080 0x7fc28c0fe4f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 --2- 172.21.15.77:0/3067650710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fce80 0x7fc28c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 -- 172.21.15.77:0/3067650710 >> 172.21.15.77:0/3067650710 conn(0x7fc28c0f8590 msgr2=0x7fc28c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 -- 172.21.15.77:0/3067650710 shutdown_connections 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.138+0000 7fc2935e2700 1 -- 172.21.15.77:0/3067650710 wait complete. 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 Processor -- start 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 -- start start 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fe080 0x7fc28c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c070860 con 0x7fc28c0fce80 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc2935e2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0709d0 con 0x7fc28c0fe080 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57710/0 (socket says 172.21.15.77:57710) 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 -- 172.21.15.77:0/1765521874 learned_addr learned my addr 172.21.15.77:0/1765521874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc290b7d700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fe080 0x7fc28c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 -- 172.21.15.77:0/1765521874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fe080 msgr2=0x7fc28c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fe080 0x7fc28c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 -- 172.21.15.77:0/1765521874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c005040 con 0x7fc28c0fce80 2024-04-26T05:00:14.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc29137e700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc288002a50 tx=0x7fc288004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.139+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc288007cc0 con 0x7fc28c0fce80 2024-04-26T05:00:14.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.140+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28800f040 con 0x7fc28c0fce80 2024-04-26T05:00:14.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.140+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c070cb0 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.140+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28800d940 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.140+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c115a80 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.141+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c05f5c0 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.141+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc28800dbe0 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.141+0000 7fc281ffb700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 0x7fc2780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.142+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc28808d7f0 con 0x7fc28c0fce80 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.142+0000 7fc290b7d700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 0x7fc2780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.142+0000 7fc290b7d700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 0x7fc2780705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b970 tx=0x7fc27c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.146+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc288056bf0 con 0x7fc28c0fce80 2024-04-26T05:00:14.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:13 smithi078 ceph-mon[79689]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.438+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc28c1058c0 con 0x7fc28c0fce80 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.438+0000 7fc281ffb700 1 -- 172.21.15.77:0/1765521874 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fc28805a2b0 con 0x7fc28c0fce80 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:14.439 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 11 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:00:14.440 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:00:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 msgr2=0x7fc2780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 0x7fc2780705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b970 tx=0x7fc27c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 msgr2=0x7fc28c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc288002a50 tx=0x7fc288004600 comp rx=0 tx=0).stop 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 shutdown_connections 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc27806e140 0x7fc2780705f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc28c0fce80 0x7fc28c071de0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 --2- 172.21.15.77:0/1765521874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c0fe080 0x7fc28c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.440+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 >> 172.21.15.77:0/1765521874 conn(0x7fc28c0f8590 msgr2=0x7fc28c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.441+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 shutdown_connections 2024-04-26T05:00:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.441+0000 7fc2935e2700 1 -- 172.21.15.77:0/1765521874 wait complete. 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 -- 172.21.15.77:0/994153025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fee20 msgr2=0x7f1a200faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 --2- 172.21.15.77:0/994153025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fee20 0x7f1a200faf60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a10004940 tx=0x7f1a10009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 -- 172.21.15.77:0/994153025 shutdown_connections 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 --2- 172.21.15.77:0/994153025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fee20 0x7f1a200faf60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 --2- 172.21.15.77:0/994153025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fe4d0 0x7f1a200fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 -- 172.21.15.77:0/994153025 >> 172.21.15.77:0/994153025 conn(0x7f1a200f6bd0 msgr2=0x7f1a200f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 -- 172.21.15.77:0/994153025 shutdown_connections 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 -- 172.21.15.77:0/994153025 wait complete. 2024-04-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.541+0000 7f1a263e1700 1 Processor -- start 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 -- start start 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fee20 0x7f1a2010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2010e4c0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20112e60 con 0x7f1a200fee20 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57720/0 (socket says 172.21.15.77:57720) 2024-04-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 -- 172.21.15.77:0/2550818478 learned_addr learned my addr 172.21.15.77:0/2550818478 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1f7fe700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fee20 0x7f1a2010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 -- 172.21.15.77:0/2550818478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fee20 msgr2=0x7f1a2010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fee20 0x7f1a2010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 -- 172.21.15.77:0/2550818478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10005040 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1ffff700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b790 tx=0x7f1a14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1400f6f0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a201130c0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1400fd20 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a140103a0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.542+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a20113630 con 0x7f1a200fe4d0 2024-04-26T05:00:14.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.543+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2004c710 con 0x7f1a200fe4d0 2024-04-26T05:00:14.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.544+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1a14011400 con 0x7f1a200fe4d0 2024-04-26T05:00:14.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.544+0000 7f1a1cff9700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 0x7f1a080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.544+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1a1408d910 con 0x7f1a200fe4d0 2024-04-26T05:00:14.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.545+0000 7f1a1f7fe700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 0x7f1a080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.545+0000 7f1a1f7fe700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 0x7f1a080705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000b3b0 tx=0x7f1a10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.549+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1a1405a840 con 0x7f1a200fe4d0 2024-04-26T05:00:14.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.796+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a20113ae0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.797+0000 7f1a1cff9700 1 -- 172.21.15.77:0/2550818478 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1889 (secure 0 0 0) 0x7f1a1405a3d0 con 0x7f1a200fe4d0 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:e19 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 19 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:00:14.798 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:00:11.414274+0000 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 81 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:00:14.799 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=14470} 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 14470 members: 14470 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{0:14470} state up:active seq 155 join_fscid=1 addr [v2:172.21.15.77:6828/2377141710,v1:172.21.15.77:6829/2377141710] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:00:14.800 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:14.801 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:24261} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T05:00:14.801 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T05:00:14.801 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 msgr2=0x7f1a080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 0x7f1a080705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000b3b0 tx=0x7f1a10009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 msgr2=0x7f1a2010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b790 tx=0x7f1a14009250 comp rx=0 tx=0).stop 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 shutdown_connections 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1a0806e140 0x7f1a080705f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a200fe4d0 0x7f1a2010d8f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 --2- 172.21.15.77:0/2550818478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a200fee20 0x7f1a2010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 >> 172.21.15.77:0/2550818478 conn(0x7f1a200f6bd0 msgr2=0x7f1a200f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 shutdown_connections 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.799+0000 7f1a263e1700 1 -- 172.21.15.77:0/2550818478 wait complete. 2024-04-26T05:00:14.802 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 19 2024-04-26T05:00:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.903+0000 7feaeaa3a700 1 -- 172.21.15.77:0/3769656135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fe080 msgr2=0x7feae40fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.903+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/3769656135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fe080 0x7feae40fe4f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feacc004940 tx=0x7feacc009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- 172.21.15.77:0/3769656135 shutdown_connections 2024-04-26T05:00:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/3769656135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fe080 0x7feae40fe4f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/3769656135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fce80 0x7feae40fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- 172.21.15.77:0/3769656135 >> 172.21.15.77:0/3769656135 conn(0x7feae40f8590 msgr2=0x7feae40fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- 172.21.15.77:0/3769656135 shutdown_connections 2024-04-26T05:00:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- 172.21.15.77:0/3769656135 wait complete. 2024-04-26T05:00:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 Processor -- start 2024-04-26T05:00:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- start start 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 0x7feae410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4110640 con 0x7feae40fce80 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.904+0000 7feaeaa3a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4110de0 con 0x7feae40fe080 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57734/0 (socket says 172.21.15.77:57734) 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 -- 172.21.15.77:0/2398908540 learned_addr learned my addr 172.21.15.77:0/2398908540 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feae3fff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 0x7feae410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 -- 172.21.15.77:0/2398908540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 msgr2=0x7feae410ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 0x7feae410ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 -- 172.21.15.77:0/2398908540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc005040 con 0x7feae40fce80 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feae3fff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 0x7feae410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:00:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feadbfff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fead400b790 tx=0x7fead4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead400f6f0 con 0x7feae40fce80 2024-04-26T05:00:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fead400fd20 con 0x7feae40fce80 2024-04-26T05:00:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41110c0 con 0x7feae40fce80 2024-04-26T05:00:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead40103a0 con 0x7feae40fce80 2024-04-26T05:00:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.905+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae4111600 con 0x7feae40fce80 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.906+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4101a10 con 0x7feae40fce80 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.907+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fead4011400 con 0x7feae40fce80 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.907+0000 7feae17fa700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 0x7fead00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.907+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fead408d9b0 con 0x7feae40fce80 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.907+0000 7feae3fff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 0x7fead00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.907+0000 7feae3fff700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 0x7fead00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b970 tx=0x7feacc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:14.912+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fead405a860 con 0x7feae40fce80 2024-04-26T05:00:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:14 smithi077 ceph-mon[91833]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:14 smithi077 ceph-mon[91833]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:14 smithi077 ceph-mon[91833]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:14 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1765521874' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:14.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:14 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2550818478' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:00:15.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.149+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feae4105ab0 con 0x7fead006e140 2024-04-26T05:00:15.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.150+0000 7feae17fa700 1 -- 172.21.15.77:0/2398908540 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feae4105ab0 con 0x7fead006e140 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "osd", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "12/21 daemons upgraded", 2024-04-26T05:00:15.151 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mds daemons", 2024-04-26T05:00:15.152 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:00:15.152 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:00:15.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 msgr2=0x7fead00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 0x7fead00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b970 tx=0x7feacc009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 msgr2=0x7feae410fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fead400b790 tx=0x7fead4009220 comp rx=0 tx=0).stop 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 shutdown_connections 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fead006e140 0x7fead00705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae40fce80 0x7feae410fa70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 --2- 172.21.15.77:0/2398908540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feae40fe080 0x7feae410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 >> 172.21.15.77:0/2398908540 conn(0x7feae40f8590 msgr2=0x7feae40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 shutdown_connections 2024-04-26T05:00:15.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.153+0000 7feaeaa3a700 1 -- 172.21.15.77:0/2398908540 wait complete. 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 -- 172.21.15.77:0/1746451055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fed30 msgr2=0x7fac500fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 --2- 172.21.15.77:0/1746451055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fed30 0x7fac500fae70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac44004940 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 -- 172.21.15.77:0/1746451055 shutdown_connections 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 --2- 172.21.15.77:0/1746451055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fed30 0x7fac500fae70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 --2- 172.21.15.77:0/1746451055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fe3e0 0x7fac500fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 -- 172.21.15.77:0/1746451055 >> 172.21.15.77:0/1746451055 conn(0x7fac500f6b80 msgr2=0x7fac500f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:15.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.266+0000 7fac549e2700 1 -- 172.21.15.77:0/1746451055 shutdown_connections 2024-04-26T05:00:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:14 smithi078 ceph-mon[79689]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:14 smithi078 ceph-mon[79689]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:14 smithi078 ceph-mon[79689]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:14 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1765521874' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:14 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2550818478' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.267+0000 7fac549e2700 1 -- 172.21.15.77:0/1746451055 wait complete. 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.267+0000 7fac549e2700 1 Processor -- start 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.267+0000 7fac549e2700 1 -- start start 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fed30 0x7fac5010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:15.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5010e4a0 con 0x7fac500fe3e0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5010ecf0 con 0x7fac500fed30 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57754/0 (socket says 172.21.15.77:57754) 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 -- 172.21.15.77:0/1240169162 learned_addr learned my addr 172.21.15.77:0/1240169162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4dd9b700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fed30 0x7fac5010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 -- 172.21.15.77:0/1240169162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fed30 msgr2=0x7fac5010dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fed30 0x7fac5010dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 -- 172.21.15.77:0/1240169162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44005040 con 0x7fac500fe3e0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac4e59c700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fac40002a80 tx=0x7fac400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac40007cc0 con 0x7fac500fe3e0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5010efd0 con 0x7fac500fe3e0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac4000f040 con 0x7fac500fe3e0 2024-04-26T05:00:15.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac400096f0 con 0x7fac500fe3e0 2024-04-26T05:00:15.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.268+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5010f510 con 0x7fac500fe3e0 2024-04-26T05:00:15.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.270+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5005f5c0 con 0x7fac500fe3e0 2024-04-26T05:00:15.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.270+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fac4001c400 con 0x7fac500fe3e0 2024-04-26T05:00:15.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.270+0000 7fac3effd700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 0x7fac380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:15.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.270+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fac4008d2f0 con 0x7fac500fe3e0 2024-04-26T05:00:15.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.270+0000 7fac4dd9b700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 0x7fac380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:15.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.271+0000 7fac4dd9b700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 0x7fac380705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b3b0 tx=0x7fac44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:15.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.275+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fac4005a210 con 0x7fac500fe3e0 2024-04-26T05:00:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.571+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fac5010f890 con 0x7fac500fe3e0 2024-04-26T05:00:15.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.571+0000 7fac3effd700 1 -- 172.21.15.77:0/1240169162 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+347 (secure 0 0 0) 0x7fac40059da0 con 0x7fac500fe3e0 2024-04-26T05:00:15.572 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:15.572 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:00:15.572 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in unknown state 2024-04-26T05:00:15.573 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:15.573 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:00:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 msgr2=0x7fac380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 0x7fac380705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b3b0 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:15.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 msgr2=0x7fac5010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fac40002a80 tx=0x7fac400046b0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 shutdown_connections 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fac3806e140 0x7fac380705f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac500fe3e0 0x7fac5010d960 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 --2- 172.21.15.77:0/1240169162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac500fed30 0x7fac5010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 >> 172.21.15.77:0/1240169162 conn(0x7fac500f6b80 msgr2=0x7fac500f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 shutdown_connections 2024-04-26T05:00:15.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:15.574+0000 7fac549e2700 1 -- 172.21.15.77:0/1240169162 wait complete. 2024-04-26T05:00:15.846 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:15 smithi077 ceph-mon[91833]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T05:00:15.846 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:15 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1240169162' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:00:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:15 smithi078 ceph-mon[79689]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T05:00:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:15 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1240169162' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:00:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:16 smithi077 ceph-mon[91833]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:16 smithi078 ceph-mon[79689]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:18 smithi078 ceph-mon[79689]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T05:00:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:18 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:18.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:18 smithi077 ceph-mon[91833]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T05:00:18.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:18.364 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:18 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:19.365 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:19.365 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:19 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:19.365 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:19 smithi078 ceph-mon[79689]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 7.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-26T05:00:19.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:19.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:19 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:19.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:19 smithi077 ceph-mon[91833]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 7.5 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-26T05:00:21.524 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:21 smithi077 ceph-mon[91833]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-26T05:00:21.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:21 smithi078 ceph-mon[79689]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: Detected new or changed devices on smithi077 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:22.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi077.urxtgp"]}]: dispatch 2024-04-26T05:00:22.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:22 smithi077 ceph-mon[91833]: Upgrade: It appears safe to stop mds.cephfs.smithi077.urxtgp 2024-04-26T05:00:23.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: Detected new or changed devices on smithi077 2024-04-26T05:00:23.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:23.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:23.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:23.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:00:23.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:23.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:23.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:23.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi077.urxtgp"]}]: dispatch 2024-04-26T05:00:23.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:22 smithi078 ceph-mon[79689]: Upgrade: It appears safe to stop mds.cephfs.smithi077.urxtgp 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: Upgrade: Updating mds.cephfs.smithi077.urxtgp 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: Deploying daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T05:00:24.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:24 smithi077 ceph-mon[91833]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 653 B/s rd, 5.6 KiB/s wr, 3 op/s 2024-04-26T05:00:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: Upgrade: Updating mds.cephfs.smithi077.urxtgp 2024-04-26T05:00:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi077.urxtgp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:00:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: Deploying daemon mds.cephfs.smithi077.urxtgp on smithi077 2024-04-26T05:00:24.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:24 smithi078 ceph-mon[79689]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 653 B/s rd, 5.6 KiB/s wr, 3 op/s 2024-04-26T05:00:25.620 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T05:00:25.564+0000 7fe623188700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: osdmap e82: 6 total, 6 up, 6 in 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: Standby daemon mds.cephfs.smithi078.wkrgkp assigned to filesystem cephfs as rank 0 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:25.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:25 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:replay} 2 up:standby 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: osdmap e82: 6 total, 6 up, 6 in 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: Standby daemon mds.cephfs.smithi078.wkrgkp assigned to filesystem cephfs as rank 0 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:26.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:25 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:replay} 2 up:standby 2024-04-26T05:00:26.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:26 smithi077 ceph-mon[91833]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:00:27.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:26 smithi078 ceph-mon[79689]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:00:27.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:00:28.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:00:28.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:28 smithi077 ceph-mon[91833]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 1 op/s 2024-04-26T05:00:29.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:28 smithi078 ceph-mon[79689]: pgmap v197: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 1 op/s 2024-04-26T05:00:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:30 smithi077 ceph-mon[91833]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.7 MiB/s rd, 3 op/s 2024-04-26T05:00:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:30 smithi078 ceph-mon[79689]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.7 MiB/s rd, 3 op/s 2024-04-26T05:00:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:32.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:32 smithi077 ceph-mon[91833]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-26T05:00:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:32 smithi078 ceph-mon[79689]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-26T05:00:35.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:34 smithi077 ceph-mon[91833]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:00:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:34 smithi078 ceph-mon[79689]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:00:36.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:35 smithi077 ceph-mon[91833]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.000999985 2024-04-26T05:00:36.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:35 smithi077 ceph-mon[91833]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.000999985 2024-04-26T05:00:36.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:35 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:reconnect 2024-04-26T05:00:36.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:35 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:reconnect} 2 up:standby 2024-04-26T05:00:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:35 smithi078 ceph-mon[79689]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.000999985 2024-04-26T05:00:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:35 smithi078 ceph-mon[79689]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.000999985 2024-04-26T05:00:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:35 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:reconnect 2024-04-26T05:00:36.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:35 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:reconnect} 2 up:standby 2024-04-26T05:00:36.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:36 smithi077 ceph-mon[91833]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-26T05:00:36.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:36 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:rejoin 2024-04-26T05:00:36.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:36 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:rejoin} 2 up:standby 2024-04-26T05:00:36.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:36 smithi077 ceph-mon[91833]: daemon mds.cephfs.smithi078.wkrgkp is now active in filesystem cephfs as rank 0 2024-04-26T05:00:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:36 smithi078 ceph-mon[79689]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-26T05:00:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:36 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:rejoin 2024-04-26T05:00:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:36 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.wkrgkp=up:rejoin} 2 up:standby 2024-04-26T05:00:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:36 smithi078 ceph-mon[79689]: daemon mds.cephfs.smithi078.wkrgkp is now active in filesystem cephfs as rank 0 2024-04-26T05:00:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:00:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:active 2024-04-26T05:00:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi078.wkrgkp=up:active} 2 up:standby 2024-04-26T05:00:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:00:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] up:active 2024-04-26T05:00:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi078.wkrgkp=up:active} 2 up:standby 2024-04-26T05:00:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:38.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:38 smithi077 ceph-mon[91833]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:00:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:38 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] up:boot 2024-04-26T05:00:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:38 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi078.wkrgkp=up:active} 3 up:standby 2024-04-26T05:00:39.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:00:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:38 smithi078 ceph-mon[79689]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:00:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:38 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] up:boot 2024-04-26T05:00:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:38 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi078.wkrgkp=up:active} 3 up:standby 2024-04-26T05:00:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:00:41.143 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:40 smithi077 ceph-mon[91833]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T05:00:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:40 smithi078 ceph-mon[79689]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T05:00:42.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:42.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:42.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:42 smithi077 ceph-mon[91833]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:00:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:42 smithi078 ceph-mon[79689]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:00:43.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:43.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:44.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:44.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:44 smithi077 ceph-mon[91833]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T05:00:45.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:44 smithi078 ceph-mon[79689]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T05:00:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 -- 172.21.15.77:0/1290341286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fcf60 msgr2=0x7f2fe00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:45.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 --2- 172.21.15.77:0/1290341286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fcf60 0x7f2fe00fd3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4004940 tx=0x7f2fd4009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 -- 172.21.15.77:0/1290341286 shutdown_connections 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 --2- 172.21.15.77:0/1290341286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fcf60 0x7f2fe00fd3d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 --2- 172.21.15.77:0/1290341286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 -- 172.21.15.77:0/1290341286 >> 172.21.15.77:0/1290341286 conn(0x7f2fe00f7430 msgr2=0x7f2fe00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 -- 172.21.15.77:0/1290341286 shutdown_connections 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.680+0000 7f2fe515b700 1 -- 172.21.15.77:0/1290341286 wait complete. 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 Processor -- start 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 -- start start 2024-04-26T05:00:45.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fcf60 0x7f2fe010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe010d440 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fe515b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe010dc90 con 0x7f2fe00fcf60 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fded9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fded9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47930/0 (socket says 172.21.15.77:47930) 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.681+0000 7f2fded9d700 1 -- 172.21.15.77:0/4043212542 learned_addr learned my addr 172.21.15.77:0/4043212542 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fded9d700 1 -- 172.21.15.77:0/4043212542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fcf60 msgr2=0x7f2fe010ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fde59c700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fcf60 0x7f2fe010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fded9d700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fcf60 0x7f2fe010ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fded9d700 1 -- 172.21.15.77:0/4043212542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4005040 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fded9d700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c8c0 tx=0x7f2fd00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:45.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd000a980 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe010df70 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe010e4b0 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fd000ce60 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.682+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd00115e0 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.683+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe005f5c0 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.684+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2fd000aae0 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.684+0000 7f2fc77fe700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 0x7f2fc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.684+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2fd008dc50 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.684+0000 7f2fde59c700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 0x7f2fc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:45.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.685+0000 7f2fde59c700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 0x7f2fc80705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4000c00 tx=0x7f2fd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:45.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.689+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2fd005ab30 con 0x7f2fe00fbcb0 2024-04-26T05:00:45.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.915+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe010e850 con 0x7f2fc806e140 2024-04-26T05:00:45.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.916+0000 7f2fc77fe700 1 -- 172.21.15.77:0/4043212542 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2fe010e850 con 0x7f2fc806e140 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 msgr2=0x7f2fc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 0x7f2fc80705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4000c00 tx=0x7f2fd4009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 msgr2=0x7f2fe010c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c8c0 tx=0x7f2fd00044c0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 shutdown_connections 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2fc806e140 0x7f2fc80705f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fe00fbcb0 0x7f2fe010c900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 --2- 172.21.15.77:0/4043212542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe00fcf60 0x7f2fe010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 >> 172.21.15.77:0/4043212542 conn(0x7f2fe00f7430 msgr2=0x7f2fe00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 shutdown_connections 2024-04-26T05:00:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:45.918+0000 7f2fe515b700 1 -- 172.21.15.77:0/4043212542 wait complete. 2024-04-26T05:00:45.930 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:00:46.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 -- 172.21.15.77:0/2338823633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fe080 msgr2=0x7ff5500fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 --2- 172.21.15.77:0/2338823633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fe080 0x7ff5500fe4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff544004940 tx=0x7ff544009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:46.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 -- 172.21.15.77:0/2338823633 shutdown_connections 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 --2- 172.21.15.77:0/2338823633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fe080 0x7ff5500fe4f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 --2- 172.21.15.77:0/2338823633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fcf10 0x7ff5500fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 -- 172.21.15.77:0/2338823633 >> 172.21.15.77:0/2338823633 conn(0x7ff5500f8560 msgr2=0x7ff5500fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.025+0000 7ff5571eb700 1 -- 172.21.15.77:0/2338823633 shutdown_connections 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 -- 172.21.15.77:0/2338823633 wait complete. 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 Processor -- start 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 -- start start 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fe080 0x7ff5500702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff550070860 con 0x7ff5500fcf10 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff5571eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5500709d0 con 0x7ff5500fe080 2024-04-26T05:00:46.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.026+0000 7ff554f87700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47946/0 (socket says 172.21.15.77:47946) 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 -- 172.21.15.77:0/1183477627 learned_addr learned my addr 172.21.15.77:0/1183477627 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff54ffff700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fe080 0x7ff5500702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 -- 172.21.15.77:0/1183477627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fe080 msgr2=0x7ff5500702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fe080 0x7ff5500702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 -- 172.21.15.77:0/1183477627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544005040 con 0x7ff5500fcf10 2024-04-26T05:00:46.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff554f87700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff548001d80 tx=0x7ff548009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff548007d10 con 0x7ff5500fcf10 2024-04-26T05:00:46.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff548001e80 con 0x7ff5500fcf10 2024-04-26T05:00:46.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff550070cb0 con 0x7ff5500fcf10 2024-04-26T05:00:46.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff548014640 con 0x7ff5500fcf10 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.027+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff550115a10 con 0x7ff5500fcf10 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.028+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55005f5c0 con 0x7ff5500fcf10 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.029+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff54800e960 con 0x7ff5500fcf10 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.029+0000 7ff54d7fa700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.029+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff54808d8a0 con 0x7ff5500fcf10 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.029+0000 7ff54ffff700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.030+0000 7ff54ffff700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 0x7ff5380705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff54400b970 tx=0x7ff544009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.038+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff548056c80 con 0x7ff5500fcf10 2024-04-26T05:00:46.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.272+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff550102490 con 0x7ff53806e140 2024-04-26T05:00:46.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.273+0000 7ff54d7fa700 1 -- 172.21.15.77:0/1183477627 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff550102490 con 0x7ff53806e140 2024-04-26T05:00:46.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 msgr2=0x7ff5380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 0x7ff5380705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff54400b970 tx=0x7ff544009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:46.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 msgr2=0x7ff550071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff548001d80 tx=0x7ff548009ec0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 shutdown_connections 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff53806e140 0x7ff5380705f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5500fcf10 0x7ff550071de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 --2- 172.21.15.77:0/1183477627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5500fe080 0x7ff5500702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 >> 172.21.15.77:0/1183477627 conn(0x7ff5500f8560 msgr2=0x7ff5500f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 shutdown_connections 2024-04-26T05:00:46.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.275+0000 7ff5571eb700 1 -- 172.21.15.77:0/1183477627 wait complete. 2024-04-26T05:00:46.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 -- 172.21.15.77:0/3098312182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fe080 msgr2=0x7fa9e00fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/3098312182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fe080 0x7fa9e00fe4f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0004940 tx=0x7fa9d0009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:46.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 -- 172.21.15.77:0/3098312182 shutdown_connections 2024-04-26T05:00:46.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/3098312182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fe080 0x7fa9e00fe4f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/3098312182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fce80 0x7fa9e00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.393+0000 7fa9e65a0700 1 -- 172.21.15.77:0/3098312182 >> 172.21.15.77:0/3098312182 conn(0x7fa9e00f8590 msgr2=0x7fa9e00fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:46.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.394+0000 7fa9e65a0700 1 -- 172.21.15.77:0/3098312182 shutdown_connections 2024-04-26T05:00:46.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.394+0000 7fa9e65a0700 1 -- 172.21.15.77:0/3098312182 wait complete. 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.394+0000 7fa9e65a0700 1 Processor -- start 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9e65a0700 1 -- start start 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9e65a0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9e65a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fe080 0x7fa9e010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9e65a0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e010e570 con 0x7fa9e00fce80 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9e65a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e010edc0 con 0x7fa9e00fe080 2024-04-26T05:00:46.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47958/0 (socket says 172.21.15.77:47958) 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 -- 172.21.15.77:0/588604789 learned_addr learned my addr 172.21.15.77:0/588604789 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9df7fe700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fe080 0x7fa9e010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 -- 172.21.15.77:0/588604789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fe080 msgr2=0x7fa9e010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fe080 0x7fa9e010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.395+0000 7fa9dffff700 1 -- 172.21.15.77:0/588604789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0005040 con 0x7fa9e00fce80 2024-04-26T05:00:46.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.396+0000 7fa9dffff700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4002a80 tx=0x7fa9d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.396+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d4007cc0 con 0x7fa9e00fce80 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.396+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9d400f040 con 0x7fa9e00fce80 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.396+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e010f0a0 con 0x7fa9e00fce80 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.396+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e010f5e0 con 0x7fa9e00fce80 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.397+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d400da30 con 0x7fa9e00fce80 2024-04-26T05:00:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.397+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e005f5c0 con 0x7fa9e00fce80 2024-04-26T05:00:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.398+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa9d400db90 con 0x7fa9e00fce80 2024-04-26T05:00:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.398+0000 7fa9dcff9700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 0x7fa9c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.398+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa9d408d850 con 0x7fa9e00fce80 2024-04-26T05:00:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.398+0000 7fa9df7fe700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 0x7fa9c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.399+0000 7fa9df7fe700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 0x7fa9c80705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b970 tx=0x7fa9d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.403+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa9d405a730 con 0x7fa9e00fce80 2024-04-26T05:00:46.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.618+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e010f9c0 con 0x7fa9c806e140 2024-04-26T05:00:46.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.628+0000 7fa9dcff9700 1 -- 172.21.15.77:0/588604789 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa9e010f9c0 con 0x7fa9c806e140 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (7m) 4s ago 16m 24.8M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (4m) 4s ago 16m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (4m) 56s ago 15m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 error 4s ago 15m - - 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (39s) 4s ago 10m 12.7M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (9s) 4s ago 10m 13.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (10m) 56s ago 10m 17.4M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (10m) 56s ago 10m 17.4M - 17.2.7-902-gc61472c8 0de407c1b3af 2cd4901ef58f 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (6m) 4s ago 17m 507M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (5m) 56s ago 15m 452M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:00:46.630 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (5m) 4s ago 17m 51.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (5m) 56s ago 15m 38.9M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (7m) 4s ago 16m 16.2M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (6m) 56s ago 15m 16.6M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (3m) 4s ago 14m 95.3M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (3m) 4s ago 14m 96.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (2m) 4s ago 13m 94.1M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (2m) 56s ago 13m 102M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (92s) 56s ago 12m 84.5M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (60s) 56s ago 12m 11.7M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:00:46.631 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (5m) 4s ago 15m 61.1M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 msgr2=0x7fa9c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 0x7fa9c80705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b970 tx=0x7fa9d0009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 msgr2=0x7fa9e010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4002a80 tx=0x7fa9d4004600 comp rx=0 tx=0).stop 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 shutdown_connections 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa9c806e140 0x7fa9c80705f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e00fce80 0x7fa9e010da30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 --2- 172.21.15.77:0/588604789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e00fe080 0x7fa9e010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 >> 172.21.15.77:0/588604789 conn(0x7fa9e00f8590 msgr2=0x7fa9e00f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:46.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 shutdown_connections 2024-04-26T05:00:46.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.630+0000 7fa9e65a0700 1 -- 172.21.15.77:0/588604789 wait complete. 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 -- 172.21.15.77:0/791616245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740ffee0 msgr2=0x7f4774102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 --2- 172.21.15.77:0/791616245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740ffee0 0x7f4774102300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4764004940 tx=0x7f4764009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 -- 172.21.15.77:0/791616245 shutdown_connections 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 --2- 172.21.15.77:0/791616245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740ffee0 0x7f4774102300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 --2- 172.21.15.77:0/791616245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740662c0 0x7f47740ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 -- 172.21.15.77:0/791616245 >> 172.21.15.77:0/791616245 conn(0x7f47740f7430 msgr2=0x7f47740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:46.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 -- 172.21.15.77:0/791616245 shutdown_connections 2024-04-26T05:00:46.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.731+0000 7f477bc53700 1 -- 172.21.15.77:0/791616245 wait complete. 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 Processor -- start 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 -- start start 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 0x7f477410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477410e5a0 con 0x7f47740662c0 2024-04-26T05:00:46.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f477bc53700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477410ed50 con 0x7f47740ffee0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f47799ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f47799ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47970/0 (socket says 172.21.15.77:47970) 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.732+0000 7f47799ef700 1 -- 172.21.15.77:0/75544556 learned_addr learned my addr 172.21.15.77:0/75544556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47791ee700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 0x7f477410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47791ee700 1 -- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 msgr2=0x7f477410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47791ee700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47791ee700 1 -- 172.21.15.77:0/75544556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764005040 con 0x7f47740ffee0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47799ef700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f47791ee700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 0x7f477410dfa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4764000c00 tx=0x7f4764009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476400e030 con 0x7f47740ffee0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477410efd0 con 0x7f47740ffee0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f476401c400 con 0x7f47740ffee0 2024-04-26T05:00:46.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476400b650 con 0x7f47740ffee0 2024-04-26T05:00:46.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.733+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477410f510 con 0x7f47740ffee0 2024-04-26T05:00:46.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.735+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477405f5c0 con 0x7f47740ffee0 2024-04-26T05:00:46.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.735+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f476401ca90 con 0x7f47740ffee0 2024-04-26T05:00:46.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.736+0000 7f476a7fc700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 0x7f47600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:46.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.736+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f476408f3b0 con 0x7f47740ffee0 2024-04-26T05:00:46.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.736+0000 7f47799ef700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 0x7f47600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:46.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.736+0000 7f47799ef700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 0x7f47600705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4770002f40 tx=0x7f477000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:46.740+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f476405c350 con 0x7f47740ffee0 2024-04-26T05:00:46.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:46.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:46.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:46.994 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi078.wkrgkp"]}]: dispatch 2024-04-26T05:00:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.028+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f477410f7f0 con 0x7f47740ffee0 2024-04-26T05:00:47.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.028+0000 7f476a7fc700 1 -- 172.21.15.77:0/75544556 <== mon.1 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f476405bee0 con 0x7f47740ffee0 2024-04-26T05:00:47.030 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:00:47.030 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-26T05:00:47.031 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 12 2024-04-26T05:00:47.032 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:00:47.032 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 msgr2=0x7f47600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 0x7f47600705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4770002f40 tx=0x7f477000b040 comp rx=0 tx=0).stop 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 msgr2=0x7f477410dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 0x7f477410dfa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4764000c00 tx=0x7f4764009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 shutdown_connections 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f476006e140 0x7f47600705f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47740662c0 0x7f477410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 --2- 172.21.15.77:0/75544556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47740ffee0 0x7f477410dfa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 >> 172.21.15.77:0/75544556 conn(0x7f47740f7430 msgr2=0x7f47740f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 shutdown_connections 2024-04-26T05:00:47.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.031+0000 7f477bc53700 1 -- 172.21.15.77:0/75544556 wait complete. 2024-04-26T05:00:47.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 -- 172.21.15.77:0/1293265036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58070890 msgr2=0x7f9f58070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1293265036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58070890 0x7f9f58070d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004940 tx=0x7f9f4c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:47.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 -- 172.21.15.77:0/1293265036 shutdown_connections 2024-04-26T05:00:47.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1293265036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58070890 0x7f9f58070d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1293265036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58071e40 0x7f9f58070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 -- 172.21.15.77:0/1293265036 >> 172.21.15.77:0/1293265036 conn(0x7f9f580f8590 msgr2=0x7f9f580fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 -- 172.21.15.77:0/1293265036 shutdown_connections 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 -- 172.21.15.77:0/1293265036 wait complete. 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.131+0000 7f9f5defa700 1 Processor -- start 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f5defa700 1 -- start start 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f5defa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58070890 0x7f9f5810fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f5defa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f5defa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58110850 con 0x7f9f58071e40 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f5defa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58110ff0 con 0x7f9f58070890 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48000/0 (socket says 172.21.15.77:48000) 2024-04-26T05:00:47.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 -- 172.21.15.77:0/1339190047 learned_addr learned my addr 172.21.15.77:0/1339190047 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f577fe700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58070890 0x7f9f5810fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 -- 172.21.15.77:0/1339190047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58070890 msgr2=0x7f9f5810fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58070890 0x7f9f5810fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 -- 172.21.15.77:0/1339190047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c005040 con 0x7f9f58071e40 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f56ffd700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009150 tx=0x7f9f4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c00e040 con 0x7f9f58071e40 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.132+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4c004290 con 0x7f9f58071e40 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.133+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c01c630 con 0x7f9f58071e40 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.133+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f58111270 con 0x7f9f58071e40 2024-04-26T05:00:47.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.133+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f58111830 con 0x7f9f58071e40 2024-04-26T05:00:47.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.133+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5805f5c0 con 0x7f9f58071e40 2024-04-26T05:00:47.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.134+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9f4c00bbf0 con 0x7f9f58071e40 2024-04-26T05:00:47.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.134+0000 7f9f3ffff700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 0x7f9f400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.135+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9f4c08e620 con 0x7f9f58071e40 2024-04-26T05:00:47.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.135+0000 7f9f577fe700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 0x7f9f400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.135+0000 7f9f577fe700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 0x7f9f400705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f58071b80 tx=0x7f9f480091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.139+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9f4c05b5c0 con 0x7f9f58071e40 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:47.224 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:47.225 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi078.wkrgkp"]}]: dispatch 2024-04-26T05:00:47.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.378+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f5804bed0 con 0x7f9f58071e40 2024-04-26T05:00:47.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.378+0000 7f9f3ffff700 1 -- 172.21.15.77:0/1339190047 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1890 (secure 0 0 0) 0x7f9f4c05b150 con 0x7f9f58071e40 2024-04-26T05:00:47.380 INFO:teuthology.orchestra.run.smithi077.stdout:e25 2024-04-26T05:00:47.380 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:00:47.380 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:00:47.380 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 24 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:00:36.954370+0000 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:00:47.381 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 82 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=24261} 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:00:47.382 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 24261 members: 24261 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{0:24261} state up:active seq 162 join_fscid=1 addr [v2:172.21.15.78:6824/3670372668,v1:172.21.15.78:6825/3670372668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:00:47.383 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:00:47.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 msgr2=0x7f9f400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 0x7f9f400705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f58071b80 tx=0x7f9f480091a0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 msgr2=0x7f9f581101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009150 tx=0x7f9f4c009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:47.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 shutdown_connections 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9f4006e140 0x7f9f400705f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f58070890 0x7f9f5810fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 --2- 172.21.15.77:0/1339190047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f58071e40 0x7f9f581101c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.381+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 >> 172.21.15.77:0/1339190047 conn(0x7f9f580f8590 msgr2=0x7f9f580f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.382+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 shutdown_connections 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.382+0000 7f9f5defa700 1 -- 172.21.15.77:0/1339190047 wait complete. 2024-04-26T05:00:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 25 2024-04-26T05:00:47.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- 172.21.15.77:0/1901612684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c08066f30 msgr2=0x7f6c080fd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 --2- 172.21.15.77:0/1901612684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c08066f30 0x7f6c080fd7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8004940 tx=0x7f6bf8009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:47.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- 172.21.15.77:0/1901612684 shutdown_connections 2024-04-26T05:00:47.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 --2- 172.21.15.77:0/1901612684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c080fdd10 0x7f6c08100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 --2- 172.21.15.77:0/1901612684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c08066f30 0x7f6c080fd7d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- 172.21.15.77:0/1901612684 >> 172.21.15.77:0/1901612684 conn(0x7f6c080f7450 msgr2=0x7f6c080f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- 172.21.15.77:0/1901612684 shutdown_connections 2024-04-26T05:00:47.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- 172.21.15.77:0/1901612684 wait complete. 2024-04-26T05:00:47.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 Processor -- start 2024-04-26T05:00:47.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- start start 2024-04-26T05:00:47.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c08066f30 0x7f6c08113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08114a70 con 0x7f6c080fdd10 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.484+0000 7f6c0d786700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08114be0 con 0x7f6c08066f30 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45844/0 (socket says 172.21.15.77:45844) 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 -- 172.21.15.77:0/3258089053 learned_addr learned my addr 172.21.15.77:0/3258089053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c06ffd700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c08066f30 0x7f6c08113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 -- 172.21.15.77:0/3258089053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c08066f30 msgr2=0x7f6c08113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c08066f30 0x7f6c08113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 -- 172.21.15.77:0/3258089053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8005040 con 0x7f6c080fdd10 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c067fc700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00eb40 tx=0x7f6bfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bfc007cc0 con 0x7f6c080fdd10 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c08119590 con 0x7f6c080fdd10 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bfc00f040 con 0x7f6c080fdd10 2024-04-26T05:00:47.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bfc00ca10 con 0x7f6c080fdd10 2024-04-26T05:00:47.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.485+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c08119ad0 con 0x7f6c080fdd10 2024-04-26T05:00:47.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.486+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0804c710 con 0x7f6c080fdd10 2024-04-26T05:00:47.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.487+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6bfc00cb70 con 0x7f6c080fdd10 2024-04-26T05:00:47.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.487+0000 7f6bef7fe700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 0x7f6bf00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.487+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6bfc08d6a0 con 0x7f6c080fdd10 2024-04-26T05:00:47.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.487+0000 7f6c06ffd700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 0x7f6bf00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.488+0000 7f6c06ffd700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 0x7f6bf00705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8004890 tx=0x7f6bf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.492+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6bfc05a630 con 0x7f6c080fdd10 2024-04-26T05:00:47.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.708+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c080f8f90 con 0x7f6bf006e140 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.709+0000 7f6bef7fe700 1 -- 172.21.15.77:0/3258089053 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6c080f8f90 con 0x7f6bf006e140 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T05:00:47.711 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "osd", 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "14/21 daemons upgraded", 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mds daemons", 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:00:47.712 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:00:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 msgr2=0x7f6bf00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 0x7f6bf00705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8004890 tx=0x7f6bf8009e30 comp rx=0 tx=0).stop 2024-04-26T05:00:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 msgr2=0x7f6c08114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00eb40 tx=0x7f6bfc00b040 comp rx=0 tx=0).stop 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 shutdown_connections 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bf006e140 0x7f6bf00705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c08066f30 0x7f6c08113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 --2- 172.21.15.77:0/3258089053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c080fdd10 0x7f6c08114470 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.711+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 >> 172.21.15.77:0/3258089053 conn(0x7f6c080f7450 msgr2=0x7f6c080f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.712+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 shutdown_connections 2024-04-26T05:00:47.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.712+0000 7f6c0d786700 1 -- 172.21.15.77:0/3258089053 wait complete. 2024-04-26T05:00:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- 172.21.15.77:0/3284435671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fbcb0 msgr2=0x7fa3d40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/3284435671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d40fc0c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ce0 tx=0x7fa3d0004d10 comp rx=0 tx=0).stop 2024-04-26T05:00:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- 172.21.15.77:0/3284435671 shutdown_connections 2024-04-26T05:00:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/3284435671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fcf60 0x7fa3d40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/3284435671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d40fc0c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- 172.21.15.77:0/3284435671 >> 172.21.15.77:0/3284435671 conn(0x7fa3d40f7430 msgr2=0x7fa3d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- 172.21.15.77:0/3284435671 shutdown_connections 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- 172.21.15.77:0/3284435671 wait complete. 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 Processor -- start 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- start start 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.812+0000 7fa3db8fd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d410e590 con 0x7fa3d40fcf60 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3db8fd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d410ed90 con 0x7fa3d40fbcb0 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45862/0 (socket says 172.21.15.77:45862) 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 -- 172.21.15.77:0/2374842985 learned_addr learned my addr 172.21.15.77:0/2374842985 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d9699700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 -- 172.21.15.77:0/2374842985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fbcb0 msgr2=0x7fa3d410da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:47.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d410da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 -- 172.21.15.77:0/2374842985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009940 con 0x7fa3d40fcf60 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3d8e98700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400eb40 tx=0x7fa3c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c4007cc0 con 0x7fa3d40fcf60 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c400f040 con 0x7fa3d40fcf60 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d410f010 con 0x7fa3d40fcf60 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c400ca10 con 0x7fa3d40fcf60 2024-04-26T05:00:47.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.813+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d410f550 con 0x7fa3d40fcf60 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.814+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d4101af0 con 0x7fa3d40fcf60 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.815+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3c400cc80 con 0x7fa3d40fcf60 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.815+0000 7fa3c9ffb700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.815+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa3c408d740 con 0x7fa3d40fcf60 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.815+0000 7fa3d9699700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:00:47.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.816+0000 7fa3d9699700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 0x7fa3c00705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0015040 tx=0x7fa3d0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:00:47.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:47.820+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3c405a650 con 0x7fa3d40fcf60 2024-04-26T05:00:47.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-26T05:00:47.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 6.0 KiB/s wr, 6 op/s 2024-04-26T05:00:47.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:47.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: Upgrade: It appears safe to stop mds.cephfs.smithi078.wkrgkp 2024-04-26T05:00:47.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:47.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/75544556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:47.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:47 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1339190047' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.105+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa3d405f5c0 con 0x7fa3d40fcf60 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.105+0000 7fa3c9ffb700 1 -- 172.21.15.77:0/2374842985 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+345 (secure 0 0 0) 0x7fa3c405a1e0 con 0x7fa3d40fcf60 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in error state 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:00:48.107 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:00:48.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 msgr2=0x7fa3c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:48.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 0x7fa3c00705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0015040 tx=0x7fa3d0004d10 comp rx=0 tx=0).stop 2024-04-26T05:00:48.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 msgr2=0x7fa3d410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:00:48.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400eb40 tx=0x7fa3c400b040 comp rx=0 tx=0).stop 2024-04-26T05:00:48.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 shutdown_connections 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3d40fbcb0 0x7fa3d410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 --2- 172.21.15.77:0/2374842985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3d40fcf60 0x7fa3d410df90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.107+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 >> 172.21.15.77:0/2374842985 conn(0x7fa3d40f7430 msgr2=0x7fa3d40f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.108+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 shutdown_connections 2024-04-26T05:00:48.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:00:48.108+0000 7fa3db8fd700 1 -- 172.21.15.77:0/2374842985 wait complete. 2024-04-26T05:00:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-26T05:00:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 6.0 KiB/s wr, 6 op/s 2024-04-26T05:00:48.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: Upgrade: It appears safe to stop mds.cephfs.smithi078.wkrgkp 2024-04-26T05:00:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/75544556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:00:48.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:47 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1339190047' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: Upgrade: Updating mds.cephfs.smithi078.wkrgkp 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: Deploying daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T05:00:49.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:49 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2374842985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:00:49.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: Upgrade: Updating mds.cephfs.smithi078.wkrgkp 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.wkrgkp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: Deploying daemon mds.cephfs.smithi078.wkrgkp on smithi078 2024-04-26T05:00:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:49 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2374842985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:00:50.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:50 smithi078 ceph-mon[79689]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5.8 KiB/s wr, 5 op/s 2024-04-26T05:00:50.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:50 smithi077 ceph-mon[91833]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5.8 KiB/s wr, 5 op/s 2024-04-26T05:00:50.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:50 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T05:00:50.180+0000 7fe623188700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: osdmap e83: 6 total, 6 up, 6 in 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: Standby daemon mds.cephfs.smithi078.nihewy assigned to filesystem cephfs as rank 0 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:51.394 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:51 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:replay} 2 up:standby 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: osdmap e83: 6 total, 6 up, 6 in 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: Standby daemon mds.cephfs.smithi078.nihewy assigned to filesystem cephfs as rank 0 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:00:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:00:51.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:51 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:replay} 2 up:standby 2024-04-26T05:00:52.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:52 smithi077 ceph-mon[91833]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-04-26T05:00:52.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:52 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:00:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:52 smithi078 ceph-mon[79689]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-04-26T05:00:52.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:52 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:00:54.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:54 smithi077 ceph-mon[91833]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 7.1 MiB/s rd, 2 op/s 2024-04-26T05:00:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:54 smithi078 ceph-mon[79689]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 7.1 MiB/s rd, 2 op/s 2024-04-26T05:00:56.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:56 smithi077 ceph-mon[91833]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-26T05:00:56.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:56 smithi078 ceph-mon[79689]: pgmap v213: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-26T05:00:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:00:58 smithi078 ceph-mon[79689]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-26T05:00:58.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:00:58 smithi077 ceph-mon[91833]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-26T05:01:00.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:00 smithi077 ceph-mon[91833]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-26T05:01:00.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:00 smithi078 ceph-mon[79689]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-26T05:01:01.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.000999985 2024-04-26T05:01:01.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.000999985 2024-04-26T05:01:01.445 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:reconnect 2024-04-26T05:01:01.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:reconnect} 2 up:standby 2024-04-26T05:01:01.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:01.446 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.000999985 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.000999985 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:reconnect 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:reconnect} 2 up:standby 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:02.328 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:02 smithi078 ceph-mon[79689]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:01:02.328 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:02 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:rejoin 2024-04-26T05:01:02.328 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:02 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:rejoin} 2 up:standby 2024-04-26T05:01:02.328 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:02 smithi078 ceph-mon[79689]: daemon mds.cephfs.smithi078.nihewy is now active in filesystem cephfs as rank 0 2024-04-26T05:01:02.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:02 smithi077 ceph-mon[91833]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:01:02.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:02 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:rejoin 2024-04-26T05:01:02.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:02 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi078.nihewy=up:rejoin} 2 up:standby 2024-04-26T05:01:02.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:02 smithi077 ceph-mon[91833]: daemon mds.cephfs.smithi078.nihewy is now active in filesystem cephfs as rank 0 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:active 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi078.nihewy=up:active} 2 up:standby 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:03.486 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:03 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:03.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:01:03.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6826/4041373129,v1:172.21.15.78:6827/4041373129] up:active 2024-04-26T05:01:03.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi078.nihewy=up:active} 2 up:standby 2024-04-26T05:01:03.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:03.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:03.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:03 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:04.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:04 smithi077 ceph-mon[91833]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:04 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] up:boot 2024-04-26T05:01:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:04 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi078.nihewy=up:active} 3 up:standby 2024-04-26T05:01:04.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:04 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:01:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:04 smithi078 ceph-mon[79689]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:04 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] up:boot 2024-04-26T05:01:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:04 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi078.nihewy=up:active} 3 up:standby 2024-04-26T05:01:04.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:04 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:01:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:06 smithi078 ceph-mon[79689]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T05:01:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:06 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:06.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:06 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:06.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:06 smithi077 ceph-mon[91833]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T05:01:06.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:06 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:06.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:06 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:07.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:07 smithi077 ceph-mon[91833]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:01:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:07.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:07 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:08.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:07 smithi078 ceph-mon[79689]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:01:08.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:08.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:07 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:09.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:09 smithi077 ceph-mon[91833]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T05:01:10.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:09 smithi078 ceph-mon[79689]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: Detected new or changed devices on smithi078 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi078.nihewy"]}]: dispatch 2024-04-26T05:01:11.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:11 smithi077 ceph-mon[91833]: Upgrade: It appears safe to stop mds.cephfs.smithi078.nihewy 2024-04-26T05:01:11.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: Detected new or changed devices on smithi078 2024-04-26T05:01:11.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi078.nihewy"]}]: dispatch 2024-04-26T05:01:11.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:11 smithi078 ceph-mon[79689]: Upgrade: It appears safe to stop mds.cephfs.smithi078.nihewy 2024-04-26T05:01:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:12 smithi078 ceph-mon[79689]: Upgrade: Updating mds.cephfs.smithi078.nihewy 2024-04-26T05:01:13.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:13.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:01:13.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:13.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:12 smithi078 ceph-mon[79689]: Deploying daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T05:01:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:12 smithi077 ceph-mon[91833]: Upgrade: Updating mds.cephfs.smithi078.nihewy 2024-04-26T05:01:13.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:13.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi078.nihewy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T05:01:13.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:13.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:12 smithi077 ceph-mon[91833]: Deploying daemon mds.cephfs.smithi078.nihewy on smithi078 2024-04-26T05:01:13.969 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:13 smithi078 ceph-mon[79689]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-04-26T05:01:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:13 smithi077 ceph-mon[91833]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-04-26T05:01:14.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:13 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T05:01:13.968+0000 7fe623188700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:01:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: osdmap e84: 6 total, 6 up, 6 in 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: Standby daemon mds.cephfs.smithi077.quaiyg assigned to filesystem cephfs as rank 0 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:01:15.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:14 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:replay} 2 up:standby 2024-04-26T05:01:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T05:01:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T05:01:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: osdmap e84: 6 total, 6 up, 6 in 2024-04-26T05:01:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: Standby daemon mds.cephfs.smithi077.quaiyg assigned to filesystem cephfs as rank 0 2024-04-26T05:01:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T05:01:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-26T05:01:15.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:14 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:replay} 2 up:standby 2024-04-26T05:01:16.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:15 smithi077 ceph-mon[91833]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-04-26T05:01:16.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:15 smithi078 ceph-mon[79689]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-04-26T05:01:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 -- 172.21.15.77:0/97605644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0066240 msgr2=0x7f63c0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 --2- 172.21.15.77:0/97605644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0066240 0x7f63c0101e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63bc004940 tx=0x7f63bc009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 -- 172.21.15.77:0/97605644 shutdown_connections 2024-04-26T05:01:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 --2- 172.21.15.77:0/97605644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0102360 0x7f63c0104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 --2- 172.21.15.77:0/97605644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0066240 0x7f63c0101e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.214+0000 7f63c81c2700 1 -- 172.21.15.77:0/97605644 >> 172.21.15.77:0/97605644 conn(0x7f63c00f7430 msgr2=0x7f63c00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 -- 172.21.15.77:0/97605644 shutdown_connections 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 -- 172.21.15.77:0/97605644 wait complete. 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 Processor -- start 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 -- start start 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 0x7f63c010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c0110760 con 0x7f63c0102360 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.215+0000 7f63c81c2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c0110fb0 con 0x7f63c0066240 2024-04-26T05:01:18.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c575d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c575d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50322/0 (socket says 172.21.15.77:50322) 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c575d700 1 -- 172.21.15.77:0/1082759541 learned_addr learned my addr 172.21.15.77:0/1082759541 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c5f5e700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 0x7f63c010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c5f5e700 1 -- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 msgr2=0x7f63c0110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c5f5e700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c5f5e700 1 -- 172.21.15.77:0/1082759541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc005040 con 0x7f63c0066240 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c575d700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c5f5e700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 0x7f63c010fc20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000c00 tx=0x7f63bc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63bc00e070 con 0x7f63c0066240 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63bc01c400 con 0x7f63c0066240 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c0111230 con 0x7f63c0066240 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63bc012620 con 0x7f63c0066240 2024-04-26T05:01:18.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.216+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c0111740 con 0x7f63c0066240 2024-04-26T05:01:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.217+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c01082e0 con 0x7f63c0066240 2024-04-26T05:01:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.218+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f63bc01ca90 con 0x7f63c0066240 2024-04-26T05:01:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.218+0000 7f63b67fc700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 0x7f63ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.218+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f63bc08ee70 con 0x7f63c0066240 2024-04-26T05:01:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.218+0000 7f63c575d700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 0x7f63ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.219+0000 7f63c575d700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 0x7f63ac0705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63b0007fd0 tx=0x7f63b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.223+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f63bc05bd00 con 0x7f63c0066240 2024-04-26T05:01:18.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:17 smithi077 ceph-mon[91833]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 B/s wr, 1 op/s 2024-04-26T05:01:18.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:17 smithi078 ceph-mon[79689]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 B/s wr, 1 op/s 2024-04-26T05:01:18.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.459+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63c01119b0 con 0x7f63ac06e140 2024-04-26T05:01:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.460+0000 7f63b67fc700 1 -- 172.21.15.77:0/1082759541 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f63c01119b0 con 0x7f63ac06e140 2024-04-26T05:01:18.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.462+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 msgr2=0x7f63ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.462+0000 7f63c81c2700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 0x7f63ac0705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63b0007fd0 tx=0x7f63b000b040 comp rx=0 tx=0).stop 2024-04-26T05:01:18.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.462+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 msgr2=0x7f63c010fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.462+0000 7f63c81c2700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 0x7f63c010fc20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000c00 tx=0x7f63bc009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 shutdown_connections 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f63ac06e140 0x7f63ac0705f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63c0066240 0x7f63c010fc20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 --2- 172.21.15.77:0/1082759541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63c0102360 0x7f63c0110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 >> 172.21.15.77:0/1082759541 conn(0x7f63c00f7430 msgr2=0x7f63c00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 shutdown_connections 2024-04-26T05:01:18.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.463+0000 7f63c81c2700 1 -- 172.21.15.77:0/1082759541 wait complete. 2024-04-26T05:01:18.473 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:01:18.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- 172.21.15.77:0/1879613330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 msgr2=0x7f35f41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1879613330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41047b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35e4004940 tx=0x7f35e4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:18.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- 172.21.15.77:0/1879613330 shutdown_connections 2024-04-26T05:01:18.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1879613330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41047b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1879613330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 0x7f35f4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- 172.21.15.77:0/1879613330 >> 172.21.15.77:0/1879613330 conn(0x7f35f40f74c0 msgr2=0x7f35f40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- 172.21.15.77:0/1879613330 shutdown_connections 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- 172.21.15.77:0/1879613330 wait complete. 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 Processor -- start 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- start start 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 0x7f35f410fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41106e0 con 0x7f35f41023d0 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.568+0000 7f35fbaef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4110f30 con 0x7f35f4066af0 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50328/0 (socket says 172.21.15.77:50328) 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 -- 172.21.15.77:0/1069054533 learned_addr learned my addr 172.21.15.77:0/1069054533 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f988b700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 0x7f35f410fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 -- 172.21.15.77:0/1069054533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 msgr2=0x7f35f410fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 0x7f35f410fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 -- 172.21.15.77:0/1069054533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4005040 con 0x7f35f41023d0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35f908a700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009150 tx=0x7f35e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e400e040 con 0x7f35f41023d0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35e400bbf0 con 0x7f35f41023d0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e401c630 con 0x7f35f41023d0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f41111b0 con 0x7f35f41023d0 2024-04-26T05:01:18.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.569+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f41116f0 con 0x7f35f41023d0 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.570+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f405f5c0 con 0x7f35f41023d0 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.571+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f35e400bd60 con 0x7f35f41023d0 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.571+0000 7f35ea7fc700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 0x7f35e0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.572+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f35e408ee60 con 0x7f35f41023d0 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.572+0000 7f35f988b700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 0x7f35e0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.572+0000 7f35f988b700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 0x7f35e0070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009fd0 tx=0x7f35f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.576+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f35e405bd30 con 0x7f35f41023d0 2024-04-26T05:01:18.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.801+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f4111ae0 con 0x7f35e006e150 2024-04-26T05:01:18.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.803+0000 7f35ea7fc700 1 -- 172.21.15.77:0/1069054533 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f35f4111ae0 con 0x7f35e006e150 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 msgr2=0x7f35e0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 0x7f35e0070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009fd0 tx=0x7f35f000b040 comp rx=0 tx=0).stop 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 msgr2=0x7f35f41100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009150 tx=0x7f35e4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 shutdown_connections 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f35e006e150 0x7f35e0070600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f4066af0 0x7f35f410fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 --2- 172.21.15.77:0/1069054533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35f41023d0 0x7f35f41100e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 >> 172.21.15.77:0/1069054533 conn(0x7f35f40f74c0 msgr2=0x7f35f40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:18.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 shutdown_connections 2024-04-26T05:01:18.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.805+0000 7f35fbaef700 1 -- 172.21.15.77:0/1069054533 wait complete. 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- 172.21.15.77:0/4291119203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 msgr2=0x7fe608101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 --2- 172.21.15.77:0/4291119203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe608101e00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe604004940 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- 172.21.15.77:0/4291119203 shutdown_connections 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 --2- 172.21.15.77:0/4291119203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 0x7fe608104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 --2- 172.21.15.77:0/4291119203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe608101e00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- 172.21.15.77:0/4291119203 >> 172.21.15.77:0/4291119203 conn(0x7fe6080f7430 msgr2=0x7fe6080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- 172.21.15.77:0/4291119203 shutdown_connections 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- 172.21.15.77:0/4291119203 wait complete. 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 Processor -- start 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.909+0000 7fe6103fa700 1 -- start start 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 0x7fe608110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608110780 con 0x7fe608066260 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608110fd0 con 0x7fe608102340 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50338/0 (socket says 172.21.15.77:50338) 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 -- 172.21.15.77:0/3540020813 learned_addr learned my addr 172.21.15.77:0/3540020813 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60d995700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 0x7fe608110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 -- 172.21.15.77:0/3540020813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 msgr2=0x7fe608110180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 0x7fe608110180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 -- 172.21.15.77:0/3540020813 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604005040 con 0x7fe608066260 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe60e196700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe604000c00 tx=0x7fe604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60400e040 con 0x7fe608066260 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608111250 con 0x7fe608066260 2024-04-26T05:01:18.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe60400bbf0 con 0x7fe608066260 2024-04-26T05:01:18.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60401c630 con 0x7fe608066260 2024-04-26T05:01:18.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.910+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe608111790 con 0x7fe608066260 2024-04-26T05:01:18.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.911+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60805f5c0 con 0x7fe608066260 2024-04-26T05:01:18.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.912+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe60400bd60 con 0x7fe608066260 2024-04-26T05:01:18.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.913+0000 7fe5feffd700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 0x7fe5f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:18.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.913+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe60408ef00 con 0x7fe608066260 2024-04-26T05:01:18.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.913+0000 7fe60d995700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 0x7fe5f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:18.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.914+0000 7fe60d995700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 0x7fe5f40705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004f40 tx=0x7fe5f800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:18.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:18.922+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe60405bd30 con 0x7fe608066260 2024-04-26T05:01:19.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.137+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe608111b80 con 0x7fe5f406e140 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.147+0000 7fe5feffd700 1 -- 172.21.15.77:0/3540020813 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe608111b80 con 0x7fe5f406e140 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (7m) 37s ago 16m 24.8M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (4m) 37s ago 16m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (4m) 13s ago 15m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 error 37s ago 16m - - 2024-04-26T05:01:19.148 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (71s) 37s ago 11m 12.7M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (42s) 37s ago 11m 13.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (11m) 13s ago 11m 144M - 17.2.7-902-gc61472c8 0de407c1b3af 38d4a72d9905 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (16s) 13s ago 11m 13.8M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 04b98ae4596a 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (7m) 37s ago 18m 507M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (6m) 13s ago 15m 453M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (5m) 37s ago 18m 51.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (5m) 13s ago 15m 38.9M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (8m) 37s ago 16m 16.2M - 1.5.0 0da6a335fe13 cead4e3c7d10 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (7m) 13s ago 15m 16.4M - 1.5.0 0da6a335fe13 4883a1e433ef 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (4m) 37s ago 15m 95.3M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (3m) 37s ago 14m 96.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (3m) 37s ago 14m 94.1M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (2m) 13s ago 13m 116M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (2m) 13s ago 13m 91.9M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:01:19.149 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (92s) 13s ago 12m 85.3M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:01:19.150 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (6m) 37s ago 16m 61.1M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T05:01:19.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 msgr2=0x7fe5f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 0x7fe5f40705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004f40 tx=0x7fe5f800b040 comp rx=0 tx=0).stop 2024-04-26T05:01:19.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 msgr2=0x7fe60810fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe604000c00 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 shutdown_connections 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe5f406e140 0x7fe5f40705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe608066260 0x7fe60810fc40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 --2- 172.21.15.77:0/3540020813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe608102340 0x7fe608110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 >> 172.21.15.77:0/3540020813 conn(0x7fe6080f7430 msgr2=0x7fe6080f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 shutdown_connections 2024-04-26T05:01:19.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.151+0000 7fe6103fa700 1 -- 172.21.15.77:0/3540020813 wait complete. 2024-04-26T05:01:19.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- 172.21.15.77:0/4128539778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fe080 msgr2=0x7ff1140fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4128539778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fe080 0x7ff1140fe4f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff104004940 tx=0x7ff104009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- 172.21.15.77:0/4128539778 shutdown_connections 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4128539778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fe080 0x7ff1140fe4f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4128539778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fce80 0x7ff1140fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- 172.21.15.77:0/4128539778 >> 172.21.15.77:0/4128539778 conn(0x7ff1140f8590 msgr2=0x7ff1140fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- 172.21.15.77:0/4128539778 shutdown_connections 2024-04-26T05:01:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- 172.21.15.77:0/4128539778 wait complete. 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 Processor -- start 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.253+0000 7ff11b69f700 1 -- start start 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11b69f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11b69f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fe080 0x7ff114110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11b69f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114110790 con 0x7ff1140fce80 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11b69f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff114110fe0 con 0x7ff1140fe080 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50354/0 (socket says 172.21.15.77:50354) 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 -- 172.21.15.77:0/4045612203 learned_addr learned my addr 172.21.15.77:0/4045612203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff118c3a700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fe080 0x7ff114110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 -- 172.21.15.77:0/4045612203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fe080 msgr2=0x7ff114110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fe080 0x7ff114110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 -- 172.21.15.77:0/4045612203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104005040 con 0x7ff1140fce80 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff11943b700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff110002a80 tx=0x7ff110004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff110007cc0 con 0x7ff1140fce80 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff11000f040 con 0x7ff1140fce80 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.254+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11000dad0 con 0x7ff1140fce80 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.255+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1141112c0 con 0x7ff1140fce80 2024-04-26T05:01:19.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.255+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1141117b0 con 0x7ff1140fce80 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.256+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff114101a10 con 0x7ff1140fce80 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.256+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff11000dc30 con 0x7ff1140fce80 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.257+0000 7ff109ffb700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 0x7ff1000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.257+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff11008da80 con 0x7ff1140fce80 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.257+0000 7ff118c3a700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 0x7ff1000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.257+0000 7ff118c3a700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 0x7ff1000705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff104009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:19.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.261+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff11005a790 con 0x7ff1140fce80 2024-04-26T05:01:19.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.570+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff114111a20 con 0x7ff1140fce80 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.570+0000 7ff109ffb700 1 -- 172.21.15.77:0/4045612203 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff11005a320 con 0x7ff1140fce80 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:01:19.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 3 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 13 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:01:19.572 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:01:19.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.573+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 msgr2=0x7ff1000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.573+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 0x7ff1000705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff104009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:19.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.573+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 msgr2=0x7ff11410fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.573+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff110002a80 tx=0x7ff110004600 comp rx=0 tx=0).stop 2024-04-26T05:01:19.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 shutdown_connections 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff10006e140 0x7ff1000705f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1140fce80 0x7ff11410fc50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 --2- 172.21.15.77:0/4045612203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1140fe080 0x7ff114110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 >> 172.21.15.77:0/4045612203 conn(0x7ff1140f8590 msgr2=0x7ff1140f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 shutdown_connections 2024-04-26T05:01:19.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.574+0000 7ff11b69f700 1 -- 172.21.15.77:0/4045612203 wait complete. 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 -- 172.21.15.77:0/167645118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0102340 msgr2=0x7f4fc0104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/167645118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0102340 0x7f4fc0104720 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4004940 tx=0x7f4fb4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 -- 172.21.15.77:0/167645118 shutdown_connections 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/167645118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0102340 0x7f4fc0104720 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/167645118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0066260 0x7f4fc0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 -- 172.21.15.77:0/167645118 >> 172.21.15.77:0/167645118 conn(0x7f4fc00f7430 msgr2=0x7f4fc00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 -- 172.21.15.77:0/167645118 shutdown_connections 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 -- 172.21.15.77:0/167645118 wait complete. 2024-04-26T05:01:19.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.680+0000 7f4fc5de3700 1 Processor -- start 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fc5de3700 1 -- start start 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fc5de3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fc5de3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0102340 0x7f4fc0110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fc5de3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0110750 con 0x7f4fc0066260 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fc5de3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0110fa0 con 0x7f4fc0102340 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50372/0 (socket says 172.21.15.77:50372) 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 -- 172.21.15.77:0/1385575868 learned_addr learned my addr 172.21.15.77:0/1385575868 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 -- 172.21.15.77:0/1385575868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0102340 msgr2=0x7f4fc0110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbeffd700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0102340 0x7f4fc0110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0102340 0x7f4fc0110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 -- 172.21.15.77:0/1385575868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4005040 con 0x7f4fc0066260 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.681+0000 7f4fbf7fe700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c8b0 tx=0x7f4fb0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.682+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb000a760 con 0x7f4fc0066260 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.682+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0111280 con 0x7f4fc0066260 2024-04-26T05:01:19.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.682+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb000ad90 con 0x7f4fc0066260 2024-04-26T05:01:19.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.683+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb00113a0 con 0x7f4fc0066260 2024-04-26T05:01:19.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.683+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc01117c0 con 0x7f4fc0066260 2024-04-26T05:01:19.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.683+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4fb000a8c0 con 0x7f4fc0066260 2024-04-26T05:01:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.683+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc005f5c0 con 0x7f4fc0066260 2024-04-26T05:01:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.684+0000 7f4fa7fff700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 0x7f4fa80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.684+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4fb008d3b0 con 0x7f4fc0066260 2024-04-26T05:01:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.684+0000 7f4fbeffd700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 0x7f4fa80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:19.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.685+0000 7f4fbeffd700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 0x7f4fa80705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.689+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4fb0092060 con 0x7f4fc0066260 2024-04-26T05:01:19.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.939+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4fc0107a90 con 0x7f4fc0066260 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.939+0000 7f4fa7fff700 1 -- 172.21.15.77:0/1385575868 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1748 (secure 0 0 0) 0x7f4fb005a1d0 con 0x7f4fc0066260 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout:e33 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:19.940 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 33 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:01:13.973602+0000 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:01:19.941 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 84 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=34376} 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:01:19.942 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 0 members: 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:34376} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.941+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 msgr2=0x7f4fa80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.941+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 0x7f4fa80705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.941+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 msgr2=0x7f4fc010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.941+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c8b0 tx=0x7f4fb0004470 comp rx=0 tx=0).stop 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 shutdown_connections 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4fa806e140 0x7f4fa80705f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0066260 0x7f4fc010fc10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 --2- 172.21.15.77:0/1385575868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc0102340 0x7f4fc0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:19.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 >> 172.21.15.77:0/1385575868 conn(0x7f4fc00f7430 msgr2=0x7f4fc00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 shutdown_connections 2024-04-26T05:01:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:19.942+0000 7f4fc5de3700 1 -- 172.21.15.77:0/1385575868 wait complete. 2024-04-26T05:01:19.945 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 33 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 -- 172.21.15.77:0/3813720488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 msgr2=0x7f549c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 --2- 172.21.15.77:0/3813720488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c0ff9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f548c004940 tx=0x7f548c009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 -- 172.21.15.77:0/3813720488 shutdown_connections 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 --2- 172.21.15.77:0/3813720488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 --2- 172.21.15.77:0/3813720488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c0ff9a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 -- 172.21.15.77:0/3813720488 >> 172.21.15.77:0/3813720488 conn(0x7f549c0f7430 msgr2=0x7f549c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:20.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 -- 172.21.15.77:0/3813720488 shutdown_connections 2024-04-26T05:01:20.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.048+0000 7f54a2264700 1 -- 172.21.15.77:0/3813720488 wait complete. 2024-04-26T05:01:20.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 Processor -- start 2024-04-26T05:01:20.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 -- start start 2024-04-26T05:01:20.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c10e570 con 0x7f549c0662c0 2024-04-26T05:01:20.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.049+0000 7f54a2264700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c10edc0 con 0x7f549c0ffee0 2024-04-26T05:01:20.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50396/0 (socket says 172.21.15.77:50396) 2024-04-26T05:01:20.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 -- 172.21.15.77:0/112091721 learned_addr learned my addr 172.21.15.77:0/112091721 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:20.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549b7fe700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 -- 172.21.15.77:0/112091721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 msgr2=0x7f549c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 -- 172.21.15.77:0/112091721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c005040 con 0x7f549c0662c0 2024-04-26T05:01:20.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549b7fe700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:01:20.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f549bfff700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f548c00b3b0 tx=0x7f548c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:20.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c00e040 con 0x7f549c0662c0 2024-04-26T05:01:20.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548c00bcc0 con 0x7f549c0662c0 2024-04-26T05:01:20.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548c01c630 con 0x7f549c0662c0 2024-04-26T05:01:20.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c10f040 con 0x7f549c0662c0 2024-04-26T05:01:20.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.050+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c10f580 con 0x7f549c0662c0 2024-04-26T05:01:20.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.051+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c05f5c0 con 0x7f549c0662c0 2024-04-26T05:01:20.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.052+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f548c013420 con 0x7f549c0662c0 2024-04-26T05:01:20.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.053+0000 7f5498ff9700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.053+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f548c08e820 con 0x7f549c0662c0 2024-04-26T05:01:20.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.053+0000 7f549b7fe700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.054+0000 7f549b7fe700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 0x7f54840705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5490007fd0 tx=0x7f5490009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:20.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.058+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f548c05b6f0 con 0x7f549c0662c0 2024-04-26T05:01:20.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:19 smithi077 ceph-mon[91833]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:01:20.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:19 smithi077 ceph-mon[91833]: from='client.44327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:20.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:19 smithi077 ceph-mon[91833]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:20.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:19 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/4045612203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:20.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:19 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1385575868' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:01:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:19 smithi078 ceph-mon[79689]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:01:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:19 smithi078 ceph-mon[79689]: from='client.44327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:19 smithi078 ceph-mon[79689]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:19 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/4045612203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:20.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:19 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1385575868' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:01:20.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.283+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f549c10f9c0 con 0x7f548406e140 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.284+0000 7f5498ff9700 1 -- 172.21.15.77:0/112091721 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f549c10f9c0 con 0x7f548406e140 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "osd", 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: "mon" 2024-04-26T05:01:20.285 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T05:01:20.286 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "15/21 daemons upgraded", 2024-04-26T05:01:20.286 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading mds daemons", 2024-04-26T05:01:20.286 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:01:20.286 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 msgr2=0x7f54840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 0x7f54840705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5490007fd0 tx=0x7f5490009040 comp rx=0 tx=0).stop 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 msgr2=0x7f549c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f548c00b3b0 tx=0x7f548c009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 shutdown_connections 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f549c0662c0 0x7f549c10da30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 --2- 172.21.15.77:0/112091721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f549c0ffee0 0x7f549c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 >> 172.21.15.77:0/112091721 conn(0x7f549c0f7430 msgr2=0x7f549c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:20.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.286+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 shutdown_connections 2024-04-26T05:01:20.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.287+0000 7f54a2264700 1 -- 172.21.15.77:0/112091721 wait complete. 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1319920275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fce60 msgr2=0x7f1e780fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1319920275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fce60 0x7f1e780fd270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e68004940 tx=0x7f1e68009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1319920275 shutdown_connections 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1319920275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fdfd0 0x7f1e780fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1319920275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fce60 0x7f1e780fd270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1319920275 >> 172.21.15.77:0/1319920275 conn(0x7f1e780f8560 msgr2=0x7f1e780fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1319920275 shutdown_connections 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1319920275 wait complete. 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.389+0000 7f1e7cb5e700 1 Processor -- start 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7cb5e700 1 -- start start 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7cb5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 0x7f1e78071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7cb5e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7cb5e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78070860 con 0x7f1e780fdfd0 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7cb5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780709d0 con 0x7f1e780fce60 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50406/0 (socket says 172.21.15.77:50406) 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 -- 172.21.15.77:0/1948898411 learned_addr learned my addr 172.21.15.77:0/1948898411 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7659c700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 0x7f1e78071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 -- 172.21.15.77:0/1948898411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 msgr2=0x7f1e78071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 0x7f1e78071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 -- 172.21.15.77:0/1948898411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68005040 con 0x7f1e780fdfd0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e7659c700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 0x7f1e78071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e75d9b700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c8c0 tx=0x7f1e6c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.390+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6c00a980 con 0x7f1e780fdfd0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.391+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e78070cb0 con 0x7f1e780fdfd0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.391+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e6c00ce60 con 0x7f1e780fdfd0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.391+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6c0113a0 con 0x7f1e780fdfd0 2024-04-26T05:01:20.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.391+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e78115a20 con 0x7f1e780fdfd0 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.392+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7805f5c0 con 0x7f1e780fdfd0 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.392+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e6c012850 con 0x7f1e780fdfd0 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.393+0000 7f1e66ffd700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 0x7f1e600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.393+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1e6c08dd20 con 0x7f1e780fdfd0 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.393+0000 7f1e7659c700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 0x7f1e600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:20.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.393+0000 7f1e7659c700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 0x7f1e600705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:20.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.401+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1e6c05ab50 con 0x7f1e780fdfd0 2024-04-26T05:01:20.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.715+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1e7804bed0 con 0x7f1e780fdfd0 2024-04-26T05:01:20.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.716+0000 7f1e66ffd700 1 -- 172.21.15.77:0/1948898411 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+441 (secure 0 0 0) 0x7f1e6c05a6e0 con 0x7f1e780fdfd0 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in error state 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs is degraded 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:01:20.717 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 msgr2=0x7f1e600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 0x7f1e600705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 msgr2=0x7f1e780702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c8c0 tx=0x7f1e6c0044c0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 shutdown_connections 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1e6006e140 0x7f1e600705f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e780fce60 0x7f1e78071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 --2- 172.21.15.77:0/1948898411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e780fdfd0 0x7f1e780702b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 >> 172.21.15.77:0/1948898411 conn(0x7f1e780f8560 msgr2=0x7f1e780f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:20.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 shutdown_connections 2024-04-26T05:01:20.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:20.718+0000 7f1e7cb5e700 1 -- 172.21.15.77:0/1948898411 wait complete. 2024-04-26T05:01:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:20 smithi078 ceph-mon[79689]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:20 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1948898411' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:01:21.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:20 smithi077 ceph-mon[91833]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:21.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:20 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1948898411' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:01:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:21 smithi077 ceph-mon[91833]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:01:22.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:21 smithi077 ceph-mon[91833]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:21 smithi078 ceph-mon[79689]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-26T05:01:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:21 smithi078 ceph-mon[79689]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:24 smithi078 ceph-mon[79689]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:24.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:24 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:24.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:24 smithi077 ceph-mon[91833]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:24.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:24.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:24.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:24 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.00099998 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.00099998 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:reconnect 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.78:6826/1653368163,v1:172.21.15.78:6827/1653368163] up:boot 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:reconnect} 3 up:standby 2024-04-26T05:01:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:25 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:25.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: reconnect by client.14522 192.168.0.1:0/4289005399 after 0.00099998 2024-04-26T05:01:25.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: reconnect by client.24305 192.168.0.1:0/4280036521 after 0.00099998 2024-04-26T05:01:25.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:reconnect 2024-04-26T05:01:25.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.78:6826/1653368163,v1:172.21.15.78:6827/1653368163] up:boot 2024-04-26T05:01:25.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:reconnect} 3 up:standby 2024-04-26T05:01:25.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:25 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:26 smithi078 ceph-mon[79689]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:01:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:26 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:rejoin 2024-04-26T05:01:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:26 smithi078 ceph-mon[79689]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:rejoin} 3 up:standby 2024-04-26T05:01:26.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:26 smithi078 ceph-mon[79689]: daemon mds.cephfs.smithi077.quaiyg is now active in filesystem cephfs as rank 0 2024-04-26T05:01:26.428 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:26 smithi077 ceph-mon[91833]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-04-26T05:01:26.428 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:26 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:rejoin 2024-04-26T05:01:26.428 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:26 smithi077 ceph-mon[91833]: fsmap cephfs:1/1 {0=cephfs.smithi077.quaiyg=up:rejoin} 3 up:standby 2024-04-26T05:01:26.429 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:26 smithi077 ceph-mon[91833]: daemon mds.cephfs.smithi077.quaiyg is now active in filesystem cephfs as rank 0 2024-04-26T05:01:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:27 smithi078 ceph-mon[79689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:01:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:27 smithi078 ceph-mon[79689]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:active 2024-04-26T05:01:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:27 smithi078 ceph-mon[79689]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T05:01:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:27.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:27 smithi077 ceph-mon[91833]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T05:01:27.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:27 smithi077 ceph-mon[91833]: mds.? [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] up:active 2024-04-26T05:01:27.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:27 smithi077 ceph-mon[91833]: fsmap cephfs:1 {0=cephfs.smithi077.quaiyg=up:active} 3 up:standby 2024-04-26T05:01:27.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:27.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:28 smithi078 ceph-mon[79689]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:28 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:28.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:28 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:28.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:28 smithi077 ceph-mon[91833]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-26T05:01:28.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:28 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:28.363 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:28 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:30.129 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:30 smithi078 ceph-mon[79689]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T05:01:30.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:30 smithi077 ceph-mon[91833]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T05:01:31.681 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:01:31.681 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.681 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.681 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 195 B/s wr, 5 op/s 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 241 B/s wr, 5 op/s 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all mds 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:01:31.682 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.quaiyg"}]': finished 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.urxtgp"}]': finished 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.nihewy"}]': finished 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.wkrgkp"}]': finished 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all rgw 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.683 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all ceph-exporter 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all iscsi 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all nfs 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: Upgrade: Setting container_image for all nvmeof 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:31.684 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T05:01:32.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 195 B/s wr, 5 op/s 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 241 B/s wr, 5 op/s 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.018 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all mds 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.quaiyg"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.quaiyg"}]': finished 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.urxtgp"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi077.urxtgp"}]': finished 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.nihewy"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.nihewy"}]': finished 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.wkrgkp"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi078.wkrgkp"}]': finished 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all rgw 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.019 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all ceph-exporter 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all iscsi 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all nfs 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: Upgrade: Setting container_image for all nvmeof 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.020 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:32.021 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:33.807 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:33 smithi077 ceph-mon[91833]: Upgrade: Updating node-exporter.smithi077 (1/2) 2024-04-26T05:01:33.807 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:33 smithi077 ceph-mon[91833]: Deploying daemon node-exporter.smithi077 on smithi077 2024-04-26T05:01:33.807 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:33 smithi077 ceph-mon[91833]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 241 B/s wr, 4 op/s 2024-04-26T05:01:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:33 smithi078 ceph-mon[79689]: Upgrade: Updating node-exporter.smithi077 (1/2) 2024-04-26T05:01:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:33 smithi078 ceph-mon[79689]: Deploying daemon node-exporter.smithi077 on smithi077 2024-04-26T05:01:34.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:33 smithi078 ceph-mon[79689]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 241 B/s wr, 4 op/s 2024-04-26T05:01:35.909 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:35 smithi078 ceph-mon[79689]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-26T05:01:35.909 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:35.909 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:35 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:35 smithi077 ceph-mon[91833]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-26T05:01:35.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:35.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:35 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:37.884 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:37 smithi078 ceph-mon[79689]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 725 B/s rd, 5.8 KiB/s wr, 3 op/s 2024-04-26T05:01:37.884 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:37 smithi078 ceph-mon[79689]: Upgrade: Updating node-exporter.smithi078 (2/2) 2024-04-26T05:01:37.884 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:37 smithi078 ceph-mon[79689]: Deploying daemon node-exporter.smithi078 on smithi078 2024-04-26T05:01:37.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:37 smithi077 ceph-mon[91833]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 725 B/s rd, 5.8 KiB/s wr, 3 op/s 2024-04-26T05:01:37.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:37 smithi077 ceph-mon[91833]: Upgrade: Updating node-exporter.smithi078 (2/2) 2024-04-26T05:01:37.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:37 smithi077 ceph-mon[91833]: Deploying daemon node-exporter.smithi078 on smithi078 2024-04-26T05:01:39.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:39 smithi077 ceph-mon[91833]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 362 B/s rd, 5.8 KiB/s wr, 3 op/s 2024-04-26T05:01:39.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:39.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:39.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:39 smithi078 ceph-mon[79689]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 362 B/s rd, 5.8 KiB/s wr, 3 op/s 2024-04-26T05:01:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:40.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:41.887 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:41 smithi077 ceph-mon[91833]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.9 KiB/s wr, 2 op/s 2024-04-26T05:01:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:41 smithi078 ceph-mon[79689]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.9 KiB/s wr, 2 op/s 2024-04-26T05:01:42.874 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:42.874 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:42 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:43.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:42 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.220 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:43 smithi078 ceph-mon[79689]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-26T05:01:44.220 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.220 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.220 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.221 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:43 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:43 smithi077 ceph-mon[91833]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-26T05:01:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:44.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:43 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:45.601 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:45.601 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:45.602 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:45 smithi078 ceph-mon[79689]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-26T05:01:45.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:45.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:45.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:45 smithi077 ceph-mon[91833]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-26T05:01:46.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:46.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:46.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:01:46.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:46.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:47.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:47 smithi077 ceph-mon[91833]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:47.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:47 smithi078 ceph-mon[79689]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:49.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:49 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:49.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:49 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1071534713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 msgr2=0x7f3a700fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1071534713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a700fd280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004940 tx=0x7f3a64009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1071534713 shutdown_connections 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1071534713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 0x7f3a700fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1071534713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a700fd280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1071534713 >> 172.21.15.77:0/1071534713 conn(0x7f3a700f8550 msgr2=0x7f3a700fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1071534713 shutdown_connections 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.822+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1071534713 wait complete. 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 Processor -- start 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 -- start start 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 0x7f3a7010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7010e470 con 0x7f3a700fce70 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a76ad5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7010e5e0 con 0x7f3a700fdfe0 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a74871700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a74871700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42716/0 (socket says 172.21.15.77:42716) 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.823+0000 7f3a74871700 1 -- 172.21.15.77:0/1702334973 learned_addr learned my addr 172.21.15.77:0/1702334973 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a6ffff700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 0x7f3a7010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a74871700 1 -- 172.21.15.77:0/1702334973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 msgr2=0x7f3a7010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a74871700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 0x7f3a7010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a74871700 1 -- 172.21.15.77:0/1702334973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64005040 con 0x7f3a700fce70 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a74871700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009150 tx=0x7f3a64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6400e040 con 0x7f3a700fce70 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a6400bcc0 con 0x7f3a700fce70 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a70112fe0 con 0x7f3a700fce70 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6401c630 con 0x7f3a700fce70 2024-04-26T05:01:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.824+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a70113510 con 0x7f3a700fce70 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.825+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7004c710 con 0x7f3a700fce70 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.825+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3a64013420 con 0x7f3a700fce70 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.826+0000 7f3a6d7fa700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 0x7f3a600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.826+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3a6408e8c0 con 0x7f3a700fce70 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.826+0000 7f3a6ffff700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 0x7f3a600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.826+0000 7f3a6ffff700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 0x7f3a600705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c007fd0 tx=0x7f3a5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:50.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:50.831+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3a6405b6f0 con 0x7f3a700fce70 2024-04-26T05:01:51.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.056+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a70102300 con 0x7f3a6006e140 2024-04-26T05:01:51.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.057+0000 7f3a6d7fa700 1 -- 172.21.15.77:0/1702334973 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f3a70102300 con 0x7f3a6006e140 2024-04-26T05:01:51.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 msgr2=0x7f3a600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 0x7f3a600705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c007fd0 tx=0x7f3a5c009040 comp rx=0 tx=0).stop 2024-04-26T05:01:51.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 msgr2=0x7f3a7010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009150 tx=0x7f3a64009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:51.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 shutdown_connections 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.059+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f3a6006e140 0x7f3a600705f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.060+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a700fce70 0x7f3a7010d930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.060+0000 7f3a76ad5700 1 --2- 172.21.15.77:0/1702334973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a700fdfe0 0x7f3a7010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.060+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 >> 172.21.15.77:0/1702334973 conn(0x7f3a700f8550 msgr2=0x7f3a700f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.060+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 shutdown_connections 2024-04-26T05:01:51.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.060+0000 7f3a76ad5700 1 -- 172.21.15.77:0/1702334973 wait complete. 2024-04-26T05:01:51.071 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 -- 172.21.15.77:0/566451235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 msgr2=0x7f40f00fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 --2- 172.21.15.77:0/566451235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f00fc150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009ce0 tx=0x7f40e8004d10 comp rx=0 tx=0).stop 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 -- 172.21.15.77:0/566451235 shutdown_connections 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 --2- 172.21.15.77:0/566451235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f00fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 --2- 172.21.15.77:0/566451235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f00fc150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 -- 172.21.15.77:0/566451235 >> 172.21.15.77:0/566451235 conn(0x7f40f00f74c0 msgr2=0x7f40f00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.166 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:51 smithi077 ceph-mon[91833]: Upgrade: Updating prometheus.smithi077 2024-04-26T05:01:51.166 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:51 smithi077 ceph-mon[91833]: Deploying daemon prometheus.smithi077 on smithi077 2024-04-26T05:01:51.167 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:51 smithi077 ceph-mon[91833]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 -- 172.21.15.77:0/566451235 shutdown_connections 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.164+0000 7f40f5207700 1 -- 172.21.15.77:0/566451235 wait complete. 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 Processor -- start 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 -- start start 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f010e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f010e700 con 0x7f40f00fbd40 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40f5207700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f010ef00 con 0x7f40f00fcff0 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40eed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40eed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42734/0 (socket says 172.21.15.77:42734) 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.165+0000 7f40eed9d700 1 -- 172.21.15.77:0/1375919043 learned_addr learned my addr 172.21.15.77:0/1375919043 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40ee59c700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f010e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40eed9d700 1 -- 172.21.15.77:0/1375919043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 msgr2=0x7f40f010e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40eed9d700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f010e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40eed9d700 1 -- 172.21.15.77:0/1375919043 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009940 con 0x7f40f00fbd40 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40ee59c700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40eed9d700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40e8004740 tx=0x7f40e8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e800e040 con 0x7f40f00fbd40 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40e80128e0 con 0x7f40f00fbd40 2024-04-26T05:01:51.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f010f180 con 0x7f40f00fbd40 2024-04-26T05:01:51.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e801ca80 con 0x7f40f00fbd40 2024-04-26T05:01:51.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.166+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f010f6c0 con 0x7f40f00fbd40 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.167+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f0101b80 con 0x7f40f00fbd40 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.167+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40e801cc70 con 0x7f40f00fbd40 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.168+0000 7f40d77fe700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 0x7f40d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.168+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f40e808f8e0 con 0x7f40f00fbd40 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.168+0000 7f40ee59c700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 0x7f40d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.168+0000 7f40ee59c700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 0x7f40d80705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40e4007fd0 tx=0x7f40e4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.174+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f40e805c700 con 0x7f40f00fbd40 2024-04-26T05:01:51.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.393+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40f010f8e0 con 0x7f40d806e140 2024-04-26T05:01:51.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.395+0000 7f40d77fe700 1 -- 172.21.15.77:0/1375919043 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f40f010f8e0 con 0x7f40d806e140 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 msgr2=0x7f40d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 0x7f40d80705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40e4007fd0 tx=0x7f40e4009040 comp rx=0 tx=0).stop 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 msgr2=0x7f40f010dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40e8004740 tx=0x7f40e8004d10 comp rx=0 tx=0).stop 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 shutdown_connections 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f40d806e140 0x7f40d80705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f00fbd40 0x7f40f010dbc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 --2- 172.21.15.77:0/1375919043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f00fcff0 0x7f40f010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 >> 172.21.15.77:0/1375919043 conn(0x7f40f00f74c0 msgr2=0x7f40f0066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 shutdown_connections 2024-04-26T05:01:51.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.397+0000 7f40f5207700 1 -- 172.21.15.77:0/1375919043 wait complete. 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 -- 172.21.15.77:0/3242006113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 msgr2=0x7faefc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 --2- 172.21.15.77:0/3242006113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc0fd3d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faeec004940 tx=0x7faeec009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 -- 172.21.15.77:0/3242006113 shutdown_connections 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 --2- 172.21.15.77:0/3242006113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc0fd3d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 --2- 172.21.15.77:0/3242006113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 0x7faefc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.498+0000 7faf03933700 1 -- 172.21.15.77:0/3242006113 >> 172.21.15.77:0/3242006113 conn(0x7faefc0f7430 msgr2=0x7faefc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 -- 172.21.15.77:0/3242006113 shutdown_connections 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 -- 172.21.15.77:0/3242006113 wait complete. 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 Processor -- start 2024-04-26T05:01:51.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 -- start start 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 0x7faefc10db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc10e640 con 0x7faefc0fcf60 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf03933700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc10edf0 con 0x7faefc0fbcb0 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf00ece700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf00ece700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42750/0 (socket says 172.21.15.77:42750) 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf00ece700 1 -- 172.21.15.77:0/3278755973 learned_addr learned my addr 172.21.15.77:0/3278755973 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.499+0000 7faf016cf700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 0x7faefc10db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf00ece700 1 -- 172.21.15.77:0/3278755973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 msgr2=0x7faefc10db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf00ece700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 0x7faefc10db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf00ece700 1 -- 172.21.15.77:0/3278755973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec005040 con 0x7faefc0fcf60 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf00ece700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faeec00b3b0 tx=0x7faeec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeec00e040 con 0x7faefc0fcf60 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeec00bbf0 con 0x7faefc0fcf60 2024-04-26T05:01:51.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc10f070 con 0x7faefc0fcf60 2024-04-26T05:01:51.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeec01c630 con 0x7faefc0fcf60 2024-04-26T05:01:51.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.500+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc10f5b0 con 0x7faefc0fcf60 2024-04-26T05:01:51.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.501+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc101af0 con 0x7faefc0fcf60 2024-04-26T05:01:51.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.502+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faeec00bd60 con 0x7faefc0fcf60 2024-04-26T05:01:51.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.502+0000 7faef1ffb700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 0x7faee80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.502+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7faeec08e7e0 con 0x7faefc0fcf60 2024-04-26T05:01:51.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.502+0000 7faf016cf700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 0x7faee80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.503+0000 7faf016cf700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 0x7faee80705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef8004f60 tx=0x7faef800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.507+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faeec05b610 con 0x7faefc0fcf60 2024-04-26T05:01:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:51 smithi078 ceph-mon[79689]: Upgrade: Updating prometheus.smithi077 2024-04-26T05:01:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:51 smithi078 ceph-mon[79689]: Deploying daemon prometheus.smithi077 on smithi077 2024-04-26T05:01:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:51 smithi078 ceph-mon[79689]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.715+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faefc10f7d0 con 0x7faee806e140 2024-04-26T05:01:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.726+0000 7faef1ffb700 1 -- 172.21.15.77:0/3278755973 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7faefc10f7d0 con 0x7faee806e140 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (8m) 8s ago 17m 24.8M - 0.25.0 c8568f914cd2 2230041c4b50 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (5m) 8s ago 17m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (5m) 9s ago 16m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 error 8s ago 17m - - 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (104s) 8s ago 11m 85.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (74s) 8s ago 11m 15.2M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (28s) 9s ago 11m 12.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a25782a60fae 2024-04-26T05:01:51.727 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (49s) 9s ago 11m 14.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 04b98ae4596a 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (7m) 8s ago 18m 509M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (6m) 9s ago 16m 453M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (6m) 8s ago 18m 53.3M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (6m) 9s ago 16m 39.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (17s) 8s ago 17m 8568k - 1.5.0 0da6a335fe13 5e33f11c0332 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (12s) 9s ago 16m 3607k - 1.5.0 0da6a335fe13 ea333f6d1319 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (5m) 8s ago 15m 95.9M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (4m) 8s ago 15m 99.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (3m) 8s ago 14m 95.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (3m) 9s ago 14m 117M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (2m) 9s ago 13m 92.8M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (2m) 9s ago 13m 86.3M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:01:51.728 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (6m) 8s ago 16m 61.1M - 2.43.0 a07b618ecd1d ffd6c30ddb1d 2024-04-26T05:01:51.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.727+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 msgr2=0x7faee80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.727+0000 7faf03933700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 0x7faee80705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef8004f60 tx=0x7faef800b040 comp rx=0 tx=0).stop 2024-04-26T05:01:51.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 msgr2=0x7faefc10e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faeec00b3b0 tx=0x7faeec009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 shutdown_connections 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faee806e140 0x7faee80705f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faefc0fbcb0 0x7faefc10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 --2- 172.21.15.77:0/3278755973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faefc0fcf60 0x7faefc10e040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 >> 172.21.15.77:0/3278755973 conn(0x7faefc0f7430 msgr2=0x7faefc065c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 shutdown_connections 2024-04-26T05:01:51.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.728+0000 7faf03933700 1 -- 172.21.15.77:0/3278755973 wait complete. 2024-04-26T05:01:51.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3197174309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 msgr2=0x7fb0040fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3197174309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb0040fc150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faff4004940 tx=0x7faff4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:51.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3197174309 shutdown_connections 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3197174309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 0x7fb0040fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3197174309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb0040fc150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3197174309 >> 172.21.15.77:0/3197174309 conn(0x7fb0040f74c0 msgr2=0x7fb0040f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3197174309 shutdown_connections 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3197174309 wait complete. 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.829+0000 7fb00a3b9700 1 Processor -- start 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 -- start start 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 0x7fb004112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004066c50 con 0x7fb0040fbd40 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004066d90 con 0x7fb0040fcff0 2024-04-26T05:01:51.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42770/0 (socket says 172.21.15.77:42770) 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 -- 172.21.15.77:0/3385272762 learned_addr learned my addr 172.21.15.77:0/3385272762 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb0037fe700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 0x7fb004112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 -- 172.21.15.77:0/3385272762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 msgr2=0x7fb004112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 0x7fb004112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 -- 172.21.15.77:0/3385272762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4005040 con 0x7fb0040fbd40 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb003fff700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faff4009150 tx=0x7faff4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff400e040 con 0x7fb0040fbd40 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faff400bcc0 con 0x7fb0040fbd40 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.830+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb004113280 con 0x7fb0040fbd40 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.831+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff401c630 con 0x7fb0040fbd40 2024-04-26T05:01:51.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.831+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb004113810 con 0x7fb0040fbd40 2024-04-26T05:01:51.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.832+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00405f5c0 con 0x7fb0040fbd40 2024-04-26T05:01:51.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.832+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faff4013420 con 0x7fb0040fbd40 2024-04-26T05:01:51.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.832+0000 7fb000ff9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 0x7fafec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:51.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.833+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7faff408e8f0 con 0x7fb0040fbd40 2024-04-26T05:01:51.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.833+0000 7fb0037fe700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 0x7fafec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:51.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.833+0000 7fb0037fe700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 0x7fafec0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faff8006fd0 tx=0x7faff8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:51.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:51.837+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faff4057bf0 con 0x7fb0040fbd40 2024-04-26T05:01:52.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.130+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb004113af0 con 0x7fb0040fbd40 2024-04-26T05:01:52.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.131+0000 7fb000ff9700 1 -- 172.21.15.77:0/3385272762 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7faff405b2b0 con 0x7fb0040fbd40 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:01:52.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:01:52.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 msgr2=0x7fafec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 0x7fafec0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faff8006fd0 tx=0x7faff8008040 comp rx=0 tx=0).stop 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 msgr2=0x7fb004110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faff4009150 tx=0x7faff4009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 shutdown_connections 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fafec06e140 0x7fafec0705f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0040fbd40 0x7fb004110730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 --2- 172.21.15.77:0/3385272762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0040fcff0 0x7fb004112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 >> 172.21.15.77:0/3385272762 conn(0x7fb0040f74c0 msgr2=0x7fb0040f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 shutdown_connections 2024-04-26T05:01:52.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.133+0000 7fb00a3b9700 1 -- 172.21.15.77:0/3385272762 wait complete. 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.235+0000 7f76374b4700 1 -- 172.21.15.77:0/593455078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 msgr2=0x7f76300fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.235+0000 7f76374b4700 1 --2- 172.21.15.77:0/593455078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f76300fe840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f762c004940 tx=0x7f762c009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- 172.21.15.77:0/593455078 shutdown_connections 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 --2- 172.21.15.77:0/593455078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 0x7f76300faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 --2- 172.21.15.77:0/593455078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f76300fe840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- 172.21.15.77:0/593455078 >> 172.21.15.77:0/593455078 conn(0x7f76300f6bd0 msgr2=0x7f76300f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- 172.21.15.77:0/593455078 shutdown_connections 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- 172.21.15.77:0/593455078 wait complete. 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 Processor -- start 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- start start 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 0x7f763010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763010e460 con 0x7f76300fe430 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f76374b4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763010e5d0 con 0x7f76300fed80 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.237+0000 7f7635250700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42782/0 (socket says 172.21.15.77:42782) 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 -- 172.21.15.77:0/1427782973 learned_addr learned my addr 172.21.15.77:0/1427782973 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7634a4f700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 0x7f763010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 -- 172.21.15.77:0/1427782973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 msgr2=0x7f763010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 0x7f763010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 -- 172.21.15.77:0/1427782973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c005040 con 0x7f76300fe430 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7635250700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b3b0 tx=0x7f762c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762c00e040 con 0x7f76300fe430 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f762c00bbf0 con 0x7f76300fe430 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762c01c630 con 0x7f76300fe430 2024-04-26T05:01:52.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7630112fd0 con 0x7f76300fe430 2024-04-26T05:01:52.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7630113500 con 0x7f76300fe430 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.238+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763004c710 con 0x7f76300fe430 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.240+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f762c00bd60 con 0x7f76300fe430 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.240+0000 7f7625ffb700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 0x7f761c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.240+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f762c08e730 con 0x7f76300fe430 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.240+0000 7f7634a4f700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 0x7f761c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.240+0000 7f7634a4f700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 0x7f761c0705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7620004f40 tx=0x7f762000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.244+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f762c05b600 con 0x7f76300fe430 2024-04-26T05:01:52.322 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:52 smithi077 ceph-mon[91833]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.495+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76301058e0 con 0x7f76300fe430 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.495+0000 7f7625ffb700 1 -- 172.21.15.77:0/1427782973 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1935 (secure 0 0 0) 0x7f762c05b190 con 0x7f76300fe430 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stdout:e36 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:01:52.496 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 36 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:01:26.009556+0000 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:01:52.497 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 84 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=34376} 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:01:52.498 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 34376 members: 34376 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:34376} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:52.499 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6826/1653368163,v1:172.21.15.78:6827/1653368163] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 msgr2=0x7f761c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 0x7f761c0705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7620004f40 tx=0x7f762000b040 comp rx=0 tx=0).stop 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 msgr2=0x7f763010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b3b0 tx=0x7f762c009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 shutdown_connections 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f761c06e140 0x7f761c0705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76300fe430 0x7f763010d920 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 --2- 172.21.15.77:0/1427782973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76300fed80 0x7f763010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 >> 172.21.15.77:0/1427782973 conn(0x7f76300f6bd0 msgr2=0x7f76300f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.497+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 shutdown_connections 2024-04-26T05:01:52.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.498+0000 7f76374b4700 1 -- 172.21.15.77:0/1427782973 wait complete. 2024-04-26T05:01:52.501 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-04-26T05:01:52.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:52 smithi078 ceph-mon[79689]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:52.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.599+0000 7f9e3399d700 1 -- 172.21.15.77:0/821372146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 msgr2=0x7f9e2c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.599+0000 7f9e3399d700 1 --2- 172.21.15.77:0/821372146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c0fd3d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009ce0 tx=0x7f9e1c004d10 comp rx=0 tx=0).stop 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 -- 172.21.15.77:0/821372146 shutdown_connections 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 --2- 172.21.15.77:0/821372146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c0fd3d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 --2- 172.21.15.77:0/821372146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 -- 172.21.15.77:0/821372146 >> 172.21.15.77:0/821372146 conn(0x7f9e2c0f7430 msgr2=0x7f9e2c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 -- 172.21.15.77:0/821372146 shutdown_connections 2024-04-26T05:01:52.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.600+0000 7f9e3399d700 1 -- 172.21.15.77:0/821372146 wait complete. 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 Processor -- start 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 -- start start 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c110790 con 0x7f9e2c0fcf60 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e3399d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c110f70 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e30f38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e31739700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e30f38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42806/0 (socket says 172.21.15.77:42806) 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e31739700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:60190/0 (socket says 172.21.15.77:60190) 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.601+0000 7f9e30f38700 1 -- 172.21.15.77:0/3765698284 learned_addr learned my addr 172.21.15.77:0/3765698284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e31739700 1 -- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 msgr2=0x7f9e2c110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e31739700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e31739700 1 -- 172.21.15.77:0/3765698284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c009940 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e31739700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009fd0 tx=0x7f9e2400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e24004e90 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c1111f0 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e2400f040 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e2400ca50 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.602+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c111730 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.603+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c1018f0 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.604+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9e2400cc00 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.604+0000 7f9e29ffb700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 0x7f9e14070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.604+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f9e2408d650 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.604+0000 7f9e30f38700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 0x7f9e14070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.605+0000 7f9e30f38700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 0x7f9e14070600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c003400 tx=0x7f9e1c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.609+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mon.1 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9e2405a4b0 con 0x7f9e2c0fbcb0 2024-04-26T05:01:52.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.828+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2c111950 con 0x7f9e1406e150 2024-04-26T05:01:52.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.830+0000 7f9e29ffb700 1 -- 172.21.15.77:0/3765698284 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f9e2c111950 con 0x7f9e1406e150 2024-04-26T05:01:52.830 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:01:52.830 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T05:01:52.830 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T05:01:52.830 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "osd", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mon", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "mds" 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "16/21 daemons upgraded", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:01:52.831 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:01:52.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 msgr2=0x7f9e14070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 0x7f9e14070600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c003400 tx=0x7f9e1c004d10 comp rx=0 tx=0).stop 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 msgr2=0x7f9e2c10fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009fd0 tx=0x7f9e2400b040 comp rx=0 tx=0).stop 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 shutdown_connections 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f9e1406e150 0x7f9e14070600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e2c0fbcb0 0x7f9e2c10fc50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 --2- 172.21.15.77:0/3765698284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e2c0fcf60 0x7f9e2c110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 >> 172.21.15.77:0/3765698284 conn(0x7f9e2c0f7430 msgr2=0x7f9e2c065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 shutdown_connections 2024-04-26T05:01:52.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.832+0000 7f9e3399d700 1 -- 172.21.15.77:0/3765698284 wait complete. 2024-04-26T05:01:52.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 -- 172.21.15.77:0/4157825993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fce60 msgr2=0x7f6b2c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 --2- 172.21.15.77:0/4157825993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c0fd270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b28004940 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 -- 172.21.15.77:0/4157825993 shutdown_connections 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 --2- 172.21.15.77:0/4157825993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 --2- 172.21.15.77:0/4157825993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c0fd270 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.933+0000 7f6b336d3700 1 -- 172.21.15.77:0/4157825993 >> 172.21.15.77:0/4157825993 conn(0x7f6b2c0f8540 msgr2=0x7f6b2c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.934+0000 7f6b336d3700 1 -- 172.21.15.77:0/4157825993 shutdown_connections 2024-04-26T05:01:52.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.934+0000 7f6b336d3700 1 -- 172.21.15.77:0/4157825993 wait complete. 2024-04-26T05:01:52.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.934+0000 7f6b336d3700 1 Processor -- start 2024-04-26T05:01:52.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.934+0000 7f6b336d3700 1 -- start start 2024-04-26T05:01:52.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b336d3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b336d3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b336d3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c10e470 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b336d3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c10e5e0 con 0x7f6b2c0fce60 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42824/0 (socket says 172.21.15.77:42824) 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 -- 172.21.15.77:0/1680216367 learned_addr learned my addr 172.21.15.77:0/1680216367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b3146f700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 -- 172.21.15.77:0/1680216367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fce60 msgr2=0x7f6b2c10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 -- 172.21.15.77:0/1680216367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28005040 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.935+0000 7f6b30c6e700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002a50 tx=0x7f6b1c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.936+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c007cc0 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.936+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b1c00f040 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.936+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c113040 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.936+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c00da70 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.936+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c1135d0 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.937+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c04c710 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.937+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6b1c00dc10 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.937+0000 7f6b21ffb700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 0x7f6b180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.938+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6b1c08d270 con 0x7f6b2c0fdfd0 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.938+0000 7f6b3146f700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 0x7f6b180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:01:52.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.938+0000 7f6b3146f700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 0x7f6b180705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b970 tx=0x7f6b28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:01:52.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:52.942+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6b1c05a020 con 0x7f6b2c0fdfd0 2024-04-26T05:01:53.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.223+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6b2c1058e0 con 0x7f6b2c0fdfd0 2024-04-26T05:01:53.225 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:53 smithi077 ceph-mon[91833]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:53.225 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:53 smithi077 ceph-mon[91833]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:53.225 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:53 smithi077 ceph-mon[91833]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:53.225 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:53 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3385272762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:53.225 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:53 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1427782973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:01:53.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.223+0000 7f6b21ffb700 1 -- 172.21.15.77:0/1680216367 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+345 (secure 0 0 0) 0x7f6b1c059bb0 con 0x7f6b2c0fdfd0 2024-04-26T05:01:53.225 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T05:01:53.226 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:01:53.226 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in error state 2024-04-26T05:01:53.226 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:01:53.226 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:01:53.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 msgr2=0x7f6b180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 0x7f6b180705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b970 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 msgr2=0x7f6b2c10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002a50 tx=0x7f6b1c0046b0 comp rx=0 tx=0).stop 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 shutdown_connections 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6b1806e140 0x7f6b180705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b2c0fce60 0x7f6b2c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 --2- 172.21.15.77:0/1680216367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c0fdfd0 0x7f6b2c10de70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 >> 172.21.15.77:0/1680216367 conn(0x7f6b2c0f8540 msgr2=0x7f6b2c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 shutdown_connections 2024-04-26T05:01:53.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:01:53.226+0000 7f6b336d3700 1 -- 172.21.15.77:0/1680216367 wait complete. 2024-04-26T05:01:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:53 smithi078 ceph-mon[79689]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:53 smithi078 ceph-mon[79689]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:53 smithi078 ceph-mon[79689]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:53 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3385272762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:01:53.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:53 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1427782973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:01:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:54 smithi077 ceph-mon[91833]: from='client.44357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:54.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:54 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1680216367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:01:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:54 smithi078 ceph-mon[79689]: from='client.44357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:01:54.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:54 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1680216367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:01:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:55 smithi077 ceph-mon[91833]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:55 smithi078 ceph-mon[79689]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:56.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:56.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:56.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:56 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:56.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:56 smithi077 ceph-mon[91833]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:57.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:57.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:01:57.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:56 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:01:57.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:56 smithi078 ceph-mon[79689]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:59.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:01:59 smithi077 ceph-mon[91833]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:01:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:01:59 smithi078 ceph-mon[79689]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:00.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:00.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:00.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:00 smithi077 ceph-mon[91833]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:01.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:00 smithi078 ceph-mon[79689]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:01.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:02.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:02.993 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:02 smithi077 ceph-mon[91833]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:03.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:02 smithi078 ceph-mon[79689]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.244 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:05 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:05.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:05 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:06.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:06 smithi077 ceph-mon[91833]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:06.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:06 smithi077 ceph-mon[91833]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:06.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:06 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T05:02:06.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:06 smithi078 ceph-mon[79689]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:06.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:06 smithi078 ceph-mon[79689]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:06.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:06 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T05:02:07.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:07 smithi077 ceph-mon[91833]: Upgrade: Updating alertmanager.smithi077 2024-04-26T05:02:07.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:07 smithi077 ceph-mon[91833]: Deploying daemon alertmanager.smithi077 on smithi077 2024-04-26T05:02:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:07 smithi078 ceph-mon[79689]: Upgrade: Updating alertmanager.smithi077 2024-04-26T05:02:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:07 smithi078 ceph-mon[79689]: Deploying daemon alertmanager.smithi077 on smithi077 2024-04-26T05:02:08.297 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:08 smithi077 ceph-mon[91833]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:08.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:08 smithi078 ceph-mon[79689]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:09.444 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:09 smithi077 ceph-mon[91833]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:09.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:09 smithi078 ceph-mon[79689]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:11.370 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:11 smithi077 ceph-mon[91833]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:11.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:11 smithi078 ceph-mon[79689]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:12.693 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:12.693 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:12.693 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:12 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:13.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:12 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:13.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:13 smithi077 ceph-mon[91833]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:14.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:13 smithi078 ceph-mon[79689]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:15.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:15 smithi077 ceph-mon[91833]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:16.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:15 smithi078 ceph-mon[79689]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:16.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:17.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:17.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:17.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:17.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:17 smithi077 ceph-mon[91833]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:17.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:17 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:17.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:17 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:17 smithi078 ceph-mon[79689]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:17 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:18.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:17 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:19.902 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:19 smithi077 ceph-mon[91833]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:20.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:19 smithi078 ceph-mon[79689]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.414 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.414 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.415 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.416 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:21 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T05:02:21.768 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi077:9093"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.769 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:21.770 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:21 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:22.680 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:22 smithi077 ceph-mon[91833]: Upgrade: Updating grafana.smithi077 2024-04-26T05:02:22.681 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:22 smithi077 ceph-mon[91833]: Deploying daemon grafana.smithi077 on smithi077 2024-04-26T05:02:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:22 smithi078 ceph-mon[79689]: Upgrade: Updating grafana.smithi077 2024-04-26T05:02:22.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:22 smithi078 ceph-mon[79689]: Deploying daemon grafana.smithi077 on smithi077 2024-04-26T05:02:23.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- 172.21.15.77:0/514792980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fb460 msgr2=0x7f4b880fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 --2- 172.21.15.77:0/514792980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fb460 0x7f4b880fb870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b84004940 tx=0x7f4b84009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- 172.21.15.77:0/514792980 shutdown_connections 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 --2- 172.21.15.77:0/514792980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fc710 0x7f4b880fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 --2- 172.21.15.77:0/514792980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fb460 0x7f4b880fb870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- 172.21.15.77:0/514792980 >> 172.21.15.77:0/514792980 conn(0x7f4b880f6be0 msgr2=0x7f4b880f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- 172.21.15.77:0/514792980 shutdown_connections 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- 172.21.15.77:0/514792980 wait complete. 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 Processor -- start 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.331+0000 7f4b90706700 1 -- start start 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fb460 0x7f4b88102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88112c90 con 0x7f4b880fc710 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88112dd0 con 0x7f4b880fb460 2024-04-26T05:02:23.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49158/0 (socket says 172.21.15.77:49158) 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 -- 172.21.15.77:0/4162181488 learned_addr learned my addr 172.21.15.77:0/4162181488 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8e4a2700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fb460 0x7f4b88102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 -- 172.21.15.77:0/4162181488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fb460 msgr2=0x7f4b88102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fb460 0x7f4b88102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 -- 172.21.15.77:0/4162181488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84005040 con 0x7f4b880fc710 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b8dca1700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b790 tx=0x7f4b78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7800f6f0 con 0x7f4b880fc710 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b7800fd20 con 0x7f4b880fc710 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b881130b0 con 0x7f4b880fc710 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b780103a0 con 0x7f4b880fc710 2024-04-26T05:02:23.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.332+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b88113670 con 0x7f4b880fc710 2024-04-26T05:02:23.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.333+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8804c710 con 0x7f4b880fc710 2024-04-26T05:02:23.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.334+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b78011400 con 0x7f4b880fc710 2024-04-26T05:02:23.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.334+0000 7f4b7effd700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 0x7f4b740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.334+0000 7f4b8e4a2700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 0x7f4b740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.334+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4b7808d9d0 con 0x7f4b880fc710 2024-04-26T05:02:23.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.335+0000 7f4b8e4a2700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 0x7f4b740705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400b970 tx=0x7f4b84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:23.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.339+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b7805a7e0 con 0x7f4b880fc710 2024-04-26T05:02:23.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.560+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b881009b0 con 0x7f4b7406e140 2024-04-26T05:02:23.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.561+0000 7f4b7effd700 1 -- 172.21.15.77:0/4162181488 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f4b881009b0 con 0x7f4b7406e140 2024-04-26T05:02:23.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 msgr2=0x7f4b740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 0x7f4b740705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400b970 tx=0x7f4b84009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:23.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 msgr2=0x7f4b881126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b790 tx=0x7f4b78009250 comp rx=0 tx=0).stop 2024-04-26T05:02:23.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 shutdown_connections 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f4b7406e140 0x7f4b740705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b880fb460 0x7f4b88102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 --2- 172.21.15.77:0/4162181488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b880fc710 0x7f4b881126c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 >> 172.21.15.77:0/4162181488 conn(0x7f4b880f6be0 msgr2=0x7f4b880f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 shutdown_connections 2024-04-26T05:02:23.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.563+0000 7f4b90706700 1 -- 172.21.15.77:0/4162181488 wait complete. 2024-04-26T05:02:23.575 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:02:23.598 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:23 smithi077 ceph-mon[91833]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:23.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 -- 172.21.15.77:0/374325520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 msgr2=0x7f62d40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 --2- 172.21.15.77:0/374325520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d40fe440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62c0004940 tx=0x7f62c0009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:23.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 -- 172.21.15.77:0/374325520 shutdown_connections 2024-04-26T05:02:23.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 --2- 172.21.15.77:0/374325520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d40fe440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 --2- 172.21.15.77:0/374325520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 0x7f62d40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 -- 172.21.15.77:0/374325520 >> 172.21.15.77:0/374325520 conn(0x7f62d40f8540 msgr2=0x7f62d40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 -- 172.21.15.77:0/374325520 shutdown_connections 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.666+0000 7f62daec4700 1 -- 172.21.15.77:0/374325520 wait complete. 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 Processor -- start 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 -- start start 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 0x7f62d410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d410e450 con 0x7f62d40fdfd0 2024-04-26T05:02:23.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62daec4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d410e5c0 con 0x7f62d40fce60 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49170/0 (socket says 172.21.15.77:49170) 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 -- 172.21.15.77:0/2840163445 learned_addr learned my addr 172.21.15.77:0/2840163445 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d8c60700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 0x7f62d410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 -- 172.21.15.77:0/2840163445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 msgr2=0x7f62d410d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 0x7f62d410d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.667+0000 7f62d3fff700 1 -- 172.21.15.77:0/2840163445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0005040 con 0x7f62d40fdfd0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62d3fff700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b3b0 tx=0x7f62c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c000e040 con 0x7f62d40fdfd0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c000bcc0 con 0x7f62d40fdfd0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d4112fc0 con 0x7f62d40fdfd0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c001c630 con 0x7f62d40fdfd0 2024-04-26T05:02:23.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.668+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d4113550 con 0x7f62d40fdfd0 2024-04-26T05:02:23.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.669+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d404c710 con 0x7f62d40fdfd0 2024-04-26T05:02:23.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.669+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f62c0013980 con 0x7f62d40fdfd0 2024-04-26T05:02:23.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.670+0000 7f62d17fa700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 0x7f62c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:23.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.670+0000 7f62d8c60700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 0x7f62c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:23.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.670+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f62c008e810 con 0x7f62d40fdfd0 2024-04-26T05:02:23.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.670+0000 7f62d8c60700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 0x7f62c40705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62c8004990 tx=0x7f62c8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:23.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.674+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f62c005b640 con 0x7f62d40fdfd0 2024-04-26T05:02:23.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:23 smithi078 ceph-mon[79689]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:23.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.893+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62d4102390 con 0x7f62c406e140 2024-04-26T05:02:23.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.894+0000 7f62d17fa700 1 -- 172.21.15.77:0/2840163445 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f62d4102390 con 0x7f62c406e140 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 msgr2=0x7f62c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 0x7f62c40705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62c8004990 tx=0x7f62c8009ee0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 msgr2=0x7f62d410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b3b0 tx=0x7f62c0009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 shutdown_connections 2024-04-26T05:02:23.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f62c406e140 0x7f62c40705f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d40fce60 0x7f62d410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 --2- 172.21.15.77:0/2840163445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62d40fdfd0 0x7f62d410de50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:23.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 >> 172.21.15.77:0/2840163445 conn(0x7f62d40f8540 msgr2=0x7f62d40f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:23.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 shutdown_connections 2024-04-26T05:02:23.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.896+0000 7f62daec4700 1 -- 172.21.15.77:0/2840163445 wait complete. 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 -- 172.21.15.77:0/2097055712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 msgr2=0x7f37f80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 --2- 172.21.15.77:0/2097055712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f80fc0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009ce0 tx=0x7f37f0004d10 comp rx=0 tx=0).stop 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 -- 172.21.15.77:0/2097055712 shutdown_connections 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 --2- 172.21.15.77:0/2097055712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 0x7f37f80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 --2- 172.21.15.77:0/2097055712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f80fc0c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 -- 172.21.15.77:0/2097055712 >> 172.21.15.77:0/2097055712 conn(0x7f37f80f7430 msgr2=0x7f37f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 -- 172.21.15.77:0/2097055712 shutdown_connections 2024-04-26T05:02:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:23.999+0000 7f37fd78c700 1 -- 172.21.15.77:0/2097055712 wait complete. 2024-04-26T05:02:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 Processor -- start 2024-04-26T05:02:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 -- start start 2024-04-26T05:02:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 0x7f37f8112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8066330 con 0x7f37f80fbcb0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37fd78c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8066470 con 0x7f37f80fcf60 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49196/0 (socket says 172.21.15.77:49196) 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 -- 172.21.15.77:0/3468263970 learned_addr learned my addr 172.21.15.77:0/3468263970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f67fc700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 0x7f37f8112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 -- 172.21.15.77:0/3468263970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 msgr2=0x7f37f8112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 0x7f37f8112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 -- 172.21.15.77:0/3468263970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0009940 con 0x7f37f80fbcb0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.000+0000 7f37f6ffd700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37f0004740 tx=0x7f37f0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.001+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37f000e040 con 0x7f37f80fbcb0 2024-04-26T05:02:24.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.001+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f8113320 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.001+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37f00128e0 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.001+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37f001c9a0 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.001+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f81138e0 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.002+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f805f5c0 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.002+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f37f0012a50 con 0x7f37f80fbcb0 2024-04-26T05:02:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.002+0000 7f37df7fe700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 0x7f37e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.003+0000 7f37f67fc700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 0x7f37e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.003+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f37f008f8b0 con 0x7f37f80fbcb0 2024-04-26T05:02:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.003+0000 7f37f67fc700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 0x7f37e00705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0041d0 tx=0x7f37ec0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.007+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f37f0058b30 con 0x7f37f80fbcb0 2024-04-26T05:02:24.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.229+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37f8101320 con 0x7f37e006e140 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.238+0000 7f37df7fe700 1 -- 172.21.15.77:0/3468263970 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f37f8101320 con 0x7f37e006e140 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (13s) 8s ago 17m 15.9M - 0.25.0 c8568f914cd2 7b04d2f66749 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (5m) 8s ago 18m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (5m) 42s ago 17m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 error 8s ago 17m - - 2024-04-26T05:02:24.240 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (2m) 8s ago 12m 85.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (107s) 8s ago 12m 15.4M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (60s) 42s ago 12m 12.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a25782a60fae 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (82s) 42s ago 12m 14.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 04b98ae4596a 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (8m) 8s ago 19m 511M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (7m) 42s ago 16m 453M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (7m) 8s ago 19m 56.0M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (6m) 42s ago 16m 39.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (49s) 8s ago 17m 8887k - 1.5.0 0da6a335fe13 5e33f11c0332 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (45s) 42s ago 16m 3607k - 1.5.0 0da6a335fe13 ea333f6d1319 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (5m) 8s ago 16m 96.1M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:02:24.241 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (4m) 8s ago 15m 99.4M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (4m) 8s ago 15m 95.7M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (3m) 42s ago 14m 117M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (3m) 42s ago 14m 92.8M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (2m) 42s ago 13m 86.3M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (29s) 8s ago 17m 44.7M - 2.43.0 a07b618ecd1d 8ef737e04a1f 2024-04-26T05:02:24.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 msgr2=0x7f37e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 0x7f37e00705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0041d0 tx=0x7f37ec0043c0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 msgr2=0x7f37f8110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37f0004740 tx=0x7f37f0004d10 comp rx=0 tx=0).stop 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 shutdown_connections 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f37e006e140 0x7f37e00705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37f80fbcb0 0x7f37f8110890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 --2- 172.21.15.77:0/3468263970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f80fcf60 0x7f37f8112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 >> 172.21.15.77:0/3468263970 conn(0x7f37f80f7430 msgr2=0x7f37f80f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 shutdown_connections 2024-04-26T05:02:24.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.241+0000 7f37fd78c700 1 -- 172.21.15.77:0/3468263970 wait complete. 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 -- 172.21.15.77:0/1881278692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fce60 msgr2=0x7f5aac0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 --2- 172.21.15.77:0/1881278692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fce60 0x7f5aac0fd270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0004940 tx=0x7f5aa0009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 -- 172.21.15.77:0/1881278692 shutdown_connections 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 --2- 172.21.15.77:0/1881278692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 --2- 172.21.15.77:0/1881278692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fce60 0x7f5aac0fd270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 -- 172.21.15.77:0/1881278692 >> 172.21.15.77:0/1881278692 conn(0x7f5aac0f8540 msgr2=0x7f5aac0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 -- 172.21.15.77:0/1881278692 shutdown_connections 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 -- 172.21.15.77:0/1881278692 wait complete. 2024-04-26T05:02:24.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.343+0000 7f5ab232f700 1 Processor -- start 2024-04-26T05:02:24.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 -- start start 2024-04-26T05:02:24.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 0x7f5aac10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac10e5e0 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac10edd0 con 0x7f5aac0fce60 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49220/0 (socket says 172.21.15.77:49220) 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 -- 172.21.15.77:0/3497899049 learned_addr learned my addr 172.21.15.77:0/3497899049 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aabfff700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 0x7f5aac10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 -- 172.21.15.77:0/3497899049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 msgr2=0x7f5aac10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 0x7f5aac10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 -- 172.21.15.77:0/3497899049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0005040 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aabfff700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 0x7f5aac10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aab7fe700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b790 tx=0x7f5a9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a9c00f770 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac10f0b0 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a9c00fda0 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a9c00d310 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.344+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac10f670 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.345+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac05f5c0 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.346+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a9c010420 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.346+0000 7f5aa8ff9700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 0x7f5a940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.346+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5a9c08dc40 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.346+0000 7f5aabfff700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 0x7f5a940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.347+0000 7f5aabfff700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 0x7f5a940705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b3b0 tx=0x7f5aa0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.351+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5a9c05aa70 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.465 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:24 smithi077 ceph-mon[91833]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:24.465 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:24 smithi077 ceph-mon[91833]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:24.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.638+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5aac10f8f0 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.638+0000 7f5aa8ff9700 1 -- 172.21.15.77:0/3497899049 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5a9c05a600 con 0x7f5aac0fdfd0 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:02:24.643 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:02:24.644 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 msgr2=0x7f5a940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 0x7f5a940705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b3b0 tx=0x7f5aa0009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 msgr2=0x7f5aac10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b790 tx=0x7f5a9c009250 comp rx=0 tx=0).stop 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 shutdown_connections 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f5a9406e140 0x7f5a940705f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aac0fce60 0x7f5aac10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 --2- 172.21.15.77:0/3497899049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5aac0fdfd0 0x7f5aac10df50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 >> 172.21.15.77:0/3497899049 conn(0x7f5aac0f8540 msgr2=0x7f5aac0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 shutdown_connections 2024-04-26T05:02:24.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.641+0000 7f5ab232f700 1 -- 172.21.15.77:0/3497899049 wait complete. 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 -- 172.21.15.77:0/359363667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fcf70 msgr2=0x7fd2680fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 --2- 172.21.15.77:0/359363667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fcf70 0x7fd2680fd3e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd258004940 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 -- 172.21.15.77:0/359363667 shutdown_connections 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 --2- 172.21.15.77:0/359363667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fcf70 0x7fd2680fd3e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 --2- 172.21.15.77:0/359363667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fbcc0 0x7fd2680fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 -- 172.21.15.77:0/359363667 >> 172.21.15.77:0/359363667 conn(0x7fd2680f7420 msgr2=0x7fd2680f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 -- 172.21.15.77:0/359363667 shutdown_connections 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.741+0000 7fd2706a5700 1 -- 172.21.15.77:0/359363667 wait complete. 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 Processor -- start 2024-04-26T05:02:24.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 -- start start 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fcf70 0x7fd268115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268066ab0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd2706a5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268066bf0 con 0x7fd2680fcf70 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49246/0 (socket says 172.21.15.77:49246) 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 -- 172.21.15.77:0/1359611218 learned_addr learned my addr 172.21.15.77:0/1359611218 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26dc40700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fcf70 0x7fd268115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 -- 172.21.15.77:0/1359611218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fcf70 msgr2=0x7fd268115060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fcf70 0x7fd268115060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 -- 172.21.15.77:0/1359611218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258005040 con 0x7fd2680fbcc0 2024-04-26T05:02:24.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.742+0000 7fd26e441700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c8c0 tx=0x7fd2640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.743+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26400a980 con 0x7fd2680fbcc0 2024-04-26T05:02:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.743+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26400ce60 con 0x7fd2680fbcc0 2024-04-26T05:02:24.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.743+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2681155a0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.743+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2640115e0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.743+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681159c0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.744+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26805f5c0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.745+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd264012400 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.745+0000 7fd25effd700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 0x7fd254070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.745+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd26408dc20 con 0x7fd2680fbcc0 2024-04-26T05:02:24.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.745+0000 7fd26dc40700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 0x7fd254070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.746+0000 7fd26dc40700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 0x7fd254070600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd258004f90 tx=0x7fd258009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.750+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd264056fc0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:24 smithi078 ceph-mon[79689]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:24.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:24 smithi078 ceph-mon[79689]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.996+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd26804bed0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.996+0000 7fd25effd700 1 -- 172.21.15.77:0/1359611218 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1935 (secure 0 0 0) 0x7fd2640126b0 con 0x7fd2680fbcc0 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:e36 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:02:24.998 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 36 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:01:26.009556+0000 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 84 2024-04-26T05:02:24.999 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=34376} 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:02:25.000 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 34376 members: 34376 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:34376} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:02:25.001 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6826/1653368163,v1:172.21.15.78:6827/1653368163] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 msgr2=0x7fd254070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 0x7fd254070600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd258004f90 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 msgr2=0x7fd268112b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c8c0 tx=0x7fd2640044c0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 shutdown_connections 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd25406e150 0x7fd254070600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2680fbcc0 0x7fd268112b10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 --2- 172.21.15.77:0/1359611218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2680fcf70 0x7fd268115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 >> 172.21.15.77:0/1359611218 conn(0x7fd2680f7420 msgr2=0x7fd2680f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 shutdown_connections 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:24.998+0000 7fd2706a5700 1 -- 172.21.15.77:0/1359611218 wait complete. 2024-04-26T05:02:25.002 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-04-26T05:02:25.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 -- 172.21.15.77:0/3252291878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fce60 msgr2=0x7fd7580fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 --2- 172.21.15.77:0/3252291878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fce60 0x7fd7580fd270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004940 tx=0x7fd74c009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 -- 172.21.15.77:0/3252291878 shutdown_connections 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 --2- 172.21.15.77:0/3252291878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fdfd0 0x7fd7580fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 --2- 172.21.15.77:0/3252291878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fce60 0x7fd7580fd270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 -- 172.21.15.77:0/3252291878 >> 172.21.15.77:0/3252291878 conn(0x7fd7580f8540 msgr2=0x7fd7580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 -- 172.21.15.77:0/3252291878 shutdown_connections 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.100+0000 7fd75eb39700 1 -- 172.21.15.77:0/3252291878 wait complete. 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 Processor -- start 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 -- start start 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fce60 0x7fd758071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758072200 con 0x7fd7580fdfd0 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd75eb39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580707f0 con 0x7fd7580fce60 2024-04-26T05:02:25.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49262/0 (socket says 172.21.15.77:49262) 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.101+0000 7fd757fff700 1 -- 172.21.15.77:0/2847688756 learned_addr learned my addr 172.21.15.77:0/2847688756 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd75c8d5700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fce60 0x7fd758071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd757fff700 1 -- 172.21.15.77:0/2847688756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fce60 msgr2=0x7fd758071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd757fff700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fce60 0x7fd758071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd757fff700 1 -- 172.21.15.77:0/2847688756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c005040 con 0x7fd7580fdfd0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd757fff700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd744002a50 tx=0x7fd7440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd744007cc0 con 0x7fd7580fdfd0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd74400f040 con 0x7fd7580fdfd0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758070aa0 con 0x7fd7580fdfd0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd744009640 con 0x7fd7580fdfd0 2024-04-26T05:02:25.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.102+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758071060 con 0x7fd7580fdfd0 2024-04-26T05:02:25.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.103+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75804c710 con 0x7fd7580fdfd0 2024-04-26T05:02:25.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.103+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd7440097c0 con 0x7fd7580fdfd0 2024-04-26T05:02:25.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.104+0000 7fd7557fa700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 0x7fd7480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.104+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd74408d9c0 con 0x7fd7580fdfd0 2024-04-26T05:02:25.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.104+0000 7fd75c8d5700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 0x7fd7480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.104+0000 7fd75c8d5700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 0x7fd7480705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3b0 tx=0x7fd74c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:25.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.109+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd74405a770 con 0x7fd7580fdfd0 2024-04-26T05:02:25.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.335+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd758102330 con 0x7fd74806e140 2024-04-26T05:02:25.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.336+0000 7fd7557fa700 1 -- 172.21.15.77:0/2847688756 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fd758102330 con 0x7fd74806e140 2024-04-26T05:02:25.337 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": true, 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [ 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "osd", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "mon", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr", 2024-04-26T05:02:25.338 INFO:teuthology.orchestra.run.smithi077.stdout: "mds" 2024-04-26T05:02:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-04-26T05:02:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": "16/21 daemons upgraded", 2024-04-26T05:02:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "Currently upgrading grafana daemons", 2024-04-26T05:02:25.339 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:02:25.339 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 msgr2=0x7fd7480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 0x7fd7480705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3b0 tx=0x7fd74c009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 msgr2=0x7fd7580702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd744002a50 tx=0x7fd7440046b0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 shutdown_connections 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd74806e140 0x7fd7480705f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7580fce60 0x7fd758071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 --2- 172.21.15.77:0/2847688756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7580fdfd0 0x7fd7580702b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 >> 172.21.15.77:0/2847688756 conn(0x7fd7580f8540 msgr2=0x7fd7580f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 shutdown_connections 2024-04-26T05:02:25.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.338+0000 7fd75eb39700 1 -- 172.21.15.77:0/2847688756 wait complete. 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 -- 172.21.15.77:0/3285077839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fbcb0 msgr2=0x7f10e00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 --2- 172.21.15.77:0/3285077839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fbcb0 0x7f10e00fc0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10dc004940 tx=0x7f10dc009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 -- 172.21.15.77:0/3285077839 shutdown_connections 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 --2- 172.21.15.77:0/3285077839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fcf60 0x7f10e00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 --2- 172.21.15.77:0/3285077839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fbcb0 0x7f10e00fc0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 -- 172.21.15.77:0/3285077839 >> 172.21.15.77:0/3285077839 conn(0x7f10e00f7430 msgr2=0x7f10e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 -- 172.21.15.77:0/3285077839 shutdown_connections 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.450+0000 7f10e875f700 1 -- 172.21.15.77:0/3285077839 wait complete. 2024-04-26T05:02:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 Processor -- start 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 -- start start 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fbcb0 0x7f10e0110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0066330 con 0x7f10e00fcf60 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e875f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0066470 con 0x7f10e00fbcb0 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e64fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fbcb0 0x7f10e0110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49276/0 (socket says 172.21.15.77:49276) 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 -- 172.21.15.77:0/3664772174 learned_addr learned my addr 172.21.15.77:0/3664772174 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 -- 172.21.15.77:0/3664772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fbcb0 msgr2=0x7f10e0110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fbcb0 0x7f10e0110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 -- 172.21.15.77:0/3664772174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc005040 con 0x7f10e00fcf60 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.451+0000 7f10e5cfa700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c830 tx=0x7f10d0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.452+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d00106f0 con 0x7f10e00fcf60 2024-04-26T05:02:25.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.452+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10d0010d20 con 0x7f10e00fcf60 2024-04-26T05:02:25.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.452+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e010c560 con 0x7f10e00fcf60 2024-04-26T05:02:25.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.452+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d00113a0 con 0x7f10e00fcf60 2024-04-26T05:02:25.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.452+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e010cb20 con 0x7f10e00fcf60 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.453+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e005f5c0 con 0x7f10e00fcf60 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.453+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f10d0012400 con 0x7f10e00fcf60 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.454+0000 7f10d6ffd700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 0x7f10cc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.454+0000 7f10e64fb700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 0x7f10cc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.454+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f10d008dba0 con 0x7f10e00fcf60 2024-04-26T05:02:25.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.454+0000 7f10e64fb700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 0x7f10cc070600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:25.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.458+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f10d0056ea0 con 0x7f10e00fcf60 2024-04-26T05:02:25.569 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:25 smithi077 ceph-mon[91833]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:25.569 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:25 smithi077 ceph-mon[91833]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:25.569 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:25 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3497899049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:25.569 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:25 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1359611218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:02:25.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:25 smithi078 ceph-mon[79689]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:25.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:25 smithi078 ceph-mon[79689]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:25.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:25 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3497899049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:25.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:25 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1359611218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.766+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f10e010ce50 con 0x7f10e00fcf60 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.766+0000 7f10d6ffd700 1 -- 172.21.15.77:0/3664772174 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+345 (secure 0 0 0) 0x7f10d005a560 con 0x7f10e00fcf60 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem with deprecated feature inline_data 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: daemon grafana.smithi077 on smithi077 is in error state 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:02:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:02:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 msgr2=0x7f10cc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 0x7f10cc070600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10dc000c00 tx=0x7f10dc009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 msgr2=0x7f10e0112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c830 tx=0x7f10d0004470 comp rx=0 tx=0).stop 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 shutdown_connections 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f10cc06e150 0x7f10cc070600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e00fbcb0 0x7f10e0110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 --2- 172.21.15.77:0/3664772174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e00fcf60 0x7f10e0112c70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 >> 172.21.15.77:0/3664772174 conn(0x7f10e00f7430 msgr2=0x7f10e00f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 shutdown_connections 2024-04-26T05:02:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:25.769+0000 7f10e875f700 1 -- 172.21.15.77:0/3664772174 wait complete. 2024-04-26T05:02:26.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:26 smithi077 ceph-mon[91833]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:26.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:26 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3664772174' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:02:26.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:26 smithi078 ceph-mon[79689]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:26.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:26 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3664772174' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:02:27.433 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:27 smithi077 ceph-mon[91833]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:27.433 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:27.433 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:27.433 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:27 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:27.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:27 smithi078 ceph-mon[79689]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:27.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:27.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:27.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:27 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:29.678 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:29 smithi077 ceph-mon[91833]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:29.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:29 smithi078 ceph-mon[79689]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:31.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:31 smithi077 ceph-mon[91833]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:31.743 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:31 smithi078 ceph-mon[79689]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:31.767 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:33 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:33.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:33 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:33 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:33 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:34.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:34 smithi077 ceph-mon[91833]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:34.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:34 smithi078 ceph-mon[79689]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:36.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:36.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.495 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.496 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T05:02:36.497 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.498 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:36.499 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T05:02:36.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-26T05:02:36.520 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T05:02:36.521 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:36.522 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: Upgrade: Finalizing container_image settings 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: Upgrade: Complete! 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:37 smithi077 ceph-mon[91833]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi077:3000"}]: dispatch 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: Upgrade: Finalizing container_image settings 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: Upgrade: Complete! 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:02:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:02:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:37.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:37 smithi078 ceph-mon[79689]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-26T05:02:39.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:39 smithi077 ceph-mon[91833]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:39 smithi078 ceph-mon[79689]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:41.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:41 smithi077 ceph-mon[91833]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:41 smithi078 ceph-mon[79689]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:43 smithi077 ceph-mon[91833]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:43 smithi078 ceph-mon[79689]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:45.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:45 smithi077 ceph-mon[91833]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:45.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:45 smithi078 ceph-mon[79689]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:46 smithi077 ceph-mon[91833]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:47.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:46 smithi078 ceph-mon[79689]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:02:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:02:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:48 smithi077 ceph-mon[91833]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:48 smithi078 ceph-mon[79689]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:50 smithi077 ceph-mon[91833]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:50 smithi078 ceph-mon[79689]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:53.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:52 smithi077 ceph-mon[91833]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:52 smithi078 ceph-mon[79689]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:54 smithi077 ceph-mon[91833]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:54 smithi078 ceph-mon[79689]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:55.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 -- 172.21.15.77:0/3298385982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fdfd0 msgr2=0x7f98c40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 --2- 172.21.15.77:0/3298385982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fdfd0 0x7f98c40fe440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004940 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 -- 172.21.15.77:0/3298385982 shutdown_connections 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 --2- 172.21.15.77:0/3298385982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fdfd0 0x7f98c40fe440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 --2- 172.21.15.77:0/3298385982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fce60 0x7f98c40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 -- 172.21.15.77:0/3298385982 >> 172.21.15.77:0/3298385982 conn(0x7f98c40f8540 msgr2=0x7f98c40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 -- 172.21.15.77:0/3298385982 shutdown_connections 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.885+0000 7f98c8d96700 1 -- 172.21.15.77:0/3298385982 wait complete. 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 Processor -- start 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 -- start start 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fdfd0 0x7f98c410ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c410e3c0 con 0x7f98c40fce60 2024-04-26T05:02:55.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c8d96700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c410e530 con 0x7f98c40fdfd0 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41536/0 (socket says 172.21.15.77:41536) 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 -- 172.21.15.77:0/2831696064 learned_addr learned my addr 172.21.15.77:0/2831696064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c1d9b700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fdfd0 0x7f98c410ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 -- 172.21.15.77:0/2831696064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fdfd0 msgr2=0x7f98c410ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fdfd0 0x7f98c410ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 -- 172.21.15.77:0/2831696064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8005040 con 0x7f98c40fce60 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.886+0000 7f98c259c700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00ec90 tx=0x7f98bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.887+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc00eea0 con 0x7f98c40fce60 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.887+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c4112f90 con 0x7f98c40fce60 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.887+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98bc00f040 con 0x7f98c40fce60 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.887+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc009640 con 0x7f98c40fce60 2024-04-26T05:02:55.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.887+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c4113440 con 0x7f98c40fce60 2024-04-26T05:02:55.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.888+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c404c710 con 0x7f98c40fce60 2024-04-26T05:02:55.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.888+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98bc00c940 con 0x7f98c40fce60 2024-04-26T05:02:55.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.888+0000 7f98b2ffd700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 0x7f98ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.889+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f98bc08e1a0 con 0x7f98c40fce60 2024-04-26T05:02:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.889+0000 7f98c1d9b700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 0x7f98ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:55.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.889+0000 7f98c1d9b700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 0x7f98ac0705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009b30 tx=0x7f98b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:55.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:55.893+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f98bc05af50 con 0x7f98c40fce60 2024-04-26T05:02:56.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.111+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98c41022a0 con 0x7f98ac06e140 2024-04-26T05:02:56.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.112+0000 7f98b2ffd700 1 -- 172.21.15.77:0/2831696064 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f98c41022a0 con 0x7f98ac06e140 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 msgr2=0x7f98ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 0x7f98ac0705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009b30 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 msgr2=0x7f98c410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00ec90 tx=0x7f98bc00b040 comp rx=0 tx=0).stop 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 shutdown_connections 2024-04-26T05:02:56.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f98ac06e140 0x7f98ac0705f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98c40fce60 0x7f98c410d880 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 --2- 172.21.15.77:0/2831696064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98c40fdfd0 0x7f98c410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.114+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 >> 172.21.15.77:0/2831696064 conn(0x7f98c40f8540 msgr2=0x7f98c40f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.115+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 shutdown_connections 2024-04-26T05:02:56.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:56.115+0000 7f98c8d96700 1 -- 172.21.15.77:0/2831696064 wait complete. 2024-04-26T05:02:56.897 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch ps' 2024-04-26T05:02:57.195 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:02:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:56 smithi077 ceph-mon[91833]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:56 smithi078 ceph-mon[79689]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:58.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:57 smithi077 ceph-mon[91833]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:58.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:57 smithi078 ceph-mon[79689]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:02:58.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 -- 172.21.15.77:0/3708758737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fb7f0 msgr2=0x7f20840fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 --2- 172.21.15.77:0/3708758737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fb7f0 0x7f20840fbbc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2078004940 tx=0x7f2078009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 -- 172.21.15.77:0/3708758737 shutdown_connections 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 --2- 172.21.15.77:0/3708758737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fc190 0x7f2084101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 --2- 172.21.15.77:0/3708758737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fb7f0 0x7f20840fbbc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 -- 172.21.15.77:0/3708758737 >> 172.21.15.77:0/3708758737 conn(0x7f2084071460 msgr2=0x7f2084071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 -- 172.21.15.77:0/3708758737 shutdown_connections 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.931+0000 7f208a6f4700 1 -- 172.21.15.77:0/3708758737 wait complete. 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 Processor -- start 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 -- start start 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 0x7f2084113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20841145e0 con 0x7f20840fc190 2024-04-26T05:02:58.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.932+0000 7f208a6f4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208410dc00 con 0x7f20840fb7f0 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f2083fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 0x7f2084113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46008/0 (socket says 172.21.15.77:46008) 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f2083fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 0x7f2084113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.77:55128/0 (socket says 172.21.15.77:55128) 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 -- 172.21.15.77:0/697597926 learned_addr learned my addr 172.21.15.77:0/697597926 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 -- 172.21.15.77:0/697597926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 msgr2=0x7f2084113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 0x7f2084113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 -- 172.21.15.77:0/697597926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078005040 con 0x7f20840fc190 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f20837fe700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2074009b80 tx=0x7f2074004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2074004e70 con 0x7f20840fc190 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208410dee0 con 0x7f20840fc190 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f207400f040 con 0x7f20840fc190 2024-04-26T05:02:58.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.933+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f207400da60 con 0x7f20840fc190 2024-04-26T05:02:58.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.934+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208410e420 con 0x7f20840fc190 2024-04-26T05:02:58.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.935+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2084065f30 con 0x7f20840fc190 2024-04-26T05:02:58.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.935+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2074007cc0 con 0x7f20840fc190 2024-04-26T05:02:58.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.935+0000 7f2080ff9700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 0x7f206c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:02:58.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.935+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f207408d8b0 con 0x7f20840fc190 2024-04-26T05:02:58.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.935+0000 7f2083fff700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 0x7f206c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:02:58.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.936+0000 7f2083fff700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 0x7f206c0705f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2078004f90 tx=0x7f2078009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:02:58.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:58.940+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2074056bb0 con 0x7f20840fc190 2024-04-26T05:02:59.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.147+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2084109f60 con 0x7f206c06e140 2024-04-26T05:02:59.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.157+0000 7f2080ff9700 1 -- 172.21.15.77:0/697597926 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2084109f60 con 0x7f206c06e140 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (48s) 28s ago 18m 15.9M - 0.25.0 c8568f914cd2 7b04d2f66749 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (6m) 28s ago 18m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (6m) 77s ago 17m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (33s) 28s ago 18m 55.1M - 9.4.12 2bacad6d85d8 31fd925532b7 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (2m) 28s ago 13m 85.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (2m) 28s ago 12m 15.5M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (95s) 77s ago 12m 12.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a25782a60fae 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (116s) 77s ago 12m 14.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 04b98ae4596a 2024-04-26T05:02:59.158 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (8m) 28s ago 19m 517M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (7m) 77s ago 17m 453M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (7m) 28s ago 19m 57.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (7m) 77s ago 17m 39.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (84s) 28s ago 18m 8296k - 1.5.0 0da6a335fe13 5e33f11c0332 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (80s) 77s ago 17m 3607k - 1.5.0 0da6a335fe13 ea333f6d1319 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (6m) 28s ago 16m 96.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (5m) 28s ago 16m 99.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (4m) 28s ago 15m 95.7M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (4m) 77s ago 15m 117M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (3m) 77s ago 14m 92.8M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (3m) 77s ago 14m 86.3M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:02:59.159 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (63s) 28s ago 17m 44.9M - 2.43.0 a07b618ecd1d 8ef737e04a1f 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 msgr2=0x7f206c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 0x7f206c0705f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2078004f90 tx=0x7f2078009e30 comp rx=0 tx=0).stop 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 msgr2=0x7f2084113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2074009b80 tx=0x7f2074004600 comp rx=0 tx=0).stop 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 shutdown_connections 2024-04-26T05:02:59.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206c06e140 0x7f206c0705f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:59.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20840fb7f0 0x7f2084113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:59.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 --2- 172.21.15.77:0/697597926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20840fc190 0x7f2084113f90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:02:59.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 >> 172.21.15.77:0/697597926 conn(0x7f2084071460 msgr2=0x7f20840f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:02:59.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 shutdown_connections 2024-04-26T05:02:59.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:02:59.159+0000 7f208a6f4700 1 -- 172.21.15.77:0/697597926 wait complete. 2024-04-26T05:02:59.194 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:02:58 smithi077 ceph-mon[91833]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:02:58 smithi078 ceph-mon[79689]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:02:59.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade status' 2024-04-26T05:03:00.117 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:03:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:00 smithi077 ceph-mon[91833]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:03:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:00 smithi077 ceph-mon[91833]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:00 smithi078 ceph-mon[79689]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:03:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:00 smithi078 ceph-mon[79689]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/2443442700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d40fb520 msgr2=0x7fa1d40fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/2443442700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fb990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4004940 tx=0x7fa1c4009e30 comp rx=0 tx=0).stop 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/2443442700 shutdown_connections 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/2443442700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fb990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/2443442700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d4102f60 0x7fa1d4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/2443442700 >> 172.21.15.77:0/2443442700 conn(0x7fa1d40714d0 msgr2=0x7fa1d40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:01.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/2443442700 shutdown_connections 2024-04-26T05:03:01.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.823+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/2443442700 wait complete. 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 Processor -- start 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 -- start start 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40fe6a0 con 0x7fa1d4102f60 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d9f6b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40fd180 con 0x7fa1d40fb520 2024-04-26T05:03:01.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.824+0000 7fa1d2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46022/0 (socket says 172.21.15.77:46022) 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 -- 172.21.15.77:0/91372634 learned_addr learned my addr 172.21.15.77:0/91372634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d37fe700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 -- 172.21.15.77:0/91372634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d40fb520 msgr2=0x7fa1d40fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 -- 172.21.15.77:0/91372634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4005040 con 0x7fa1d4102f60 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d2ffd700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b3b0 tx=0x7fa1c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1c400e040 con 0x7fa1d4102f60 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d40fd3d0 con 0x7fa1d4102f60 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1c400bbc0 con 0x7fa1d4102f60 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1c401c3f0 con 0x7fa1d4102f60 2024-04-26T05:03:01.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.825+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d411a160 con 0x7fa1d4102f60 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.826+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d4100250 con 0x7fa1d4102f60 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.827+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa1c400bd30 con 0x7fa1d4102f60 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.827+0000 7fa1cbfff700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 0x7fa1c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.827+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa1c408ee70 con 0x7fa1d4102f60 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.827+0000 7fa1d37fe700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 0x7fa1c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:01.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.828+0000 7fa1d37fe700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 0x7fa1c00705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc004990 tx=0x7fa1bc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:01.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:01.832+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa1c40580f0 con 0x7fa1d4102f60 2024-04-26T05:03:02.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.050+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1d4109fe0 con 0x7fa1c006e140 2024-04-26T05:03:02.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.051+0000 7fa1cbfff700 1 -- 172.21.15.77:0/91372634 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa1d4109fe0 con 0x7fa1c006e140 2024-04-26T05:03:02.051 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "target_image": null, 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "in_progress": false, 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "which": "", 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "services_complete": [], 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "progress": null, 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "message": "", 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout: "is_paused": false 2024-04-26T05:03:02.052 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 msgr2=0x7fa1c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 0x7fa1c00705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc004990 tx=0x7fa1bc009ee0 comp rx=0 tx=0).stop 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 msgr2=0x7fa1d40fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b3b0 tx=0x7fa1c4009e30 comp rx=0 tx=0).stop 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 shutdown_connections 2024-04-26T05:03:02.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa1c006e140 0x7fa1c00705f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:02.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1d40fb520 0x7fa1d40fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:02.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 --2- 172.21.15.77:0/91372634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d4102f60 0x7fa1d40fcc40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:02.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 >> 172.21.15.77:0/91372634 conn(0x7fa1d40714d0 msgr2=0x7fa1d40fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:02.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 shutdown_connections 2024-04-26T05:03:02.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:02.053+0000 7fa1d9f6b700 1 -- 172.21.15.77:0/91372634 wait complete. 2024-04-26T05:03:02.778 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph health detail' 2024-04-26T05:03:03.030 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:03:03.152 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:02 smithi077 ceph-mon[91833]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:03.152 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:02 smithi077 ceph-mon[91833]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:03:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:02 smithi078 ceph-mon[79689]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:02 smithi078 ceph-mon[79689]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 -- 172.21.15.77:0/614617086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd624101d90 msgr2=0x7fd624102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 --2- 172.21.15.77:0/614617086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd624101d90 0x7fd624102160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009ce0 tx=0x7fd61c004d10 comp rx=0 tx=0).stop 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 -- 172.21.15.77:0/614617086 shutdown_connections 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 --2- 172.21.15.77:0/614617086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6240fbd40 0x7fd6240fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 --2- 172.21.15.77:0/614617086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd624101d90 0x7fd624102160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 -- 172.21.15.77:0/614617086 >> 172.21.15.77:0/614617086 conn(0x7fd624071d00 msgr2=0x7fd624072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 -- 172.21.15.77:0/614617086 shutdown_connections 2024-04-26T05:03:04.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.677+0000 7fd6299b1700 1 -- 172.21.15.77:0/614617086 wait complete. 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 Processor -- start 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 -- start start 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624101d90 0x7fd62410f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62410f760 con 0x7fd6240fbd40 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6299b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62410f8a0 con 0x7fd624101d90 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd622ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd622ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46034/0 (socket says 172.21.15.77:46034) 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd622ffd700 1 -- 172.21.15.77:0/3683209204 learned_addr learned my addr 172.21.15.77:0/3683209204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.678+0000 7fd6227fc700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624101d90 0x7fd62410f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd622ffd700 1 -- 172.21.15.77:0/3683209204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624101d90 msgr2=0x7fd62410f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd622ffd700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624101d90 0x7fd62410f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd622ffd700 1 -- 172.21.15.77:0/3683209204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c009940 con 0x7fd6240fbd40 2024-04-26T05:03:04.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd622ffd700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ba50 tx=0x7fd61c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd61c00e040 con 0x7fd6240fbd40 2024-04-26T05:03:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd624117b90 con 0x7fd6240fbd40 2024-04-26T05:03:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd61c01cb20 con 0x7fd6240fbd40 2024-04-26T05:03:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd61c012b00 con 0x7fd6240fbd40 2024-04-26T05:03:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.679+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624118120 con 0x7fd6240fbd40 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.680+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6240fd430 con 0x7fd6240fbd40 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.680+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd61c01cc90 con 0x7fd6240fbd40 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.681+0000 7fd61b7fe700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 0x7fd6100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.681+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd61c08ff80 con 0x7fd6240fbd40 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.681+0000 7fd6227fc700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 0x7fd6100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:04.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.681+0000 7fd6227fc700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 0x7fd6100705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd60c004600 tx=0x7fd60c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:04.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.686+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd61c059200 con 0x7fd6240fbd40 2024-04-26T05:03:04.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.965+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd62405f5c0 con 0x7fd6240fbd40 2024-04-26T05:03:04.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.965+0000 7fd61b7fe700 1 -- 172.21.15.77:0/3683209204 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd61c05c8c0 con 0x7fd6240fbd40 2024-04-26T05:03:04.966 INFO:teuthology.orchestra.run.smithi077.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-26T05:03:04.966 INFO:teuthology.orchestra.run.smithi077.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-26T05:03:04.966 INFO:teuthology.orchestra.run.smithi077.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 msgr2=0x7fd6100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 0x7fd6100705f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd60c004600 tx=0x7fd60c00b040 comp rx=0 tx=0).stop 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 msgr2=0x7fd62410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ba50 tx=0x7fd61c004d10 comp rx=0 tx=0).stop 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 shutdown_connections 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd61006e140 0x7fd6100705f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6240fbd40 0x7fd62410ece0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 --2- 172.21.15.77:0/3683209204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624101d90 0x7fd62410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 >> 172.21.15.77:0/3683209204 conn(0x7fd624071d00 msgr2=0x7fd6240fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:04.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 shutdown_connections 2024-04-26T05:03:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:04.967+0000 7fd6299b1700 1 -- 172.21.15.77:0/3683209204 wait complete. 2024-04-26T05:03:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:04 smithi077 ceph-mon[91833]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:04 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3683209204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:03:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:04 smithi078 ceph-mon[79689]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:04 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3683209204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T05:03:05.687 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions' 2024-04-26T05:03:05.941 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:03:07.059 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:06 smithi077 ceph-mon[91833]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:06 smithi078 ceph-mon[79689]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 -- 172.21.15.77:0/1591725026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 msgr2=0x7fbc100fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 --2- 172.21.15.77:0/1591725026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc100fc1b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc00004940 tx=0x7fbc00009e30 comp rx=0 tx=0).stop 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 -- 172.21.15.77:0/1591725026 shutdown_connections 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 --2- 172.21.15.77:0/1591725026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc100fc1b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 --2- 172.21.15.77:0/1591725026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 0x7fbc10102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 -- 172.21.15.77:0/1591725026 >> 172.21.15.77:0/1591725026 conn(0x7fbc10071d00 msgr2=0x7fbc10072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 -- 172.21.15.77:0/1591725026 shutdown_connections 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.592+0000 7fbc183a7700 1 -- 172.21.15.77:0/1591725026 wait complete. 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 Processor -- start 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 -- start start 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 0x7fbc101140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10114740 con 0x7fbc100fbd40 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc183a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1010db30 con 0x7fbc10101d90 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc16143700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc16143700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42942/0 (socket says 172.21.15.77:42942) 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc16143700 1 -- 172.21.15.77:0/2922773174 learned_addr learned my addr 172.21.15.77:0/2922773174 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.593+0000 7fbc15942700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 0x7fbc101140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:07.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc16143700 1 -- 172.21.15.77:0/2922773174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 msgr2=0x7fbc101140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc16143700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 0x7fbc101140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc16143700 1 -- 172.21.15.77:0/2922773174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00005040 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc16143700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc08002a50 tx=0x7fbc080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc08007cc0 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1010de10 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc0800f040 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0800da30 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.594+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1010e3d0 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.595+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc100fd430 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.595+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbc0800dbb0 con 0x7fbc100fbd40 2024-04-26T05:03:07.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.596+0000 7fbc06ffd700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 0x7fbbfc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:03:07.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.596+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fbc0808d9c0 con 0x7fbc100fbd40 2024-04-26T05:03:07.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.596+0000 7fbc15942700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 0x7fbbfc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:03:07.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.596+0000 7fbc15942700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 0x7fbbfc070600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc1010ecf0 tx=0x7fbc00009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:03:07.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.600+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbc0805a770 con 0x7fbc100fbd40 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.882+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc1010f260 con 0x7fbc100fbd40 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.882+0000 7fbc06ffd700 1 -- 172.21.15.77:0/2922773174 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbc0805a300 con 0x7fbc100fbd40 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:03:07.883 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:03:07.884 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:03:07.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 msgr2=0x7fbbfc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:07.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 0x7fbbfc070600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc1010ecf0 tx=0x7fbc00009e30 comp rx=0 tx=0).stop 2024-04-26T05:03:07.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 msgr2=0x7fbc10113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:03:07.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc08002a50 tx=0x7fbc080046b0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 shutdown_connections 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbbfc06e150 0x7fbbfc070600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc100fbd40 0x7fbc10113bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 --2- 172.21.15.77:0/2922773174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc10101d90 0x7fbc101140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 >> 172.21.15.77:0/2922773174 conn(0x7fbc10071d00 msgr2=0x7fbc100655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 shutdown_connections 2024-04-26T05:03:07.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:03:07.884+0000 7fbc183a7700 1 -- 172.21.15.77:0/2922773174 wait complete. 2024-04-26T05:03:08.540 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-26T05:03:08.795 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:03:08.918 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:08 smithi077 ceph-mon[91833]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:08.919 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:08 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2922773174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:03:09.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:08 smithi078 ceph-mon[79689]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:09.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:08 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2922773174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:03:10.435 INFO:teuthology.orchestra.run.smithi077.stdout:wait for servicemap items w/ changing names to refresh 2024-04-26T05:03:11.047 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:10 smithi077 ceph-mon[91833]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:11.073 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'sleep 60' 2024-04-26T05:03:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:10 smithi078 ceph-mon[79689]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:03:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:12 smithi077 ceph-mon[91833]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:12 smithi078 ceph-mon[79689]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:14 smithi077 ceph-mon[91833]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:14 smithi078 ceph-mon[79689]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:16 smithi077 ceph-mon[91833]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:16 smithi078 ceph-mon[79689]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:19.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:18 smithi077 ceph-mon[91833]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:18 smithi078 ceph-mon[79689]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:21.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:20 smithi077 ceph-mon[91833]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:20 smithi078 ceph-mon[79689]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:22 smithi077 ceph-mon[91833]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:22 smithi078 ceph-mon[79689]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:24 smithi077 ceph-mon[91833]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:24 smithi078 ceph-mon[79689]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:26 smithi077 ceph-mon[91833]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:26 smithi078 ceph-mon[79689]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:28 smithi077 ceph-mon[91833]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:28 smithi078 ceph-mon[79689]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:30 smithi077 ceph-mon[91833]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:30 smithi078 ceph-mon[79689]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:32 smithi077 ceph-mon[91833]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:32 smithi078 ceph-mon[79689]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:35.245 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:34 smithi077 ceph-mon[91833]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:34 smithi078 ceph-mon[79689]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:36 smithi077 ceph-mon[91833]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:36 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:03:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:36 smithi078 ceph-mon[79689]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:36 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:03:38.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:03:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:03:38.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:03:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:03:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:03:38.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:03:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:38 smithi077 ceph-mon[91833]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:38 smithi078 ceph-mon[79689]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:40 smithi077 ceph-mon[91833]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:40 smithi078 ceph-mon[79689]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:42 smithi077 ceph-mon[91833]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:42 smithi078 ceph-mon[79689]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:45.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:44 smithi077 ceph-mon[91833]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:44 smithi078 ceph-mon[79689]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:46 smithi077 ceph-mon[91833]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:46 smithi078 ceph-mon[79689]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:03:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:48 smithi077 ceph-mon[91833]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:48 smithi078 ceph-mon[79689]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:51.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:50 smithi077 ceph-mon[91833]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:50 smithi078 ceph-mon[79689]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:53.223 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:52 smithi077 ceph-mon[91833]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:52 smithi078 ceph-mon[79689]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:54 smithi077 ceph-mon[91833]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:54 smithi078 ceph-mon[79689]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:56 smithi077 ceph-mon[91833]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:56 smithi078 ceph-mon[79689]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:03:58 smithi077 ceph-mon[91833]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:03:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:03:58 smithi078 ceph-mon[79689]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:00 smithi077 ceph-mon[91833]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:00 smithi078 ceph-mon[79689]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:02.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:02.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:02 smithi077 ceph-mon[91833]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:02 smithi078 ceph-mon[79689]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:04 smithi077 ceph-mon[91833]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:04 smithi078 ceph-mon[79689]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:06 smithi077 ceph-mon[91833]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:06 smithi078 ceph-mon[79689]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:08 smithi077 ceph-mon[91833]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:08 smithi078 ceph-mon[79689]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:11.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:10 smithi077 ceph-mon[91833]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:10 smithi078 ceph-mon[79689]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:12 smithi077 ceph-mon[91833]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:12 smithi078 ceph-mon[79689]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:13.728 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch ps' 2024-04-26T05:04:13.990 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:04:15.086 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:14 smithi077 ceph-mon[91833]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:14 smithi078 ceph-mon[79689]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:15.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 -- 172.21.15.77:0/2024571214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fbbe0 msgr2=0x7feacc0fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 --2- 172.21.15.77:0/2024571214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fbbe0 0x7feacc0fbfb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feabc004940 tx=0x7feabc009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 -- 172.21.15.77:0/2024571214 shutdown_connections 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 --2- 172.21.15.77:0/2024571214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fc4f0 0x7feacc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 --2- 172.21.15.77:0/2024571214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fbbe0 0x7feacc0fbfb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 -- 172.21.15.77:0/2024571214 >> 172.21.15.77:0/2024571214 conn(0x7feacc0f7870 msgr2=0x7feacc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 -- 172.21.15.77:0/2024571214 shutdown_connections 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 -- 172.21.15.77:0/2024571214 wait complete. 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.621+0000 7fead1f18700 1 Processor -- start 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7fead1f18700 1 -- start start 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7fead1f18700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fbbe0 0x7feacc115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7fead1f18700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7fead1f18700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1167a0 con 0x7feacc0fc4f0 2024-04-26T05:04:15.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7fead1f18700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc10fcf0 con 0x7feacc0fbbe0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50968/0 (socket says 172.21.15.77:50968) 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 -- 172.21.15.77:0/2420600054 learned_addr learned my addr 172.21.15.77:0/2420600054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacb7fe700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fbbe0 0x7feacc115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 -- 172.21.15.77:0/2420600054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fbbe0 msgr2=0x7feacc115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fbbe0 0x7feacc115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 -- 172.21.15.77:0/2420600054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc005040 con 0x7feacc0fc4f0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feacaffd700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feac000b790 tx=0x7feac0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.622+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac000f6f0 con 0x7feacc0fc4f0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.623+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feac000fd20 con 0x7feacc0fc4f0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.623+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac00103a0 con 0x7feacc0fc4f0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.623+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc10ff70 con 0x7feacc0fc4f0 2024-04-26T05:04:15.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.623+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc1104e0 con 0x7feacc0fc4f0 2024-04-26T05:04:15.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.623+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc066ab0 con 0x7feacc0fc4f0 2024-04-26T05:04:15.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.624+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feac0011400 con 0x7feacc0fc4f0 2024-04-26T05:04:15.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.624+0000 7feab3fff700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 0x7feab40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:15.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.625+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7feac008da10 con 0x7feacc0fc4f0 2024-04-26T05:04:15.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.625+0000 7feacb7fe700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 0x7feab40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:15.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.625+0000 7feacb7fe700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 0x7feab40705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3b0 tx=0x7feabc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:15.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.629+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feac005a840 con 0x7feacc0fc4f0 2024-04-26T05:04:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.837+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 --> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feacc10c180 con 0x7feab406e140 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.846+0000 7feab3fff700 1 -- 172.21.15.77:0/2420600054 <== mgr.34104 v2:172.21.15.77:6800/1604716213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7feacc10c180 con 0x7feab406e140 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:alertmanager.smithi077 smithi077 *:9093,9094 running (2m) 104s ago 19m 15.9M - 0.25.0 c8568f914cd2 7b04d2f66749 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi077 smithi077 running (7m) 104s ago 19m 6656k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b5d2cf292ff2 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:crash.smithi078 smithi078 running (7m) 2m ago 18m 6660k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 b0ad488fceb3 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:grafana.smithi077 smithi077 *:3000 running (109s) 104s ago 19m 55.1M - 9.4.12 2bacad6d85d8 31fd925532b7 2024-04-26T05:04:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.quaiyg smithi077 running (4m) 104s ago 14m 85.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 cd5b0ac57a88 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi077.urxtgp smithi077 running (3m) 104s ago 14m 15.5M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 bf3b6a9bcd50 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.nihewy smithi078 running (2m) 2m ago 14m 12.6M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a25782a60fae 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mds.cephfs.smithi078.wkrgkp smithi078 running (3m) 2m ago 14m 14.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 04b98ae4596a 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi077.hkvsor smithi077 *:8443,9283,8765 running (10m) 104s ago 21m 517M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 5fa3007ae5b6 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.smithi078.navvuw smithi078 *:8443,9283,8765 running (9m) 2m ago 18m 453M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 812ad1f1d8e5 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi077 smithi077 running (8m) 104s ago 21m 57.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d3426e141588 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:mon.smithi078 smithi078 running (8m) 2m ago 18m 39.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ca912136bf31 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi077 smithi077 *:9100 running (2m) 104s ago 19m 8296k - 1.5.0 0da6a335fe13 5e33f11c0332 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:node-exporter.smithi078 smithi078 *:9100 running (2m) 2m ago 18m 3607k - 1.5.0 0da6a335fe13 ea333f6d1319 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (7m) 104s ago 17m 96.2M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 ecb573fb9bdd 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi077 running (6m) 104s ago 17m 99.5M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 60022e2ec48e 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi077 running (6m) 104s ago 17m 95.7M 1557M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 9ad85529d167 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi078 running (5m) 2m ago 16m 117M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a56a9fe0bde4 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi078 running (5m) 2m ago 16m 92.8M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 61aaf454ddae 2024-04-26T05:04:15.849 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi078 running (4m) 2m ago 15m 86.3M 2581M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 2f3ab9eeb39b 2024-04-26T05:04:15.850 INFO:teuthology.orchestra.run.smithi077.stdout:prometheus.smithi077 smithi077 *:9095 running (2m) 104s ago 19m 44.9M - 2.43.0 a07b618ecd1d 8ef737e04a1f 2024-04-26T05:04:15.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.848+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 msgr2=0x7feab40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.848+0000 7fead1f18700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 0x7feab40705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3b0 tx=0x7feabc009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.848+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 msgr2=0x7feacc1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.848+0000 7fead1f18700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feac000b790 tx=0x7feac0009250 comp rx=0 tx=0).stop 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 shutdown_connections 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7feab406e140 0x7feab40705f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feacc0fbbe0 0x7feacc115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 --2- 172.21.15.77:0/2420600054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feacc0fc4f0 0x7feacc1160c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 >> 172.21.15.77:0/2420600054 conn(0x7feacc0f7870 msgr2=0x7feacc0f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 shutdown_connections 2024-04-26T05:04:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:15.849+0000 7fead1f18700 1 -- 172.21.15.77:0/2420600054 wait complete. 2024-04-26T05:04:16.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:16.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions' 2024-04-26T05:04:16.827 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:04:17.116 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:16 smithi077 ceph-mon[91833]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:17.116 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:16 smithi077 ceph-mon[91833]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:04:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:16 smithi078 ceph-mon[79689]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:16 smithi078 ceph-mon[79689]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T05:04:18.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 -- 172.21.15.77:0/1123492508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c101cd0 msgr2=0x7fa24c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:18.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 --2- 172.21.15.77:0/1123492508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c101cd0 0x7fa24c1020a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa244009ce0 tx=0x7fa244004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:18.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 -- 172.21.15.77:0/1123492508 shutdown_connections 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 --2- 172.21.15.77:0/1123492508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c102670 0x7fa24c0fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 --2- 172.21.15.77:0/1123492508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c101cd0 0x7fa24c1020a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 -- 172.21.15.77:0/1123492508 >> 172.21.15.77:0/1123492508 conn(0x7fa24c0f7860 msgr2=0x7fa24c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 -- 172.21.15.77:0/1123492508 shutdown_connections 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.668+0000 7fa253843700 1 -- 172.21.15.77:0/1123492508 wait complete. 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 Processor -- start 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 -- start start 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c101cd0 0x7fa24c112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c113500 con 0x7fa24c102670 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa253843700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c10cb20 con 0x7fa24c101cd0 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45724/0 (socket says 172.21.15.77:45724) 2024-04-26T05:04:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 -- 172.21.15.77:0/4203514586 learned_addr learned my addr 172.21.15.77:0/4203514586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa2515df700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c101cd0 0x7fa24c112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 -- 172.21.15.77:0/4203514586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c101cd0 msgr2=0x7fa24c112970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c101cd0 0x7fa24c112970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 -- 172.21.15.77:0/4203514586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009940 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.669+0000 7fa250dde700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa23c002a50 tx=0x7fa23c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.670+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23c007cc0 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.670+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c10ce00 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.670+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa23c00f040 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.670+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23c00d9c0 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.670+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c10d3c0 con 0x7fa24c102670 2024-04-26T05:04:18.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.671+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c0fd420 con 0x7fa24c102670 2024-04-26T05:04:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.671+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa23c009730 con 0x7fa24c102670 2024-04-26T05:04:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.672+0000 7fa241ffb700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 0x7fa238072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.672+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa23c08d990 con 0x7fa24c102670 2024-04-26T05:04:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.672+0000 7fa2515df700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 0x7fa238072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.672+0000 7fa2515df700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 0x7fa238072800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b740 tx=0x7fa244004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:18.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.676+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa23c056c10 con 0x7fa24c102670 2024-04-26T05:04:18.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.964+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa24c108dd0 con 0x7fa24c102670 2024-04-26T05:04:18.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.965+0000 7fa241ffb700 1 -- 172.21.15.77:0/4203514586 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa23c05a2d0 con 0x7fa24c102670 2024-04-26T05:04:18.967 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-04-26T05:04:18.967 INFO:teuthology.orchestra.run.smithi077.stdout: "mon": { 2024-04-26T05:04:18.967 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr": { 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: "osd": { 2024-04-26T05:04:18.968 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: "overall": { 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-04-26T05:04:18.969 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 msgr2=0x7fa238072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 0x7fa238072800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b740 tx=0x7fa244004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 msgr2=0x7fa24c112eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa23c002a50 tx=0x7fa23c004600 comp rx=0 tx=0).stop 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 shutdown_connections 2024-04-26T05:04:18.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa238070350 0x7fa238072800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa24c101cd0 0x7fa24c112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 --2- 172.21.15.77:0/4203514586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24c102670 0x7fa24c112eb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 >> 172.21.15.77:0/4203514586 conn(0x7fa24c0f7860 msgr2=0x7fa24c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 shutdown_connections 2024-04-26T05:04:18.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:18.967+0000 7fa253843700 1 -- 172.21.15.77:0/4203514586 wait complete. 2024-04-26T05:04:19.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:18 smithi077 ceph-mon[91833]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:18 smithi078 ceph-mon[79689]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:19.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-26T05:04:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:04:20.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:19 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/4203514586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:20.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:19 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/4203514586' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:21.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:20 smithi077 ceph-mon[91833]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:20 smithi078 ceph-mon[79689]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 -- 172.21.15.77:0/1049734453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 msgr2=0x7f7a740fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 --2- 172.21.15.77:0/1049734453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a740fbd20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004940 tx=0x7f7a5c009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 -- 172.21.15.77:0/1049734453 shutdown_connections 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 --2- 172.21.15.77:0/1049734453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a740fbd20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 --2- 172.21.15.77:0/1049734453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a741020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 -- 172.21.15.77:0/1049734453 >> 172.21.15.77:0/1049734453 conn(0x7f7a740f7860 msgr2=0x7f7a740f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 -- 172.21.15.77:0/1049734453 shutdown_connections 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.603+0000 7f7a79a70700 1 -- 172.21.15.77:0/1049734453 wait complete. 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 Processor -- start 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 -- start start 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a7410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7410f5e0 con 0x7f7a741026b0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a79a70700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7410f720 con 0x7f7a74101d10 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45748/0 (socket says 172.21.15.77:45748) 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 -- 172.21.15.77:0/2324529225 learned_addr learned my addr 172.21.15.77:0/2324529225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a737fe700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a7410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 -- 172.21.15.77:0/2324529225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 msgr2=0x7f7a7410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a7410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.604+0000 7f7a6affd700 1 -- 172.21.15.77:0/2324529225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c005040 con 0x7f7a741026b0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a737fe700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a7410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a6affd700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009100 tx=0x7f7a5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5c00e040 con 0x7f7a741026b0 2024-04-26T05:04:21.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7410f950 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a5c013420 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5c01c610 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.605+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a74117f70 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.606+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a740fd460 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.606+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a5c013590 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.607+0000 7f7a70ff9700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 0x7f7a600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.607+0000 7f7a737fe700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 0x7f7a600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.607+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7a5c08eec0 con 0x7f7a741026b0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.607+0000 7f7a737fe700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 0x7f7a600705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.612+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a5c0581c0 con 0x7f7a741026b0 2024-04-26T05:04:21.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.894+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7a7404c710 con 0x7f7a741026b0 2024-04-26T05:04:21.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.894+0000 7f7a70ff9700 1 -- 172.21.15.77:0/2324529225 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7a5c05b880 con 0x7f7a741026b0 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 msgr2=0x7f7a600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 0x7f7a600705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64008040 comp rx=0 tx=0).stop 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 msgr2=0x7f7a7410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009100 tx=0x7f7a5c009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 shutdown_connections 2024-04-26T05:04:21.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7a6006e140 0x7f7a600705f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101d10 0x7f7a7410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 --2- 172.21.15.77:0/2324529225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a741026b0 0x7f7a7410f0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:21.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 >> 172.21.15.77:0/2324529225 conn(0x7f7a740f7860 msgr2=0x7f7a740f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:21.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.896+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 shutdown_connections 2024-04-26T05:04:21.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:21.897+0000 7f7a79a70700 1 -- 172.21.15.77:0/2324529225 wait complete. 2024-04-26T05:04:21.908 INFO:teuthology.orchestra.run.smithi077.stdout:true 2024-04-26T05:04:22.216 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:21 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2324529225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:22.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:21 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2324529225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:22.539 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-26T05:04:22.791 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:04:23.195 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:22 smithi077 ceph-mon[91833]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:22 smithi078 ceph-mon[79689]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 -- 172.21.15.77:0/1320737654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 msgr2=0x7fab040fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 --2- 172.21.15.77:0/1320737654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fbd90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ce0 tx=0x7faaf8004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 -- 172.21.15.77:0/1320737654 shutdown_connections 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 --2- 172.21.15.77:0/1320737654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fbd90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 --2- 172.21.15.77:0/1320737654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 0x7fab04102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 -- 172.21.15.77:0/1320737654 >> 172.21.15.77:0/1320737654 conn(0x7fab040f7890 msgr2=0x7fab040f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 -- 172.21.15.77:0/1320737654 shutdown_connections 2024-04-26T05:04:24.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 -- 172.21.15.77:0/1320737654 wait complete. 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.439+0000 7fab09339700 1 Processor -- start 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab09339700 1 -- start start 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab09339700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 0x7fab040fda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab09339700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab09339700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040ff950 con 0x7fab04102720 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab09339700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040fe480 con 0x7fab04101d80 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45764/0 (socket says 172.21.15.77:45764) 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 -- 172.21.15.77:0/2307625096 learned_addr learned my addr 172.21.15.77:0/2307625096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab02ffd700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 0x7fab040fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 -- 172.21.15.77:0/2307625096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 msgr2=0x7fab040fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 0x7fab040fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 -- 172.21.15.77:0/2307625096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009940 con 0x7fab04102720 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7fab027fc700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faaf800be40 tx=0x7faaf8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.440+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf800e040 con 0x7fab04102720 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.441+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaf8004610 con 0x7fab04102720 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.441+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab040fe700 con 0x7fab04102720 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.441+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf801c640 con 0x7fab04102720 2024-04-26T05:04:24.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.441+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0411a170 con 0x7fab04102720 2024-04-26T05:04:24.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.442+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab040fd4d0 con 0x7fab04102720 2024-04-26T05:04:24.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.442+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faaf8012970 con 0x7fab04102720 2024-04-26T05:04:24.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.442+0000 7faaeb7fe700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 0x7faaec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:24.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.443+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7faaf808f5e0 con 0x7fab04102720 2024-04-26T05:04:24.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.443+0000 7fab02ffd700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 0x7faaec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:24.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.443+0000 7fab02ffd700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 0x7faaec0705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faafc009fd0 tx=0x7faafc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:24.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.447+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faaf8058860 con 0x7fab04102720 2024-04-26T05:04:24.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.729+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab0411a360 con 0x7fab04102720 2024-04-26T05:04:24.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.730+0000 7faaeb7fe700 1 -- 172.21.15.77:0/2307625096 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7faaf807c280 con 0x7fab04102720 2024-04-26T05:04:24.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 msgr2=0x7faaec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:24.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 0x7faaec0705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faafc009fd0 tx=0x7faafc00b040 comp rx=0 tx=0).stop 2024-04-26T05:04:24.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 msgr2=0x7fab040fdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:24.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faaf800be40 tx=0x7faaf8004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:24.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 shutdown_connections 2024-04-26T05:04:24.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7faaec06e140 0x7faaec0705f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab04101d80 0x7fab040fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 --2- 172.21.15.77:0/2307625096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab04102720 0x7fab040fdf40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:24.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 >> 172.21.15.77:0/2307625096 conn(0x7fab040f7890 msgr2=0x7fab04066570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:24.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 shutdown_connections 2024-04-26T05:04:24.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:24.732+0000 7fab09339700 1 -- 172.21.15.77:0/2307625096 wait complete. 2024-04-26T05:04:24.743 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)" 2024-04-26T05:04:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:24 smithi077 ceph-mon[91833]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:25.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:24 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2307625096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:24 smithi078 ceph-mon[79689]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:24 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2307625096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T05:04:25.365 DEBUG:teuthology.parallel:result is None 2024-04-26T05:04:25.365 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T05:04:25.379 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T05:04:25.379 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T05:04:25.633 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/mon.smithi077/config 2024-04-26T05:04:27.104 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:26 smithi077 ceph-mon[91833]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:26 smithi078 ceph-mon[79689]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:27.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 -- 172.21.15.77:0/332334604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984102de0 msgr2=0x7fe9841031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:27.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 --2- 172.21.15.77:0/332334604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984102de0 0x7fe9841031b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe974004940 tx=0x7fe974009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:27.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 -- 172.21.15.77:0/332334604 shutdown_connections 2024-04-26T05:04:27.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 --2- 172.21.15.77:0/332334604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984103780 0x7fe9840fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 --2- 172.21.15.77:0/332334604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984102de0 0x7fe9841031b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 -- 172.21.15.77:0/332334604 >> 172.21.15.77:0/332334604 conn(0x7fe9840f8950 msgr2=0x7fe9840fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 -- 172.21.15.77:0/332334604 shutdown_connections 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 -- 172.21.15.77:0/332334604 wait complete. 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.333+0000 7fe989347700 1 Processor -- start 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe989347700 1 -- start start 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe989347700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984102de0 0x7fe984113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe989347700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe989347700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984114720 con 0x7fe984103780 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe989347700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98410da80 con 0x7fe984102de0 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe982ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984102de0 0x7fe984113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45770/0 (socket says 172.21.15.77:45770) 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 -- 172.21.15.77:0/1570950614 learned_addr learned my addr 172.21.15.77:0/1570950614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 -- 172.21.15.77:0/1570950614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984102de0 msgr2=0x7fe984113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:27.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984102de0 0x7fe984113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.334+0000 7fe97a7fc700 1 -- 172.21.15.77:0/1570950614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974005040 con 0x7fe984103780 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe97a7fc700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b790 tx=0x7fe96c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96c00f6f0 con 0x7fe984103780 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe96c00fd20 con 0x7fe984103780 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98410dd60 con 0x7fe984103780 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96c0103a0 con 0x7fe984103780 2024-04-26T05:04:27.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.335+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98410e320 con 0x7fe984103780 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.336+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9841000f0 con 0x7fe984103780 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.336+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe96c011400 con 0x7fe984103780 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.337+0000 7fe97bfff700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 0x7fe970070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.337+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe96c08d910 con 0x7fe984103780 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.337+0000 7fe982ffd700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 0x7fe970070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:27.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.337+0000 7fe982ffd700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 0x7fe970070600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe974009fd0 tx=0x7fe974009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:27.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.341+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe96c05a7e0 con 0x7fe984103780 2024-04-26T05:04:27.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.580+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe98410f150 con 0x7fe984103780 2024-04-26T05:04:27.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.581+0000 7fe97bfff700 1 -- 172.21.15.77:0/1570950614 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1935 (secure 0 0 0) 0x7fe96c05a370 con 0x7fe984103780 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:e36 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 36 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-04-26T04:49:55.003675+0000 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-04-26T05:01:26.009556+0000 2024-04-26T05:04:27.582 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 60 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 84 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 1 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:in 0 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=34376} 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-04-26T05:04:27.583 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data enabled 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 1 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 34376 members: 34376 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.quaiyg{0:34376} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.77:6826/1508073120,v1:172.21.15.77:6827/1508073120] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi077.urxtgp{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.77:6828/2532102778,v1:172.21.15.77:6829/2532102778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:04:27.584 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.wkrgkp{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6824/4272329395,v1:172.21.15.78:6825/4272329395] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:04:27.585 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.cephfs.smithi078.nihewy{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.78:6826/1653368163,v1:172.21.15.78:6827/1653368163] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T05:04:27.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 msgr2=0x7fe970070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:27.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 0x7fe970070600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe974009fd0 tx=0x7fe974009e30 comp rx=0 tx=0).stop 2024-04-26T05:04:27.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 msgr2=0x7fe984114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b790 tx=0x7fe96c009220 comp rx=0 tx=0).stop 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 shutdown_connections 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe97006e150 0x7fe970070600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe984102de0 0x7fe984113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 --2- 172.21.15.77:0/1570950614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe984103780 0x7fe984114040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 >> 172.21.15.77:0/1570950614 conn(0x7fe9840f8950 msgr2=0x7fe9841060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 shutdown_connections 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:27.583+0000 7fe989347700 1 -- 172.21.15.77:0/1570950614 wait complete. 2024-04-26T05:04:27.586 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-04-26T05:04:28.238 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:27 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1570950614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:04:28.263 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-26T05:04:28.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T05:04:28.274 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:27 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1570950614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 -- 172.21.15.77:0/3536178539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 msgr2=0x7fbae01061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 --2- 172.21.15.77:0/3536178539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae01061b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbad00048d0 tx=0x7fbad0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 -- 172.21.15.77:0/3536178539 shutdown_connections 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 --2- 172.21.15.77:0/3536178539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae01061b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 --2- 172.21.15.77:0/3536178539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae00fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.370+0000 7fbae6a25640 1 -- 172.21.15.77:0/3536178539 >> 172.21.15.77:0/3536178539 conn(0x7fbae00f6c20 msgr2=0x7fbae00f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:28.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 -- 172.21.15.77:0/3536178539 shutdown_connections 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 -- 172.21.15.77:0/3536178539 wait complete. 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 Processor -- start 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 -- start start 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae0112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0113990 con 0x7fbae00fdb70 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbae6a25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae010cf50 con 0x7fbae00fd1d0 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbadf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbadf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59180/0 (socket says 172.21.15.77:59180) 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.371+0000 7fbadf7fe640 1 -- 172.21.15.77:0/2122998741 learned_addr learned my addr 172.21.15.77:0/2122998741 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadffff640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae0112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadf7fe640 1 -- 172.21.15.77:0/2122998741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 msgr2=0x7fbae0112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadf7fe640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae0112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadf7fe640 1 -- 172.21.15.77:0/2122998741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0005040 con 0x7fbae00fdb70 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadffff640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae0112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadf7fe640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad0004930 tx=0x7fbad0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad00092b0 con 0x7fbae00fdb70 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae010d1d0 con 0x7fbae00fdb70 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbad003b400 con 0x7fbae00fdb70 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad00435b0 con 0x7fbae00fdb70 2024-04-26T05:04:28.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.372+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae010d690 con 0x7fbae00fdb70 2024-04-26T05:04:28.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.373+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae0065db0 con 0x7fbae00fdb70 2024-04-26T05:04:28.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.374+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbad002f070 con 0x7fbae00fdb70 2024-04-26T05:04:28.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.374+0000 7fbadcff9640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.374+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fbad00afa50 con 0x7fbae00fdb70 2024-04-26T05:04:28.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.374+0000 7fbadffff640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.375+0000 7fbadffff640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 0x7fbac8070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbad4006fd0 tx=0x7fbad4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.379+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbad0078cd0 con 0x7fbae00fdb70 2024-04-26T05:04:28.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.556+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbae01093d0 con 0x7fbae00fdb70 2024-04-26T05:04:28.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.557+0000 7fbadcff9640 1 -- 172.21.15.77:0/2122998741 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fbad007c390 con 0x7fbae00fdb70 2024-04-26T05:04:28.558 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:28.558 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T05:01:13.968714+0000","last_up_change":"2024-04-26T04:59:52.906609+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:49:53.533004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:49:54.229975+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6803","nonce":2493290312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6805","nonce":2493290312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6809","nonce":2493290312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6807","nonce":2493290312}]},"public_addr":"172.21.15.77:6803/2493290312","cluster_addr":"172.21.15.77:6805/2493290312","heartbeat_back_addr":"172.21.15.77:6809/2493290312","heartbeat_front_addr":"172.21.15.77:6807/2493290312","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6811","nonce":3696890435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6813","nonce":3696890435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6817","nonce":3696890435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6815","nonce":3696890435}]},"public_addr":"172.21.15.77:6811/3696890435","cluster_addr":"172.21.15.77:6813/3696890435","heartbeat_back_addr":"172.21.15.77:6817/3696890435","heartbeat_front_addr":"172.21.15.77:6815/3696890435","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6819","nonce":3580780377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6821","nonce":3580780377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6825","nonce":3580780377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6823","nonce":3580780377}]},"public_addr":"172.21.15.77:6819/3580780377","cluster_addr":"172.21.15.77:6821/3580780377","heartbeat_back_addr":"172.21.15.77:6825/3580780377","heartbeat_front_addr":"172.21.15.77:6823/3580780377","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6801","nonce":2853701677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6803","nonce":2853701677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6807","nonce":2853701677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6805","nonce":2853701677}]},"public_addr":"172.21.15.78:6801/2853701677","cluster_addr":"172.21.15.78:6803/2853701677","heartbeat_back_addr":"172.21.15.78:6807/2853701677","heartbeat_front_addr":"172.21.15.78:6805/2853701677","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6809","nonce":2142591175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6811","nonce":2142591175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6815","nonce":2142591175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6813","nonce":2142591175}]},"public_addr":"172.21.15.78:6809/2142591175","cluster_addr":"172.21.15.78:6811/2142591175","heartbeat_back_addr":"172.21.15.78:6815/2142591175","heartbeat_front_addr":"172.21.15.78:6813/2142591175","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6817","nonce":1951816516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6819","nonce":1951816516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6823","nonce":1951816516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6821","nonce":1951816516}]},"public_addr":"172.21.15.78:6817/1951816516","cluster_addr":"172.21.15.78:6819/1951816516","heartbeat_back_addr":"172.21.15.78:6823/1951816516","heartbeat_front_addr":"172.21.15.78:6821/1951816516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6826/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.78:6825/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.78:6824/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.77:6828/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:6827/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.78:0/4271908245":"2024-04-27T04:54:38.567168+0000","172.21.15.77:6801/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/528446640":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3379821906":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.78:0/3124669946":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/3441382659":"2024-04-27T04:52:26.462041+0000","172.21.15.78:6829/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/192705865":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6828/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/1835173819":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/2218529211":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/529966052":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6827/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.77:0/4033800957":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6829/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/2809920581":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1055621332":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6826/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:6801/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1531077053":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/716951029":"2024-04-27T04:55:30.843837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 msgr2=0x7fbac8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 0x7fbac8070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbad4006fd0 tx=0x7fbad4008040 comp rx=0 tx=0).stop 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 msgr2=0x7fbae0113300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad0004930 tx=0x7fbad0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 shutdown_connections 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbae00fdb70 0x7fbae0113300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 --2- 172.21.15.77:0/2122998741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbae00fd1d0 0x7fbae0112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 >> 172.21.15.77:0/2122998741 conn(0x7fbae00f6c20 msgr2=0x7fbae00f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 shutdown_connections 2024-04-26T05:04:28.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.559+0000 7fbae6a25640 1 -- 172.21.15.77:0/2122998741 wait complete. 2024-04-26T05:04:28.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:47:22.739195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:49:53.533004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:49:54.229975+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T05:04:28.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T05:04:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- 172.21.15.77:0/1475976625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 msgr2=0x7f72b80fb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/1475976625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b80fb0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72b40048d0 tx=0x7f72b4009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- 172.21.15.77:0/1475976625 shutdown_connections 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/1475976625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b80fb0e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/1475976625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 0x7f72b8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- 172.21.15.77:0/1475976625 >> 172.21.15.77:0/1475976625 conn(0x7f72b80f6c60 msgr2=0x7f72b80f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- 172.21.15.77:0/1475976625 shutdown_connections 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- 172.21.15.77:0/1475976625 wait complete. 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 Processor -- start 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.668+0000 7f72bfb9d640 1 -- start start 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 0x7f72b8113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8113b40 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b810d190 con 0x7f72b81010b0 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59210/0 (socket says 172.21.15.77:59210) 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 -- 172.21.15.77:0/3285343786 learned_addr learned my addr 172.21.15.77:0/3285343786 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd939640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 0x7f72b8113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 -- 172.21.15.77:0/3285343786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 msgr2=0x7f72b8113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 0x7f72b8113000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 -- 172.21.15.77:0/3285343786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4005040 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bd138640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004930 tx=0x7f72b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b40094c0 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b810d410 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b4032e50 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b40433b0 con 0x7f72b8101a50 2024-04-26T05:04:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.669+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b810d8d0 con 0x7f72b8101a50 2024-04-26T05:04:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.670+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b80fc820 con 0x7f72b8101a50 2024-04-26T05:04:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.671+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f72b402f070 con 0x7f72b8101a50 2024-04-26T05:04:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.671+0000 7f72aa7fc640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 0x7f72a4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.671+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f72b40afaa0 con 0x7f72b8101a50 2024-04-26T05:04:28.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.671+0000 7f72bd939640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 0x7f72a4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.672+0000 7f72bd939640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 0x7f72a4070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009af0 tx=0x7f72ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.676+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f72b4078d20 con 0x7f72b8101a50 2024-04-26T05:04:28.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.852+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72b804bcc0 con 0x7f72b8101a50 2024-04-26T05:04:28.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.853+0000 7f72aa7fc640 1 -- 172.21.15.77:0/3285343786 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72b407c3e0 con 0x7f72b8101a50 2024-04-26T05:04:28.854 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 msgr2=0x7f72a4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 0x7f72a4070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009af0 tx=0x7f72ac00b040 comp rx=0 tx=0).stop 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 msgr2=0x7f72b8113540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004930 tx=0x7f72b4009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 shutdown_connections 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f72a406e0b0 0x7f72a4070560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72b8101a50 0x7f72b8113540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 --2- 172.21.15.77:0/3285343786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b81010b0 0x7f72b8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 >> 172.21.15.77:0/3285343786 conn(0x7f72b80f6c60 msgr2=0x7f72b80f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:28.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 shutdown_connections 2024-04-26T05:04:28.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.855+0000 7f72bfb9d640 1 -- 172.21.15.77:0/3285343786 wait complete. 2024-04-26T05:04:28.867 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.962+0000 7f22f779d640 1 -- 172.21.15.77:0/2525224668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 msgr2=0x7f22f00fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.962+0000 7f22f779d640 1 --2- 172.21.15.77:0/2525224668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f00fb530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0048d0 tx=0x7f22ec009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- 172.21.15.77:0/2525224668 shutdown_connections 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 --2- 172.21.15.77:0/2525224668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f00fb530 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 --2- 172.21.15.77:0/2525224668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 0x7f22f01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- 172.21.15.77:0/2525224668 >> 172.21.15.77:0/2525224668 conn(0x7f22f00710c0 msgr2=0x7f22f00714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:28.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- 172.21.15.77:0/2525224668 shutdown_connections 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- 172.21.15.77:0/2525224668 wait complete. 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 Processor -- start 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- start start 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 0x7f22f010e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f010e8e0 con 0x7f22f00fb0e0 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f779d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f010ea50 con 0x7f22f0101110 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f5539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f5539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59226/0 (socket says 172.21.15.77:59226) 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.963+0000 7f22f5539640 1 -- 172.21.15.77:0/3365809692 learned_addr learned my addr 172.21.15.77:0/3365809692 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f4d38640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 0x7f22f010e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f5539640 1 -- 172.21.15.77:0/3365809692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 msgr2=0x7f22f010e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f5539640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 0x7f22f010e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f5539640 1 -- 172.21.15.77:0/3365809692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec005040 con 0x7f22f00fb0e0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f5539640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22e0002a50 tx=0x7f22e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e0007c30 con 0x7f22f00fb0e0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22e000f040 con 0x7f22f00fb0e0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f0116ed0 con 0x7f22f00fb0e0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e000db50 con 0x7f22f00fb0e0 2024-04-26T05:04:28.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.964+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0117200 con 0x7f22f00fb0e0 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.965+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f00fc7d0 con 0x7f22f00fb0e0 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.966+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f22e001c400 con 0x7f22f00fb0e0 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.966+0000 7f22e5ffb640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 0x7f22dc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.966+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f22e008d400 con 0x7f22f00fb0e0 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.966+0000 7f22f4d38640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 0x7f22dc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:28.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.967+0000 7f22f4d38640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 0x7f22dc070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:28.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:28.971+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f22e0056730 con 0x7f22f00fb0e0 2024-04-26T05:04:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.149+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f22f004bcc0 con 0x7f22f00fb0e0 2024-04-26T05:04:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.149+0000 7f22e5ffb640 1 -- 172.21.15.77:0/3365809692 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f22e0059df0 con 0x7f22f00fb0e0 2024-04-26T05:04:29.150 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 msgr2=0x7f22dc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 0x7f22dc070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 msgr2=0x7f22f010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22e0002a50 tx=0x7f22e0004600 comp rx=0 tx=0).stop 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 shutdown_connections 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22f0101110 0x7f22f010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f22dc06e0b0 0x7f22dc070560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 --2- 172.21.15.77:0/3365809692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f00fb0e0 0x7f22f010de60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 >> 172.21.15.77:0/3365809692 conn(0x7f22f00710c0 msgr2=0x7f22f00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 shutdown_connections 2024-04-26T05:04:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.151+0000 7f22f779d640 1 -- 172.21.15.77:0/3365809692 wait complete. 2024-04-26T05:04:29.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:28 smithi077 ceph-mon[91833]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:29.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:28 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2122998741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:29.162 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:28 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3285343786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:29.163 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T05:04:29.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 -- 172.21.15.77:0/445360887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 msgr2=0x7f2f481014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/445360887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481014e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f380048d0 tx=0x7f2f38009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 -- 172.21.15.77:0/445360887 shutdown_connections 2024-04-26T05:04:29.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/445360887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f480fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/445360887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481014e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 -- 172.21.15.77:0/445360887 >> 172.21.15.77:0/445360887 conn(0x7f2f480710c0 msgr2=0x7f2f480714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 -- 172.21.15.77:0/445360887 shutdown_connections 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.258+0000 7f2f4e79d640 1 -- 172.21.15.77:0/445360887 wait complete. 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 Processor -- start 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 -- start start 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f48111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48112840 con 0x7f2f48101110 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f4e79d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4810be00 con 0x7f2f480fb0e0 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59250/0 (socket says 172.21.15.77:59250) 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.259+0000 7f2f477fe640 1 -- 172.21.15.77:0/1553898609 learned_addr learned my addr 172.21.15.77:0/1553898609 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:29.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f47fff640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f48111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f477fe640 1 -- 172.21.15.77:0/1553898609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 msgr2=0x7f2f48111c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f477fe640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f48111c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f477fe640 1 -- 172.21.15.77:0/1553898609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38005040 con 0x7f2f48101110 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f47fff640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f48111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f477fe640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00b790 tx=0x7f2f3c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f3c00c700 con 0x7f2f48101110 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f3c00cd30 con 0x7f2f48101110 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f3c013370 con 0x7f2f48101110 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4810c0e0 con 0x7f2f48101110 2024-04-26T05:04:29.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.260+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4810c600 con 0x7f2f48101110 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.261+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f480fc7d0 con 0x7f2f48101110 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.261+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2f3c012070 con 0x7f2f48101110 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.262+0000 7f2f44ff9640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 0x7f2f30070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.262+0000 7f2f47fff640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 0x7f2f30070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.262+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2f3c08d870 con 0x7f2f48101110 2024-04-26T05:04:29.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.262+0000 7f2f47fff640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 0x7f2f30070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f38004540 tx=0x7f2f38009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.266+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2f3c056af0 con 0x7f2f48101110 2024-04-26T05:04:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:28 smithi078 ceph-mon[79689]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:29.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:28 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2122998741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:29.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:28 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3285343786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:29.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.442+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2f481082b0 con 0x7f2f48101110 2024-04-26T05:04:29.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.442+0000 7f2f44ff9640 1 -- 172.21.15.77:0/1553898609 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2f3c05a1b0 con 0x7f2f48101110 2024-04-26T05:04:29.443 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 msgr2=0x7f2f30070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 0x7f2f30070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f38004540 tx=0x7f2f38009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 msgr2=0x7f2f481121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00b790 tx=0x7f2f3c0091d0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 shutdown_connections 2024-04-26T05:04:29.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f2f3006e0b0 0x7f2f30070560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f48101110 0x7f2f481121b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 --2- 172.21.15.77:0/1553898609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f480fb0e0 0x7f2f48111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 >> 172.21.15.77:0/1553898609 conn(0x7f2f480710c0 msgr2=0x7f2f480f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 shutdown_connections 2024-04-26T05:04:29.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.444+0000 7f2f4e79d640 1 -- 172.21.15.77:0/1553898609 wait complete. 2024-04-26T05:04:29.456 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T05:04:29.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- 172.21.15.77:0/2788882886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fd190 msgr2=0x7f6e940fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2788882886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fd190 0x7f6e940fd560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c004900 tx=0x7f6e8c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- 172.21.15.77:0/2788882886 shutdown_connections 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2788882886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fdb30 0x7f6e94106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2788882886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fd190 0x7f6e940fd560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- 172.21.15.77:0/2788882886 >> 172.21.15.77:0/2788882886 conn(0x7f6e940f6c30 msgr2=0x7f6e940f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- 172.21.15.77:0/2788882886 shutdown_connections 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- 172.21.15.77:0/2788882886 wait complete. 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 Processor -- start 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.549+0000 7f6e9942e640 1 -- start start 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 0x7f6e940fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940fcfb0 con 0x7f6e940fdb30 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940fbad0 con 0x7f6e940fd190 2024-04-26T05:04:29.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59262/0 (socket says 172.21.15.77:59262) 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 -- 172.21.15.77:0/2280132891 learned_addr learned my addr 172.21.15.77:0/2280132891 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e92ffd640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 0x7f6e940fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 -- 172.21.15.77:0/2280132891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 msgr2=0x7f6e940fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 0x7f6e940fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 -- 172.21.15.77:0/2280132891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c005040 con 0x7f6e940fdb30 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e92ffd640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 0x7f6e940fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e927fc640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800db00 tx=0x7f6e88004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e88017070 con 0x7f6e940fdb30 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940fbdb0 con 0x7f6e940fdb30 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e8800f040 con 0x7f6e940fdb30 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e88003ae0 con 0x7f6e940fdb30 2024-04-26T05:04:29.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.550+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e94119550 con 0x7f6e940fdb30 2024-04-26T05:04:29.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.551+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e94065d30 con 0x7f6e940fdb30 2024-04-26T05:04:29.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.552+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6e88003c40 con 0x7f6e940fdb30 2024-04-26T05:04:29.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.552+0000 7f6e7b7fe640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 0x7f6e7c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.552+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6e8808dd80 con 0x7f6e940fdb30 2024-04-26T05:04:29.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.552+0000 7f6e92ffd640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 0x7f6e7c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.553+0000 7f6e92ffd640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 0x7f6e7c070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0043d0 tx=0x7f6e8c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.557+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6e88057000 con 0x7f6e940fdb30 2024-04-26T05:04:29.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.768+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e94109370 con 0x7f6e940fdb30 2024-04-26T05:04:29.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.769+0000 7f6e7b7fe640 1 -- 172.21.15.77:0/2280132891 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5207 (secure 0 0 0) 0x7f6e8808f090 con 0x7f6e940fdb30 2024-04-26T05:04:29.769 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:29.769 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44341,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6827/1653368163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1653368163},{"type":"v1","addr":"172.21.15.78:6827","nonce":1653368163}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:26.009556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34376,"qdb_cluster":[34376]},"id":1}]} 2024-04-26T05:04:29.770 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 msgr2=0x7f6e7c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 0x7f6e7c070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0043d0 tx=0x7f6e8c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 msgr2=0x7f6e940fb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800db00 tx=0x7f6e88004e60 comp rx=0 tx=0).stop 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 shutdown_connections 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6e7c06e0b0 0x7f6e7c070560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e940fdb30 0x7f6e940fb590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 --2- 172.21.15.77:0/2280132891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e940fd190 0x7f6e940fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 >> 172.21.15.77:0/2280132891 conn(0x7f6e940f6c30 msgr2=0x7f6e940f8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 shutdown_connections 2024-04-26T05:04:29.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.771+0000 7f6e9942e640 1 -- 172.21.15.77:0/2280132891 wait complete. 2024-04-26T05:04:29.782 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-26T05:04:29.783 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- 172.21.15.77:0/3780694177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 msgr2=0x7f42481006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3780694177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f42481006c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f423c0048d0 tx=0x7f423c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- 172.21.15.77:0/3780694177 shutdown_connections 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3780694177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 0x7f42480fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3780694177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f42481006c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- 172.21.15.77:0/3780694177 >> 172.21.15.77:0/3780694177 conn(0x7f4248071990 msgr2=0x7f4248071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- 172.21.15.77:0/3780694177 shutdown_connections 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- 172.21.15.77:0/3780694177 wait complete. 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 Processor -- start 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- start start 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 0x7f4248112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248113b10 con 0x7f42481002f0 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.877+0000 7f424f0ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424810d160 con 0x7f42480fbb60 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59288/0 (socket says 172.21.15.77:59288) 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 -- 172.21.15.77:0/3556853392 learned_addr learned my addr 172.21.15.77:0/3556853392 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f424ce4a640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 0x7f4248112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 -- 172.21.15.77:0/3556853392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 msgr2=0x7f4248112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:29.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 0x7f4248112fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 -- 172.21.15.77:0/3556853392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c005040 con 0x7f42481002f0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f4247fff640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4234002a50 tx=0x7f4234004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4234007c30 con 0x7f42481002f0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424810d440 con 0x7f42481002f0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f423400f040 con 0x7f42481002f0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423400db50 con 0x7f42481002f0 2024-04-26T05:04:29.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.878+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424810d960 con 0x7f42481002f0 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.879+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42480fcb70 con 0x7f42481002f0 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.879+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4234009070 con 0x7f42481002f0 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.880+0000 7f42457fa640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 0x7f4238070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.880+0000 7f424ce4a640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 0x7f4238070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.880+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f423408d550 con 0x7f42481002f0 2024-04-26T05:04:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.880+0000 7f424ce4a640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 0x7f4238070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f423c003080 tx=0x7f423c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:29.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:29.885+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f42340567d0 con 0x7f42481002f0 2024-04-26T05:04:30.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.098+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f4248108560 con 0x7f42481002f0 2024-04-26T05:04:30.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.099+0000 7f42457fa640 1 -- 172.21.15.77:0/3556853392 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v36) v1 ==== 107+0+4852 (secure 0 0 0) 0x7f4234059e90 con 0x7f42481002f0 2024-04-26T05:04:30.100 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:30.100 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T04:50:10.502841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6827/132419939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":132419939},{"type":"v1","addr":"172.21.15.77:6827","nonce":132419939}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:30.100 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 14 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 msgr2=0x7f4238070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 0x7f4238070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f423c003080 tx=0x7f423c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 msgr2=0x7f4248113510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4234002a50 tx=0x7f4234004600 comp rx=0 tx=0).stop 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 shutdown_connections 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f423806e0b0 0x7f4238070560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42481002f0 0x7f4248113510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 --2- 172.21.15.77:0/3556853392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42480fbb60 0x7f4248112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 >> 172.21.15.77:0/3556853392 conn(0x7f4248071990 msgr2=0x7f42480fb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 shutdown_connections 2024-04-26T05:04:30.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.101+0000 7f424f0ae640 1 -- 172.21.15.77:0/3556853392 wait complete. 2024-04-26T05:04:30.111 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:30 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3365809692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:30.111 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:30 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1553898609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:30.111 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:30 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2280132891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T05:04:30.113 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.206+0000 7f6ec239d640 1 -- 172.21.15.77:0/1602593189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc0fb150 msgr2=0x7f6ebc0fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.206+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1602593189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc0fb150 0x7f6ebc0fb5c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00048d0 tx=0x7f6eb0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- 172.21.15.77:0/1602593189 shutdown_connections 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1602593189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc0fb150 0x7f6ebc0fb5c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1602593189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc101170 0x7f6ebc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- 172.21.15.77:0/1602593189 >> 172.21.15.77:0/1602593189 conn(0x7f6ebc071fd0 msgr2=0x7f6ebc0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- 172.21.15.77:0/1602593189 shutdown_connections 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- 172.21.15.77:0/1602593189 wait complete. 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 Processor -- start 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- start start 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc0fb150 0x7f6ebc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc10e970 con 0x7f6ebc101170 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.207+0000 7f6ec239d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc10eae0 con 0x7f6ebc0fb150 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59308/0 (socket says 172.21.15.77:59308) 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 -- 172.21.15.77:0/1358656052 learned_addr learned my addr 172.21.15.77:0/1358656052 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 -- 172.21.15.77:0/1358656052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc0fb150 msgr2=0x7f6ebc10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc0fb150 0x7f6ebc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 -- 172.21.15.77:0/1358656052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb0005040 con 0x7f6ebc101170 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ebb7fe640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0004930 tx=0x7f6eb0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb00094c0 con 0x7f6ebc101170 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb0032e50 con 0x7f6ebc101170 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc10ed60 con 0x7f6ebc101170 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb00433b0 con 0x7f6ebc101170 2024-04-26T05:04:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.208+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1171b0 con 0x7f6ebc101170 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.209+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc0fc870 con 0x7f6ebc101170 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.210+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6eb002f070 con 0x7f6ebc101170 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.210+0000 7f6eb8ff9640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 0x7f6ea4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.210+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6eb00afa20 con 0x7f6ebc101170 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.210+0000 7f6ebbfff640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 0x7f6ea4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.211+0000 7f6ebbfff640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 0x7f6ea4070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097e0 tx=0x7f6eac006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.215+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6eb0078ca0 con 0x7f6ebc101170 2024-04-26T05:04:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:30 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3365809692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:30 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1553898609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:30.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:30 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2280132891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T05:04:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.428+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7f6ebc10ed60 con 0x7f6ebc101170 2024-04-26T05:04:30.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.429+0000 7f6eb8ff9640 1 -- 172.21.15.77:0/1358656052 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v36) v1 ==== 107+0+4067 (secure 0 0 0) 0x7f6ebc10ed60 con 0x7f6ebc101170 2024-04-26T05:04:30.430 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:30.430 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T04:59:59.294804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:30.430 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 15 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 msgr2=0x7f6ea4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 0x7f6ea4070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097e0 tx=0x7f6eac006d20 comp rx=0 tx=0).stop 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 msgr2=0x7f6ebc10e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0004930 tx=0x7f6eb0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 shutdown_connections 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6ea406e0b0 0x7f6ea4070560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ebc101170 0x7f6ebc10e430 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 --2- 172.21.15.77:0/1358656052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc0fb150 0x7f6ebc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 >> 172.21.15.77:0/1358656052 conn(0x7f6ebc071fd0 msgr2=0x7f6ebc0fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 shutdown_connections 2024-04-26T05:04:30.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.431+0000 7f6ec239d640 1 -- 172.21.15.77:0/1358656052 wait complete. 2024-04-26T05:04:30.443 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-26T05:04:30.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 -- 172.21.15.77:0/309913001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fac80 msgr2=0x7f93300fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 --2- 172.21.15.77:0/309913001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fac80 0x7f93300fb050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f932c0048d0 tx=0x7f932c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 -- 172.21.15.77:0/309913001 shutdown_connections 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 --2- 172.21.15.77:0/309913001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fb620 0x7f93300fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 --2- 172.21.15.77:0/309913001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fac80 0x7f93300fb050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 -- 172.21.15.77:0/309913001 >> 172.21.15.77:0/309913001 conn(0x7f9330071990 msgr2=0x7f9330071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 -- 172.21.15.77:0/309913001 shutdown_connections 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.537+0000 7f93380ae640 1 -- 172.21.15.77:0/309913001 wait complete. 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 Processor -- start 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 -- start start 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fac80 0x7f9330119550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933011a0e0 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93380ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9330113700 con 0x7f93300fac80 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59336/0 (socket says 172.21.15.77:59336) 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 -- 172.21.15.77:0/3042944470 learned_addr learned my addr 172.21.15.77:0/3042944470 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335e4a640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fac80 0x7f9330119550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 -- 172.21.15.77:0/3042944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fac80 msgr2=0x7f9330119550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fac80 0x7f9330119550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 -- 172.21.15.77:0/3042944470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c005040 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f9335649640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9320002870 tx=0x7f9320004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.538+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9320007c30 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.539+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f932000f040 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.539+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93301139e0 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.539+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93200095f0 con 0x7f93300fb620 2024-04-26T05:04:30.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.539+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9330113f00 con 0x7f93300fb620 2024-04-26T05:04:30.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.540+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93300fba60 con 0x7f93300fb620 2024-04-26T05:04:30.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.540+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f932000d6b0 con 0x7f93300fb620 2024-04-26T05:04:30.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.541+0000 7f93267fc640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 0x7f931c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.541+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f932008d3e0 con 0x7f93300fb620 2024-04-26T05:04:30.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.541+0000 7f9335e4a640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 0x7f931c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.541+0000 7f9335e4a640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 0x7f931c070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f932c003080 tx=0x7f932c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.545+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9320056710 con 0x7f93300fb620 2024-04-26T05:04:30.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.756+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f933010fb30 con 0x7f93300fb620 2024-04-26T05:04:30.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.757+0000 7f93267fc640 1 -- 172.21.15.77:0/3042944470 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v36) v1 ==== 107+0+4078 (secure 0 0 0) 0x7f9320059dd0 con 0x7f93300fb620 2024-04-26T05:04:30.758 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:30.758 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T04:59:59.299512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:30.758 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 16 2024-04-26T05:04:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 msgr2=0x7f931c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 0x7f931c070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f932c003080 tx=0x7f932c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 msgr2=0x7f9330119a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9320002870 tx=0x7f9320004600 comp rx=0 tx=0).stop 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 shutdown_connections 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f931c06e0b0 0x7f931c070560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93300fb620 0x7f9330119a90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 --2- 172.21.15.77:0/3042944470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93300fac80 0x7f9330119550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 >> 172.21.15.77:0/3042944470 conn(0x7f9330071990 msgr2=0x7f93300f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 shutdown_connections 2024-04-26T05:04:30.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.759+0000 7f93380ae640 1 -- 172.21.15.77:0/3042944470 wait complete. 2024-04-26T05:04:30.771 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 -- 172.21.15.77:0/2376096043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 msgr2=0x7f67700fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 --2- 172.21.15.77:0/2376096043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f67700fb0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67640048d0 tx=0x7f6764009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 -- 172.21.15.77:0/2376096043 shutdown_connections 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 --2- 172.21.15.77:0/2376096043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f67700fb0a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 --2- 172.21.15.77:0/2376096043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 0x7f6770101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 -- 172.21.15.77:0/2376096043 >> 172.21.15.77:0/2376096043 conn(0x7f67700f6be0 msgr2=0x7f67700f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 -- 172.21.15.77:0/2376096043 shutdown_connections 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 -- 172.21.15.77:0/2376096043 wait complete. 2024-04-26T05:04:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.865+0000 7f6775f9d640 1 Processor -- start 2024-04-26T05:04:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 -- start start 2024-04-26T05:04:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 0x7f677010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770066720 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677010e910 con 0x7f6770101070 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59356/0 (socket says 172.21.15.77:59356) 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 -- 172.21.15.77:0/3920660858 learned_addr learned my addr 172.21.15.77:0/3920660858 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676f7fe640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 0x7f677010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 -- 172.21.15.77:0/3920660858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 msgr2=0x7f677010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 0x7f677010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 -- 172.21.15.77:0/3920660858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764005040 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f676effd640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6764004930 tx=0x7f6764009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67640094c0 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6764032e50 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.866+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677010eb90 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.867+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67640433b0 con 0x7f6770101a10 2024-04-26T05:04:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.867+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701170c0 con 0x7f6770101a10 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.868+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67700fc7e0 con 0x7f6770101a10 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.868+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f676402f070 con 0x7f6770101a10 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.868+0000 7f6757fff640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 0x7f6758070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.868+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f67640afad0 con 0x7f6770101a10 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.869+0000 7f676f7fe640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 0x7f6758070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:30.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.869+0000 7f676f7fe640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 0x7f6758070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f67600041d0 tx=0x7f6760004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:30.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:30.873+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6764078e00 con 0x7f6770101a10 2024-04-26T05:04:31.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.092+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f6770108230 con 0x7f6770101a10 2024-04-26T05:04:31.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.093+0000 7f6757fff640 1 -- 172.21.15.77:0/3920660858 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4083 (secure 0 0 0) 0x7f676407c4c0 con 0x7f6770101a10 2024-04-26T05:04:31.094 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:31.094 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:08.603390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":153,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:31.094 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 17 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 msgr2=0x7f6758070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 0x7f6758070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f67600041d0 tx=0x7f6760004330 comp rx=0 tx=0).stop 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 msgr2=0x7f677010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6764004930 tx=0x7f6764009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 shutdown_connections 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f675806e0b0 0x7f6758070560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6770101a10 0x7f677010e3a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 --2- 172.21.15.77:0/3920660858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6770101070 0x7f677010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 >> 172.21.15.77:0/3920660858 conn(0x7f67700f6be0 msgr2=0x7f67700f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 shutdown_connections 2024-04-26T05:04:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.095+0000 7f6775f9d640 1 -- 172.21.15.77:0/3920660858 wait complete. 2024-04-26T05:04:31.106 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:31 smithi077 ceph-mon[91833]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:31.106 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:31 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3556853392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-26T05:04:31.106 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:31 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1358656052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-26T05:04:31.106 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:31 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3042944470' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-26T05:04:31.106 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:31.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-26T05:04:31.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 -- 172.21.15.77:0/1579525509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102260 msgr2=0x7f248c102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 --2- 172.21.15.77:0/1579525509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102260 0x7f248c102630 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f247c009c50 tx=0x7f247c004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:31.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 -- 172.21.15.77:0/1579525509 shutdown_connections 2024-04-26T05:04:31.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 --2- 172.21.15.77:0/1579525509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102c00 0x7f248c0fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 --2- 172.21.15.77:0/1579525509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102260 0x7f248c102630 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 -- 172.21.15.77:0/1579525509 >> 172.21.15.77:0/1579525509 conn(0x7f248c0717e0 msgr2=0x7f248c071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 -- 172.21.15.77:0/1579525509 shutdown_connections 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.201+0000 7f2491eae640 1 -- 172.21.15.77:0/1579525509 wait complete. 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 Processor -- start 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 -- start start 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 0x7f248c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c1137a0 con 0x7f248c102c00 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f2491eae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c10cdf0 con 0x7f248c102260 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59376/0 (socket says 172.21.15.77:59376) 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 -- 172.21.15.77:0/3836363485 learned_addr learned my addr 172.21.15.77:0/3836363485 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:31.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248b7fe640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 0x7f248c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 -- 172.21.15.77:0/3836363485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 msgr2=0x7f248c112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 0x7f248c112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248affd640 1 -- 172.21.15.77:0/3836363485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0098b0 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.202+0000 7f248b7fe640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 0x7f248c112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f248affd640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2480002a50 tx=0x7f2480004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2480007c30 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248000f040 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c10d0d0 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248000db50 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.203+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c10d670 con 0x7f248c102c00 2024-04-26T05:04:31.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.204+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c065530 con 0x7f248c102c00 2024-04-26T05:04:31.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.204+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f248000dcf0 con 0x7f248c102c00 2024-04-26T05:04:31.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.205+0000 7f2473fff640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 0x7f2474070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.205+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f248008d480 con 0x7f248c102c00 2024-04-26T05:04:31.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.205+0000 7f248b7fe640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 0x7f2474070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.205+0000 7f248b7fe640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 0x7f2474070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f247c000f80 tx=0x7f247c02e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.209+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2480056780 con 0x7f248c102c00 2024-04-26T05:04:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:31 smithi078 ceph-mon[79689]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:31 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3556853392' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-26T05:04:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:31 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1358656052' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-26T05:04:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:31 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3042944470' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-26T05:04:31.268 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:31.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.419+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f248c1093a0 con 0x7f248c102c00 2024-04-26T05:04:31.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.420+0000 7f2473fff640 1 -- 172.21.15.77:0/3836363485 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4080 (secure 0 0 0) 0x7f2480059e40 con 0x7f248c102c00 2024-04-26T05:04:31.421 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:31.421 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:09.411167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":154,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 18 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 msgr2=0x7f2474070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 0x7f2474070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f247c000f80 tx=0x7f247c02e040 comp rx=0 tx=0).stop 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 msgr2=0x7f248c1131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2480002a50 tx=0x7f2480004600 comp rx=0 tx=0).stop 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 shutdown_connections 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f247406e0b0 0x7f2474070560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f248c102c00 0x7f248c1131a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 --2- 172.21.15.77:0/3836363485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f248c102260 0x7f248c112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 >> 172.21.15.77:0/3836363485 conn(0x7f248c0717e0 msgr2=0x7f248c0f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 shutdown_connections 2024-04-26T05:04:31.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.422+0000 7f2491eae640 1 -- 172.21.15.77:0/3836363485 wait complete. 2024-04-26T05:04:31.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 -- 172.21.15.77:0/1445787921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 msgr2=0x7f6594102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 --2- 172.21.15.77:0/1445787921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f6594102640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f65840048d0 tx=0x7f6584009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 -- 172.21.15.77:0/1445787921 shutdown_connections 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 --2- 172.21.15.77:0/1445787921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 0x7f6594064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 --2- 172.21.15.77:0/1445787921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f6594102640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 -- 172.21.15.77:0/1445787921 >> 172.21.15.77:0/1445787921 conn(0x7f6594071fd0 msgr2=0x7f65940723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 -- 172.21.15.77:0/1445787921 shutdown_connections 2024-04-26T05:04:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.528+0000 7f659999d640 1 -- 172.21.15.77:0/1445787921 wait complete. 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 Processor -- start 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 -- start start 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 0x7f6594112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594113850 con 0x7f6594102270 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f659999d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659410ce10 con 0x7f6594064390 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f65927fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f65927fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59390/0 (socket says 172.21.15.77:59390) 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f65927fc640 1 -- 172.21.15.77:0/776705334 learned_addr learned my addr 172.21.15.77:0/776705334 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.529+0000 7f6592ffd640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 0x7f6594112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f65927fc640 1 -- 172.21.15.77:0/776705334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 msgr2=0x7f6594112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f65927fc640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 0x7f6594112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f65927fc640 1 -- 172.21.15.77:0/776705334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584005040 con 0x7f6594102270 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f65927fc640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f658800b790 tx=0x7f65880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658800c700 con 0x7f6594102270 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659410d0f0 con 0x7f6594102270 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f658800cd30 con 0x7f6594102270 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6588013370 con 0x7f6594102270 2024-04-26T05:04:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.530+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659410d610 con 0x7f6594102270 2024-04-26T05:04:31.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.531+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65940fd970 con 0x7f6594102270 2024-04-26T05:04:31.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.531+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6588012070 con 0x7f6594102270 2024-04-26T05:04:31.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.532+0000 7f657b7fe640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 0x7f657c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.532+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f658808d870 con 0x7f6594102270 2024-04-26T05:04:31.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.532+0000 7f6592ffd640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 0x7f657c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.532+0000 7f6592ffd640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 0x7f657c070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6584004540 tx=0x7f6584009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.536+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6588056af0 con 0x7f6594102270 2024-04-26T05:04:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.748+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f6594109430 con 0x7f6594102270 2024-04-26T05:04:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.749+0000 7f657b7fe640 1 -- 172.21.15.77:0/776705334 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4936 (secure 0 0 0) 0x7f658805a1b0 con 0x7f6594102270 2024-04-26T05:04:31.750 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:31.750 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:11.414274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi077.urxtgp","rank":0,"incarnation":16,"state":"up:active","state_seq":155,"addr":"172.21.15.77:6829/2377141710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2377141710},{"type":"v1","addr":"172.21.15.77:6829","nonce":2377141710}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14470,"qdb_cluster":[14470]},"id":1}]} 2024-04-26T05:04:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 19 2024-04-26T05:04:31.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 msgr2=0x7f657c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 0x7f657c070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6584004540 tx=0x7f6584009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 msgr2=0x7f65941131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f658800b790 tx=0x7f65880091d0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 shutdown_connections 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f657c06e0b0 0x7f657c070560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6594102270 0x7f65941131c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 --2- 172.21.15.77:0/776705334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6594064390 0x7f6594112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 >> 172.21.15.77:0/776705334 conn(0x7f6594071fd0 msgr2=0x7f65940fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 shutdown_connections 2024-04-26T05:04:31.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.751+0000 7f659999d640 1 -- 172.21.15.77:0/776705334 wait complete. 2024-04-26T05:04:31.763 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-26T05:04:31.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 -- 172.21.15.77:0/1671158823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101070 msgr2=0x7f0050101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 --2- 172.21.15.77:0/1671158823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101070 0x7f0050101440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f00400048d0 tx=0x7f0040009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 -- 172.21.15.77:0/1671158823 shutdown_connections 2024-04-26T05:04:31.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 --2- 172.21.15.77:0/1671158823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101a10 0x7f00500fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 --2- 172.21.15.77:0/1671158823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101070 0x7f0050101440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.857+0000 7f00548ae640 1 -- 172.21.15.77:0/1671158823 >> 172.21.15.77:0/1671158823 conn(0x7f00500f6be0 msgr2=0x7f00500f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 -- 172.21.15.77:0/1671158823 shutdown_connections 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 -- 172.21.15.77:0/1671158823 wait complete. 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 Processor -- start 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 -- start start 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101070 0x7f005010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0050066720 con 0x7f0050101a10 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f00548ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005010e8f0 con 0x7f0050101070 2024-04-26T05:04:31.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f004dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f004dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59406/0 (socket says 172.21.15.77:59406) 2024-04-26T05:04:31.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f004dd9b640 1 -- 172.21.15.77:0/721413383 learned_addr learned my addr 172.21.15.77:0/721413383 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:31.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.858+0000 7f004e59c640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101070 0x7f005010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f004dd9b640 1 -- 172.21.15.77:0/721413383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101070 msgr2=0x7f005010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:31.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f004dd9b640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101070 0x7f005010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f004dd9b640 1 -- 172.21.15.77:0/721413383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040005040 con 0x7f0050101a10 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f004dd9b640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f004400eb70 tx=0x7f004400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0044007c30 con 0x7f0050101a10 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f004400f040 con 0x7f0050101a10 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005010eb70 con 0x7f0050101a10 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f004400cad0 con 0x7f0050101a10 2024-04-26T05:04:31.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.859+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0050117130 con 0x7f0050101a10 2024-04-26T05:04:31.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.860+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00500fc7e0 con 0x7f0050101a10 2024-04-26T05:04:31.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.860+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f004400cc30 con 0x7f0050101a10 2024-04-26T05:04:31.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.861+0000 7f003effd640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 0x7f0038070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:31.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.861+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f004408d7b0 con 0x7f0050101a10 2024-04-26T05:04:31.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.861+0000 7f004e59c640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 0x7f0038070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:31.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.861+0000 7f004e59c640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 0x7f0038070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f0040009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:31.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:31.865+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0044056b60 con 0x7f0050101a10 2024-04-26T05:04:32.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.078+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f0050064350 con 0x7f0050101a10 2024-04-26T05:04:32.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.078+0000 7f003effd640 1 -- 172.21.15.77:0/721413383 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4135 (secure 0 0 0) 0x7f004405a220 con 0x7f0050101a10 2024-04-26T05:04:32.079 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:32.079 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:25.565381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:32.080 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 20 2024-04-26T05:04:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 msgr2=0x7f0038070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 0x7f0038070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0040000c00 tx=0x7f0040009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 msgr2=0x7f005010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f004400eb70 tx=0x7f004400b040 comp rx=0 tx=0).stop 2024-04-26T05:04:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 shutdown_connections 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f003806e0b0 0x7f0038070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0050101a10 0x7f005010e3b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 --2- 172.21.15.77:0/721413383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0050101070 0x7f005010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 >> 172.21.15.77:0/721413383 conn(0x7f00500f6be0 msgr2=0x7f00500f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 shutdown_connections 2024-04-26T05:04:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.080+0000 7f00548ae640 1 -- 172.21.15.77:0/721413383 wait complete. 2024-04-26T05:04:32.091 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:32 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3920660858' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-26T05:04:32.091 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:32 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3836363485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-26T05:04:32.091 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:32 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/776705334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-26T05:04:32.093 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-26T05:04:32.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/2326151579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad4101170 msgr2=0x7f1ad4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/2326151579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad4101170 0x7f1ad4101540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009c50 tx=0x7f1ac4004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:32.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/2326151579 shutdown_connections 2024-04-26T05:04:32.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/2326151579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad40fb150 0x7f1ad40fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/2326151579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad4101170 0x7f1ad4101540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.186+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/2326151579 >> 172.21.15.77:0/2326151579 conn(0x7f1ad4071fd0 msgr2=0x7f1ad40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/2326151579 shutdown_connections 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/2326151579 wait complete. 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 Processor -- start 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 -- start start 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad4101170 0x7f1ad410e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad410e950 con 0x7f1ad40fb150 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad8f1d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad410eac0 con 0x7f1ad4101170 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59420/0 (socket says 172.21.15.77:59420) 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 -- 172.21.15.77:0/76328082 learned_addr learned my addr 172.21.15.77:0/76328082 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 -- 172.21.15.77:0/76328082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad4101170 msgr2=0x7f1ad410e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad4101170 0x7f1ad410e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.187+0000 7f1ad259c640 1 -- 172.21.15.77:0/76328082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac40098b0 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ad259c640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402c1f0 tx=0x7f1ac4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac403a070 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ac4009740 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad410ed40 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac403eb40 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.188+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41171f0 con 0x7f1ad40fb150 2024-04-26T05:04:32.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.189+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad40fc870 con 0x7f1ad40fb150 2024-04-26T05:04:32.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.189+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1ac4032070 con 0x7f1ad40fb150 2024-04-26T05:04:32.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.189+0000 7f1ac2ffd640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 0x7f1abc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.190+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1ac40aff10 con 0x7f1ad40fb150 2024-04-26T05:04:32.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.190+0000 7f1ad1d9b640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 0x7f1abc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.190+0000 7f1ad1d9b640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 0x7f1abc070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4064b40 tx=0x7f1ac8006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.194+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1ac4079300 con 0x7f1ad40fb150 2024-04-26T05:04:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:32 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3920660858' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-26T05:04:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:32 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3836363485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-26T05:04:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:32 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/776705334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-26T05:04:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.406+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f1ad4063f00 con 0x7f1ad40fb150 2024-04-26T05:04:32.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.407+0000 7f1ac2ffd640 1 -- 172.21.15.77:0/76328082 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4146 (secure 0 0 0) 0x7f1ac407c9c0 con 0x7f1ad40fb150 2024-04-26T05:04:32.410 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:32.410 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:25.571994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":0,"incarnation":21,"state":"up:replay","state_seq":3,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 21 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 msgr2=0x7f1abc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 0x7f1abc070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4064b40 tx=0x7f1ac8006d20 comp rx=0 tx=0).stop 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 msgr2=0x7f1ad410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ac402c1f0 tx=0x7f1ac4004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 shutdown_connections 2024-04-26T05:04:32.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad4101170 0x7f1ad410e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1abc06e0b0 0x7f1abc070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 --2- 172.21.15.77:0/76328082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ad40fb150 0x7f1ad410ded0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 >> 172.21.15.77:0/76328082 conn(0x7f1ad4071fd0 msgr2=0x7f1ad40fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 shutdown_connections 2024-04-26T05:04:32.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.409+0000 7f1ad8f1d640 1 -- 172.21.15.77:0/76328082 wait complete. 2024-04-26T05:04:32.421 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-26T05:04:32.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.514+0000 7f0888714640 1 -- 172.21.15.77:0/2087954610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 msgr2=0x7f0880101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.514+0000 7f0888714640 1 --2- 172.21.15.77:0/2087954610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880101440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0878009c50 tx=0x7f0878004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:32.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- 172.21.15.77:0/2087954610 shutdown_connections 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 --2- 172.21.15.77:0/2087954610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0880101a10 0x7f08800fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 --2- 172.21.15.77:0/2087954610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880101440 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- 172.21.15.77:0/2087954610 >> 172.21.15.77:0/2087954610 conn(0x7f08800f6c60 msgr2=0x7f08800f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- 172.21.15.77:0/2087954610 shutdown_connections 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- 172.21.15.77:0/2087954610 wait complete. 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 Processor -- start 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- start start 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0880101a10 0x7f08801120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801126d0 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f0888714640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088010bd20 con 0x7f0880101a10 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f08864b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f08864b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59442/0 (socket says 172.21.15.77:59442) 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.515+0000 7f08864b0640 1 -- 172.21.15.77:0/743391666 learned_addr learned my addr 172.21.15.77:0/743391666 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f08864b0640 1 -- 172.21.15.77:0/743391666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0880101a10 msgr2=0x7f08801120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f08864b0640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0880101a10 0x7f08801120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f08864b0640 1 -- 172.21.15.77:0/743391666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780098b0 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f08864b0640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f087802c1f0 tx=0x7f0878004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087803a070 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0878009740 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087803eb40 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088010bfa0 con 0x7f0880101070 2024-04-26T05:04:32.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.516+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088010c4c0 con 0x7f0880101070 2024-04-26T05:04:32.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.517+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08800fc7e0 con 0x7f0880101070 2024-04-26T05:04:32.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.518+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0878032070 con 0x7f0880101070 2024-04-26T05:04:32.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.518+0000 7f0876ffd640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 0x7f086c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.518+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f08780b01c0 con 0x7f0880101070 2024-04-26T05:04:32.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.518+0000 7f0885caf640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 0x7f086c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.518+0000 7f0885caf640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 0x7f086c070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f088010d450 tx=0x7f0870007c30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.523+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f08780795b0 con 0x7f0880101070 2024-04-26T05:04:32.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.742+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f088010d290 con 0x7f0880101070 2024-04-26T05:04:32.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.743+0000 7f0876ffd640 1 -- 172.21.15.77:0/743391666 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4151 (secure 0 0 0) 0x7f088010d290 con 0x7f0880101070 2024-04-26T05:04:32.744 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:32.744 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:34.718824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":160,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:32.744 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 22 2024-04-26T05:04:32.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.744+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 msgr2=0x7f086c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 0x7f086c070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f088010d450 tx=0x7f0870007c30 comp rx=0 tx=0).stop 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 msgr2=0x7f0880111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f087802c1f0 tx=0x7f0878004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 shutdown_connections 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0880101a10 0x7f08801120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f086c06e0b0 0x7f086c070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 --2- 172.21.15.77:0/743391666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0880101070 0x7f0880111b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 >> 172.21.15.77:0/743391666 conn(0x7f08800f6c60 msgr2=0x7f08800f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 shutdown_connections 2024-04-26T05:04:32.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.745+0000 7f0888714640 1 -- 172.21.15.77:0/743391666 wait complete. 2024-04-26T05:04:32.756 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- 172.21.15.77:0/2935061632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101070 msgr2=0x7fd9b0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 --2- 172.21.15.77:0/2935061632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101070 0x7fd9b0101440 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd9980048d0 tx=0x7fd998009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- 172.21.15.77:0/2935061632 shutdown_connections 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 --2- 172.21.15.77:0/2935061632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101a10 0x7fd9b00fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 --2- 172.21.15.77:0/2935061632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101070 0x7fd9b0101440 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- 172.21.15.77:0/2935061632 >> 172.21.15.77:0/2935061632 conn(0x7fd9b00f6be0 msgr2=0x7fd9b00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- 172.21.15.77:0/2935061632 shutdown_connections 2024-04-26T05:04:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- 172.21.15.77:0/2935061632 wait complete. 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 Processor -- start 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- start start 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101070 0x7fd9b010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0066720 con 0x7fd9b0101a10 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.850+0000 7fd9b679d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b010e900 con 0x7fd9b0101070 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59454/0 (socket says 172.21.15.77:59454) 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 -- 172.21.15.77:0/3861135880 learned_addr learned my addr 172.21.15.77:0/3861135880 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9affff640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101070 0x7fd9b010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 -- 172.21.15.77:0/3861135880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101070 msgr2=0x7fd9b010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101070 0x7fd9b010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 -- 172.21.15.77:0/3861135880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998005040 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9af7fe640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c790 tx=0x7fd9a00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a000a820 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9a000ae50 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b010ebe0 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a00115e0 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.851+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b0117170 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.852+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b00fc7e0 con 0x7fd9b0101a10 2024-04-26T05:04:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.852+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd9a0012400 con 0x7fd9b0101a10 2024-04-26T05:04:32.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.853+0000 7fd9acff9640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 0x7fd99c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:32.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.853+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd9a008ddf0 con 0x7fd9b0101a10 2024-04-26T05:04:32.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.853+0000 7fd9affff640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 0x7fd99c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:32.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.853+0000 7fd9affff640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 0x7fd99c070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd998000c00 tx=0x7fd998009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:32.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:32.858+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd9a00570f0 con 0x7fd9b0101a10 2024-04-26T05:04:33.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.072+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fd9b0065d20 con 0x7fd9b0101a10 2024-04-26T05:04:33.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.073+0000 7fd9acff9640 1 -- 172.21.15.77:0/3861135880 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4148 (secure 0 0 0) 0x7fd9a005a7b0 con 0x7fd9b0101a10 2024-04-26T05:04:33.074 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:33.074 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:34.949478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":161,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:33.074 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 23 2024-04-26T05:04:33.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 msgr2=0x7fd99c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 0x7fd99c070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd998000c00 tx=0x7fd998009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 msgr2=0x7fd9b010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c790 tx=0x7fd9a00043c0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 shutdown_connections 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd99c06e0b0 0x7fd99c070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9b0101a10 0x7fd9b010e390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 --2- 172.21.15.77:0/3861135880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9b0101070 0x7fd9b010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 >> 172.21.15.77:0/3861135880 conn(0x7fd9b00f6be0 msgr2=0x7fd9b00f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 shutdown_connections 2024-04-26T05:04:33.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.075+0000 7fd9b679d640 1 -- 172.21.15.77:0/3861135880 wait complete. 2024-04-26T05:04:33.087 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-26T05:04:33.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 -- 172.21.15.77:0/2187486656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 msgr2=0x7fc314101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 --2- 172.21.15.77:0/2187486656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc314101440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3080048d0 tx=0x7fc308009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 -- 172.21.15.77:0/2187486656 shutdown_connections 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 --2- 172.21.15.77:0/2187486656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc3140fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 --2- 172.21.15.77:0/2187486656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc314101440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 -- 172.21.15.77:0/2187486656 >> 172.21.15.77:0/2187486656 conn(0x7fc3140f6be0 msgr2=0x7fc3140f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 -- 172.21.15.77:0/2187486656 shutdown_connections 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.179+0000 7fc31b11d640 1 -- 172.21.15.77:0/2187486656 wait complete. 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 Processor -- start 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 -- start start 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc31410c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314066680 con 0x7fc314101070 2024-04-26T05:04:33.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc31b11d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31410c730 con 0x7fc314101a10 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59472/0 (socket says 172.21.15.77:59472) 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 -- 172.21.15.77:0/4183647315 learned_addr learned my addr 172.21.15.77:0/4183647315 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc313fff640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc31410c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 -- 172.21.15.77:0/4183647315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 msgr2=0x7fc31410c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc31410c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc318eb9640 1 -- 172.21.15.77:0/4183647315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308005040 con 0x7fc314101070 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.180+0000 7fc313fff640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc31410c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc318eb9640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc308004930 tx=0x7fc308009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3080094c0 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc308032e50 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31410c9b0 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3080433b0 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.181+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3141194c0 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.182+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3140fc7e0 con 0x7fc314101070 2024-04-26T05:04:33.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.182+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc30802f070 con 0x7fc314101070 2024-04-26T05:04:33.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.182+0000 7fc3117fa640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 0x7fc304070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.183+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc3080afbd0 con 0x7fc314101070 2024-04-26T05:04:33.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.183+0000 7fc313fff640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 0x7fc304070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.183+0000 7fc313fff640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 0x7fc304070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc300006fd0 tx=0x7fc300008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.187+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc308078e50 con 0x7fc314101070 2024-04-26T05:04:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:33 smithi078 ceph-mon[79689]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:33 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/721413383' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-26T05:04:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:33 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/76328082' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-26T05:04:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:33 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/743391666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-26T05:04:33.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.398+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fc314064350 con 0x7fc314101070 2024-04-26T05:04:33.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.399+0000 7fc3117fa640 1 -- 172.21.15.77:0/4183647315 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4157 (secure 0 0 0) 0x7fc30807c510 con 0x7fc314101070 2024-04-26T05:04:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:33.400 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:36.954370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":0,"incarnation":21,"state":"up:active","state_seq":162,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-04-26T05:04:33.400 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 24 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 msgr2=0x7fc304070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 0x7fc304070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc300006fd0 tx=0x7fc300008040 comp rx=0 tx=0).stop 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 msgr2=0x7fc31410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc308004930 tx=0x7fc308009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 shutdown_connections 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc314101a10 0x7fc31410c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc30406e0b0 0x7fc304070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 --2- 172.21.15.77:0/4183647315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc314101070 0x7fc31410bcb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 >> 172.21.15.77:0/4183647315 conn(0x7fc3140f6be0 msgr2=0x7fc3140f7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 shutdown_connections 2024-04-26T05:04:33.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.401+0000 7fc31b11d640 1 -- 172.21.15.77:0/4183647315 wait complete. 2024-04-26T05:04:33.411 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:33 smithi077 ceph-mon[91833]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:33.411 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:33 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/721413383' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-26T05:04:33.412 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:33 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/76328082' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-26T05:04:33.412 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:33 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/743391666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-26T05:04:33.413 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 -- 172.21.15.77:0/2132420797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 msgr2=0x7f9390064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 --2- 172.21.15.77:0/2132420797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390064800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f93840048d0 tx=0x7f9384009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 -- 172.21.15.77:0/2132420797 shutdown_connections 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 --2- 172.21.15.77:0/2132420797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390064800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 --2- 172.21.15.77:0/2132420797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 0x7f9390102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 -- 172.21.15.77:0/2132420797 >> 172.21.15.77:0/2132420797 conn(0x7f9390071fd0 msgr2=0x7f93900723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 -- 172.21.15.77:0/2132420797 shutdown_connections 2024-04-26T05:04:33.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.507+0000 7f93956ae640 1 -- 172.21.15.77:0/2132420797 wait complete. 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 Processor -- start 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 -- start start 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 0x7f93901131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93901137c0 con 0x7f9390064390 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f93956ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939010ce10 con 0x7f9390102270 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59496/0 (socket says 172.21.15.77:59496) 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 -- 172.21.15.77:0/1849764894 learned_addr learned my addr 172.21.15.77:0/1849764894 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938e7fc640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 0x7f93901131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 -- 172.21.15.77:0/1849764894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 msgr2=0x7f93901131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 0x7f93901131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.508+0000 7f938effd640 1 -- 172.21.15.77:0/1849764894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384005040 con 0x7f9390064390 2024-04-26T05:04:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f938effd640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f938000eb70 tx=0x7f938000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9380007c30 con 0x7f9390064390 2024-04-26T05:04:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f938000f040 con 0x7f9390064390 2024-04-26T05:04:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939010d090 con 0x7f9390064390 2024-04-26T05:04:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93800095b0 con 0x7f9390064390 2024-04-26T05:04:33.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.509+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939010d5b0 con 0x7f9390064390 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.510+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900fd970 con 0x7f9390064390 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.510+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9380009730 con 0x7f9390064390 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.511+0000 7f93777fe640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 0x7f9378070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.511+0000 7f938e7fc640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 0x7f9378070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.511+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f938008d9c0 con 0x7f9390064390 2024-04-26T05:04:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.511+0000 7f938e7fc640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 0x7f9378070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.516+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9380056c40 con 0x7f9390064390 2024-04-26T05:04:33.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.725+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f9390063f00 con 0x7f9390064390 2024-04-26T05:04:33.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.725+0000 7f93777fe640 1 -- 172.21.15.77:0/1849764894 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+5004 (secure 0 0 0) 0x7f938005a300 con 0x7f9390064390 2024-04-26T05:04:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:33.726 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:36.954370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi078.wkrgkp","rank":0,"incarnation":21,"state":"up:active","state_seq":162,"addr":"172.21.15.78:6825/3670372668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3670372668},{"type":"v1","addr":"172.21.15.78:6825","nonce":3670372668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24261,"qdb_cluster":[24261]},"id":1}]} 2024-04-26T05:04:33.726 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 25 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 msgr2=0x7f9378070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 0x7f9378070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 msgr2=0x7f9390112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f938000eb70 tx=0x7f938000b040 comp rx=0 tx=0).stop 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 shutdown_connections 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9390102270 0x7f93901131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f937806e0b0 0x7f9378070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 --2- 172.21.15.77:0/1849764894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9390064390 0x7f9390112c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 >> 172.21.15.77:0/1849764894 conn(0x7f9390071fd0 msgr2=0x7f93900fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 shutdown_connections 2024-04-26T05:04:33.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.727+0000 7f93956ae640 1 -- 172.21.15.77:0/1849764894 wait complete. 2024-04-26T05:04:33.739 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-26T05:04:33.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.832+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2088419632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 msgr2=0x7fc438101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.832+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2088419632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc438101540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4300048d0 tx=0x7fc430009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2088419632 shutdown_connections 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2088419632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 0x7fc4380fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2088419632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc438101540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2088419632 >> 172.21.15.77:0/2088419632 conn(0x7fc438071fd0 msgr2=0x7fc4380723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2088419632 shutdown_connections 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2088419632 wait complete. 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 Processor -- start 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- start start 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 0x7fc43810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43810e930 con 0x7fc438101170 2024-04-26T05:04:33.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc43cf9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43810eaa0 con 0x7fc4380fb150 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc435d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc435d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59510/0 (socket says 172.21.15.77:59510) 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.833+0000 7fc435d9b640 1 -- 172.21.15.77:0/2871868998 learned_addr learned my addr 172.21.15.77:0/2871868998 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc43659c640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 0x7fc43810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc435d9b640 1 -- 172.21.15.77:0/2871868998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 msgr2=0x7fc43810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc435d9b640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 0x7fc43810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc435d9b640 1 -- 172.21.15.77:0/2871868998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430005040 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc435d9b640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b790 tx=0x7fc42c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc42c00c700 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc42c00cd30 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc438116f20 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc42c013370 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.834+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc438117250 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.835+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4380fc870 con 0x7fc438101170 2024-04-26T05:04:33.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.835+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc42c012070 con 0x7fc438101170 2024-04-26T05:04:33.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.836+0000 7fc426ffd640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 0x7fc420070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:33.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.836+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc42c08d780 con 0x7fc438101170 2024-04-26T05:04:33.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.836+0000 7fc43659c640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 0x7fc420070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:33.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.836+0000 7fc43659c640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 0x7fc420070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc430031870 tx=0x7fc430009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:33.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:33.840+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc42c056a00 con 0x7fc438101170 2024-04-26T05:04:34.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.057+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fc438063f00 con 0x7fc438101170 2024-04-26T05:04:34.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.057+0000 7fc426ffd640 1 -- 172.21.15.77:0/2871868998 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4203 (secure 0 0 0) 0x7fc42c05a0c0 con 0x7fc438101170 2024-04-26T05:04:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:34.059 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:50.182005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:34.059 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 26 2024-04-26T05:04:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 msgr2=0x7fc420070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 0x7fc420070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc430031870 tx=0x7fc430009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 msgr2=0x7fc43810e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00b790 tx=0x7fc42c0091d0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 shutdown_connections 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc42006e0b0 0x7fc420070560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101170 0x7fc43810e3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 --2- 172.21.15.77:0/2871868998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4380fb150 0x7fc43810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 >> 172.21.15.77:0/2871868998 conn(0x7fc438071fd0 msgr2=0x7fc4380fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 shutdown_connections 2024-04-26T05:04:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.060+0000 7fc43cf9d640 1 -- 172.21.15.77:0/2871868998 wait complete. 2024-04-26T05:04:34.071 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-26T05:04:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 -- 172.21.15.77:0/1703952250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 msgr2=0x7f96200fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 --2- 172.21.15.77:0/1703952250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f96200fb0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f96140048d0 tx=0x7f9614009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 -- 172.21.15.77:0/1703952250 shutdown_connections 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 --2- 172.21.15.77:0/1703952250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f96200fb0a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 --2- 172.21.15.77:0/1703952250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 0x7f9620101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 -- 172.21.15.77:0/1703952250 >> 172.21.15.77:0/1703952250 conn(0x7f96200f6c60 msgr2=0x7f96200f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 -- 172.21.15.77:0/1703952250 shutdown_connections 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 -- 172.21.15.77:0/1703952250 wait complete. 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.166+0000 7f962686e640 1 Processor -- start 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f962686e640 1 -- start start 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f962686e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 0x7f962010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f962686e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f962686e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9620066740 con 0x7f9620101a10 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f962686e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962010e8f0 con 0x7f9620101070 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59534/0 (socket says 172.21.15.77:59534) 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 -- 172.21.15.77:0/2813522674 learned_addr learned my addr 172.21.15.77:0/2813522674 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961ffff640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 0x7f962010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 -- 172.21.15.77:0/2813522674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 msgr2=0x7f962010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 0x7f962010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 -- 172.21.15.77:0/2813522674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614005040 con 0x7f9620101a10 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.167+0000 7f961f7fe640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9614004930 tx=0x7f9614009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.168+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96140094c0 con 0x7f9620101a10 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.168+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962010eb70 con 0x7f9620101a10 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.168+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9614032e50 con 0x7f9620101a10 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.168+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96140433b0 con 0x7f9620101a10 2024-04-26T05:04:34.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.168+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96201170d0 con 0x7f9620101a10 2024-04-26T05:04:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.169+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96200fc7e0 con 0x7f9620101a10 2024-04-26T05:04:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.169+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f961402f070 con 0x7f9620101a10 2024-04-26T05:04:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.169+0000 7f961cff9640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 0x7f9608070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.170+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f96140afa20 con 0x7f9620101a10 2024-04-26T05:04:34.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.170+0000 7f961ffff640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 0x7f9608070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.170+0000 7f961ffff640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 0x7f9608070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9610004ed0 tx=0x7f961000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.174+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9614078d50 con 0x7f9620101a10 2024-04-26T05:04:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:34 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/3861135880' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-26T05:04:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:34 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/4183647315' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-26T05:04:34.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:34 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1849764894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-26T05:04:34.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.386+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f9620064350 con 0x7f9620101a10 2024-04-26T05:04:34.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.387+0000 7f961cff9640 1 -- 172.21.15.77:0/2813522674 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4214 (secure 0 0 0) 0x7f961407c410 con 0x7f9620101a10 2024-04-26T05:04:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:34.388 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:50.191289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":0,"incarnation":27,"state":"up:replay","state_seq":2,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:34.388 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2024-04-26T05:04:34.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 msgr2=0x7f9608070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 0x7f9608070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9610004ed0 tx=0x7f961000b040 comp rx=0 tx=0).stop 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 msgr2=0x7f962010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9614004930 tx=0x7f9614009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 shutdown_connections 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f960806e0b0 0x7f9608070560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9620101a10 0x7f962010e3b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 --2- 172.21.15.77:0/2813522674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9620101070 0x7f962010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 >> 172.21.15.77:0/2813522674 conn(0x7f96200f6c60 msgr2=0x7f96200f7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 shutdown_connections 2024-04-26T05:04:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.389+0000 7f962686e640 1 -- 172.21.15.77:0/2813522674 wait complete. 2024-04-26T05:04:34.400 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:34 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/3861135880' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-26T05:04:34.400 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:34 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/4183647315' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-26T05:04:34.400 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:34 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1849764894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-26T05:04:34.401 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-26T05:04:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 -- 172.21.15.77:0/2477851589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 msgr2=0x7f83cc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 --2- 172.21.15.77:0/2477851589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc1025b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009c50 tx=0x7f83c8004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 -- 172.21.15.77:0/2477851589 shutdown_connections 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 --2- 172.21.15.77:0/2477851589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83cc102b80 0x7f83cc0fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 --2- 172.21.15.77:0/2477851589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc1025b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 -- 172.21.15.77:0/2477851589 >> 172.21.15.77:0/2477851589 conn(0x7f83cc0f7db0 msgr2=0x7f83cc0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 -- 172.21.15.77:0/2477851589 shutdown_connections 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.496+0000 7f83d462e640 1 -- 172.21.15.77:0/2477851589 wait complete. 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 Processor -- start 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 -- start start 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83cc102b80 0x7f83cc113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc113750 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d462e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc10cdf0 con 0x7f83cc102b80 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59550/0 (socket says 172.21.15.77:59550) 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 -- 172.21.15.77:0/4227499370 learned_addr learned my addr 172.21.15.77:0/4227499370 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 -- 172.21.15.77:0/4227499370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83cc102b80 msgr2=0x7f83cc113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83cc102b80 0x7f83cc113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.497+0000 7f83d23ca640 1 -- 172.21.15.77:0/4227499370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80098b0 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83d23ca640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f83c8004a30 tx=0x7f83c8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c803a070 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc10d070 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83c8035b20 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c803eaa0 con 0x7f83cc1021e0 2024-04-26T05:04:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.498+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc10d610 con 0x7f83cc1021e0 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.499+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc0fd950 con 0x7f83cc1021e0 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.499+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83c8032070 con 0x7f83cc1021e0 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.500+0000 7f83c2ffd640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 0x7f83b8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.500+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f83c80b0480 con 0x7f83cc1021e0 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.500+0000 7f83d1bc9640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 0x7f83b8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.500+0000 7f83d1bc9640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 0x7f83b8070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83cc10e4d0 tx=0x7f83bc007c30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.504+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83c8079700 con 0x7f83cc1021e0 2024-04-26T05:04:34.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.714+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f83cc109320 con 0x7f83cc1021e0 2024-04-26T05:04:34.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.715+0000 7f83c2ffd640 1 -- 172.21.15.77:0/4227499370 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4219 (secure 0 0 0) 0x7f83c807cdc0 con 0x7f83cc1021e0 2024-04-26T05:04:34.716 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:34.716 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:00:59.684361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":0,"incarnation":27,"state":"up:reconnect","state_seq":165,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:34.716 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-04-26T05:04:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 msgr2=0x7f83b8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 0x7f83b8070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83cc10e4d0 tx=0x7f83bc007c30 comp rx=0 tx=0).stop 2024-04-26T05:04:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 msgr2=0x7f83cc112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f83c8004a30 tx=0x7f83c8004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 shutdown_connections 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83cc102b80 0x7f83cc113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f83b806e0b0 0x7f83b8070560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 --2- 172.21.15.77:0/4227499370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83cc1021e0 0x7f83cc112c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 >> 172.21.15.77:0/4227499370 conn(0x7f83cc0f7db0 msgr2=0x7f83cc0f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 shutdown_connections 2024-04-26T05:04:34.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.717+0000 7f83d462e640 1 -- 172.21.15.77:0/4227499370 wait complete. 2024-04-26T05:04:34.729 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-26T05:04:34.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 -- 172.21.15.77:0/1143398538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0880fb9b0 msgr2=0x7fa0880fbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 --2- 172.21.15.77:0/1143398538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0880fb9b0 0x7fa0880fbe00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0048d0 tx=0x7fa07c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 -- 172.21.15.77:0/1143398538 shutdown_connections 2024-04-26T05:04:34.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 --2- 172.21.15.77:0/1143398538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0880fb9b0 0x7fa0880fbe00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 --2- 172.21.15.77:0/1143398538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa088101110 0x7fa0881014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 -- 172.21.15.77:0/1143398538 >> 172.21.15.77:0/1143398538 conn(0x7fa088070900 msgr2=0x7fa088070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:34.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 -- 172.21.15.77:0/1143398538 shutdown_connections 2024-04-26T05:04:34.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.823+0000 7fa08fc65640 1 -- 172.21.15.77:0/1143398538 wait complete. 2024-04-26T05:04:34.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 Processor -- start 2024-04-26T05:04:34.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 -- start start 2024-04-26T05:04:34.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0880fb9b0 0x7fa088112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088113850 con 0x7fa088101110 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08fc65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08810cea0 con 0x7fa0880fb9b0 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59562/0 (socket says 172.21.15.77:59562) 2024-04-26T05:04:34.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 -- 172.21.15.77:0/315179147 learned_addr learned my addr 172.21.15.77:0/315179147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08da01640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0880fb9b0 0x7fa088112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 -- 172.21.15.77:0/315179147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0880fb9b0 msgr2=0x7fa088112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0880fb9b0 0x7fa088112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 -- 172.21.15.77:0/315179147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c005040 con 0x7fa088101110 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa08d200640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa07c004930 tx=0x7fa07c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.824+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07c0094c0 con 0x7fa088101110 2024-04-26T05:04:34.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.825+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08810d120 con 0x7fa088101110 2024-04-26T05:04:34.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.825+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa07c032e50 con 0x7fa088101110 2024-04-26T05:04:34.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.825+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07c0433b0 con 0x7fa088101110 2024-04-26T05:04:34.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.825+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08810d5e0 con 0x7fa088101110 2024-04-26T05:04:34.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.825+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0880fc9c0 con 0x7fa088101110 2024-04-26T05:04:34.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.826+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa07c02f070 con 0x7fa088101110 2024-04-26T05:04:34.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.826+0000 7fa0867fc640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 0x7fa070070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:34.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.826+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa07c0afb50 con 0x7fa088101110 2024-04-26T05:04:34.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.826+0000 7fa08da01640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 0x7fa070070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:34.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.827+0000 7fa08da01640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 0x7fa070070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa080004550 tx=0x7fa080009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:34.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:34.831+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa07c078e80 con 0x7fa088101110 2024-04-26T05:04:35.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.044+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fa088108230 con 0x7fa088101110 2024-04-26T05:04:35.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.045+0000 7fa0867fc640 1 -- 172.21.15.77:0/315179147 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4216 (secure 0 0 0) 0x7fa07c07c540 con 0x7fa088101110 2024-04-26T05:04:35.046 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:35.046 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:00.039508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":0,"incarnation":27,"state":"up:rejoin","state_seq":166,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 msgr2=0x7fa070070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 0x7fa070070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa080004550 tx=0x7fa080009040 comp rx=0 tx=0).stop 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 msgr2=0x7fa088113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa07c004930 tx=0x7fa07c009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 shutdown_connections 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fa07006e0b0 0x7fa070070560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa088101110 0x7fa088113250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 --2- 172.21.15.77:0/315179147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0880fb9b0 0x7fa088112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 >> 172.21.15.77:0/315179147 conn(0x7fa088070900 msgr2=0x7fa0880f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 shutdown_connections 2024-04-26T05:04:35.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.047+0000 7fa08fc65640 1 -- 172.21.15.77:0/315179147 wait complete. 2024-04-26T05:04:35.059 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-26T05:04:35.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- 172.21.15.77:0/3490089235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 msgr2=0x7fc1f00fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 --2- 172.21.15.77:0/3490089235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f00fb0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80048d0 tx=0x7fc1d8009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- 172.21.15.77:0/3490089235 shutdown_connections 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 --2- 172.21.15.77:0/3490089235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f00fb0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 --2- 172.21.15.77:0/3490089235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- 172.21.15.77:0/3490089235 >> 172.21.15.77:0/3490089235 conn(0x7fc1f00f6be0 msgr2=0x7fc1f00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- 172.21.15.77:0/3490089235 shutdown_connections 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- 172.21.15.77:0/3490089235 wait complete. 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 Processor -- start 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.153+0000 7fc1f635d640 1 -- start start 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0115a30 con 0x7fc1f0101a10 2024-04-26T05:04:35.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0115ba0 con 0x7fc1f0101070 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59580/0 (socket says 172.21.15.77:59580) 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 -- 172.21.15.77:0/1315021882 learned_addr learned my addr 172.21.15.77:0/1315021882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1effff640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 -- 172.21.15.77:0/1315021882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 msgr2=0x7fc1f0114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0114ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 -- 172.21.15.77:0/1315021882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8005040 con 0x7fc1f0101a10 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1effff640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ef7fe640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004930 tx=0x7fc1d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d80094c0 con 0x7fc1f0101a10 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1d8032e50 con 0x7fc1f0101a10 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f010f1d0 con 0x7fc1f0101a10 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d80433b0 con 0x7fc1f0101a10 2024-04-26T05:04:35.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.154+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f010f690 con 0x7fc1f0101a10 2024-04-26T05:04:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.155+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f00fc7e0 con 0x7fc1f0101a10 2024-04-26T05:04:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.156+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc1d802f070 con 0x7fc1f0101a10 2024-04-26T05:04:35.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.156+0000 7fc1ecff9640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 0x7fc1dc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.156+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc1d80afaa0 con 0x7fc1f0101a10 2024-04-26T05:04:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.156+0000 7fc1effff640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 0x7fc1dc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.157+0000 7fc1effff640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 0x7fc1dc070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0007fd0 tx=0x7fc1e0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.161+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc1d8078d20 con 0x7fc1f0101a10 2024-04-26T05:04:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:35 smithi078 ceph-mon[79689]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:35 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2871868998' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-26T05:04:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:35 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2813522674' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-26T05:04:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:35 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/4227499370' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-26T05:04:35.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.370+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fc1f0064350 con 0x7fc1f0101a10 2024-04-26T05:04:35.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.371+0000 7fc1ecff9640 1 -- 172.21.15.77:0/1315021882 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+4225 (secure 0 0 0) 0x7fc1d807c3e0 con 0x7fc1f0101a10 2024-04-26T05:04:35.371 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:35.372 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:02.042359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":0,"incarnation":27,"state":"up:active","state_seq":167,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24275,"qdb_cluster":[24275]},"id":1}]} 2024-04-26T05:04:35.372 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-04-26T05:04:35.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 msgr2=0x7fc1dc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 0x7fc1dc070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0007fd0 tx=0x7fc1e0009e90 comp rx=0 tx=0).stop 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 msgr2=0x7fc1f0115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004930 tx=0x7fc1d8009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 shutdown_connections 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fc1dc06e0b0 0x7fc1dc070560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f0101a10 0x7fc1f0115430 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 --2- 172.21.15.77:0/1315021882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f0101070 0x7fc1f0114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 >> 172.21.15.77:0/1315021882 conn(0x7fc1f00f6be0 msgr2=0x7fc1f00f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 shutdown_connections 2024-04-26T05:04:35.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.373+0000 7fc1f635d640 1 -- 172.21.15.77:0/1315021882 wait complete. 2024-04-26T05:04:35.383 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:35 smithi077 ceph-mon[91833]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:35.383 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:35 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2871868998' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-26T05:04:35.384 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:35 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2813522674' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-26T05:04:35.384 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:35 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/4227499370' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-26T05:04:35.385 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-26T05:04:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 -- 172.21.15.77:0/1378604331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 msgr2=0x7efd481027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 --2- 172.21.15.77:0/1378604331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd481027c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efd440048d0 tx=0x7efd44009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 -- 172.21.15.77:0/1378604331 shutdown_connections 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 --2- 172.21.15.77:0/1378604331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd480faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 --2- 172.21.15.77:0/1378604331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd481027c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 -- 172.21.15.77:0/1378604331 >> 172.21.15.77:0/1378604331 conn(0x7efd48071990 msgr2=0x7efd48071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 -- 172.21.15.77:0/1378604331 shutdown_connections 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.478+0000 7efd50614640 1 -- 172.21.15.77:0/1378604331 wait complete. 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 Processor -- start 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 -- start start 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd48113410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48113a10 con 0x7efd481023f0 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4810d060 con 0x7efd48102d90 2024-04-26T05:04:35.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59596/0 (socket says 172.21.15.77:59596) 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 -- 172.21.15.77:0/271117078 learned_addr learned my addr 172.21.15.77:0/271117078 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4dbaf640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd48113410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 -- 172.21.15.77:0/271117078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 msgr2=0x7efd48113410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd48113410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 -- 172.21.15.77:0/271117078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44005040 con 0x7efd481023f0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4dbaf640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd48113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd4e3b0640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efd44004930 tx=0x7efd44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd440094c0 con 0x7efd481023f0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd44032e50 con 0x7efd481023f0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4810d2e0 con 0x7efd481023f0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd440433b0 con 0x7efd481023f0 2024-04-26T05:04:35.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.479+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4810d7a0 con 0x7efd481023f0 2024-04-26T05:04:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.480+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd48065570 con 0x7efd481023f0 2024-04-26T05:04:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.481+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efd4402f070 con 0x7efd481023f0 2024-04-26T05:04:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.481+0000 7efd3effd640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 0x7efd34070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.481+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efd440afa30 con 0x7efd481023f0 2024-04-26T05:04:35.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.481+0000 7efd4dbaf640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 0x7efd34070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.482+0000 7efd4dbaf640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 0x7efd34070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd4810e790 tx=0x7efd3800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.486+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efd44078cb0 con 0x7efd481023f0 2024-04-26T05:04:35.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.697+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7efd480638a0 con 0x7efd481023f0 2024-04-26T05:04:35.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.698+0000 7efd3effd640 1 -- 172.21.15.77:0/271117078 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+5072 (secure 0 0 0) 0x7efd4407c370 con 0x7efd481023f0 2024-04-26T05:04:35.699 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:35.699 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:02.042359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi078.nihewy","rank":0,"incarnation":27,"state":"up:active","state_seq":167,"addr":"172.21.15.78:6827/4041373129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":4041373129},{"type":"v1","addr":"172.21.15.78:6827","nonce":4041373129}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24275,"qdb_cluster":[24275]},"id":1}]} 2024-04-26T05:04:35.700 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 31 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.699+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 msgr2=0x7efd34070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.699+0000 7efd50614640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 0x7efd34070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd4810e790 tx=0x7efd3800b040 comp rx=0 tx=0).stop 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.699+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 msgr2=0x7efd48112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.699+0000 7efd50614640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efd44004930 tx=0x7efd44009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 shutdown_connections 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd48102d90 0x7efd48113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7efd3406e0b0 0x7efd34070560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 --2- 172.21.15.77:0/271117078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd481023f0 0x7efd48112ed0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 >> 172.21.15.77:0/271117078 conn(0x7efd48071990 msgr2=0x7efd48105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 shutdown_connections 2024-04-26T05:04:35.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.700+0000 7efd50614640 1 -- 172.21.15.77:0/271117078 wait complete. 2024-04-26T05:04:35.711 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/1400864451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 msgr2=0x7f1b640fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/1400864451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b640fa8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b600048d0 tx=0x7f1b60009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/1400864451 shutdown_connections 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/1400864451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b640fa8c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/1400864451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 0x7f1b64102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/1400864451 >> 172.21.15.77:0/1400864451 conn(0x7f1b640717e0 msgr2=0x7f1b64071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:35.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/1400864451 shutdown_connections 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/1400864451 wait complete. 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 Processor -- start 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.806+0000 7f1b6bf5d640 1 -- start start 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 0x7f1b64112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64113890 con 0x7f1b64102bd0 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6410cee0 con 0x7f1b64102230 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59616/0 (socket says 172.21.15.77:59616) 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 -- 172.21.15.77:0/2729341831 learned_addr learned my addr 172.21.15.77:0/2729341831 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b69cf9640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 0x7f1b64112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 -- 172.21.15.77:0/2729341831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 msgr2=0x7f1b64112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 0x7f1b64112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 -- 172.21.15.77:0/2729341831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60005040 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b694f8640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002bda0 tx=0x7f1b60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b600094c0 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6410d160 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b60032e50 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6003b640 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.807+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6410d680 con 0x7f1b64102bd0 2024-04-26T05:04:35.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.808+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b64064350 con 0x7f1b64102bd0 2024-04-26T05:04:35.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.809+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1b6002f070 con 0x7f1b64102bd0 2024-04-26T05:04:35.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.809+0000 7f1b567fc640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 0x7f1b50070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:35.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.809+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1b600b00c0 con 0x7f1b64102bd0 2024-04-26T05:04:35.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.809+0000 7f1b69cf9640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 0x7f1b50070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:35.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.810+0000 7f1b69cf9640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 0x7f1b50070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b580041d0 tx=0x7f1b58009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:35.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:35.814+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1b60079340 con 0x7f1b64102bd0 2024-04-26T05:04:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.026+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f1b64065530 con 0x7f1b64102bd0 2024-04-26T05:04:36.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.026+0000 7f1b567fc640 1 -- 172.21.15.77:0/2729341831 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+4271 (secure 0 0 0) 0x7f1b6002d040 con 0x7f1b64102bd0 2024-04-26T05:04:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:36.027 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:13.968935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:36.027 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-04-26T05:04:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 msgr2=0x7f1b50070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 0x7f1b50070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b580041d0 tx=0x7f1b58009040 comp rx=0 tx=0).stop 2024-04-26T05:04:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 msgr2=0x7f1b64113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b6002bda0 tx=0x7f1b60009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 shutdown_connections 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f1b5006e0b0 0x7f1b50070560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b64102bd0 0x7f1b64113290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 --2- 172.21.15.77:0/2729341831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b64102230 0x7f1b64112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 >> 172.21.15.77:0/2729341831 conn(0x7f1b640717e0 msgr2=0x7f1b640fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 shutdown_connections 2024-04-26T05:04:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.028+0000 7f1b6bf5d640 1 -- 172.21.15.77:0/2729341831 wait complete. 2024-04-26T05:04:36.041 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-26T05:04:36.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 -- 172.21.15.77:0/1260229461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 msgr2=0x7f7b70102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 --2- 172.21.15.77:0/1260229461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70102640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009c50 tx=0x7f7b60004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 -- 172.21.15.77:0/1260229461 shutdown_connections 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 --2- 172.21.15.77:0/1260229461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 0x7f7b70064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 --2- 172.21.15.77:0/1260229461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70102640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 -- 172.21.15.77:0/1260229461 >> 172.21.15.77:0/1260229461 conn(0x7f7b70071fd0 msgr2=0x7f7b700723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 -- 172.21.15.77:0/1260229461 shutdown_connections 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.135+0000 7f7b754ae640 1 -- 172.21.15.77:0/1260229461 wait complete. 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 Processor -- start 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 -- start start 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 0x7f7b70112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70113770 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b754ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7010cdc0 con 0x7f7b70064390 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59630/0 (socket says 172.21.15.77:59630) 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 -- 172.21.15.77:0/778516166 learned_addr learned my addr 172.21.15.77:0/778516166 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6effd640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 0x7f7b70112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 -- 172.21.15.77:0/778516166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 msgr2=0x7f7b70112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 0x7f7b70112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.136+0000 7f7b6e7fc640 1 -- 172.21.15.77:0/778516166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600098b0 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b6e7fc640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b64002a50 tx=0x7f7b64004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b64007c30 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7010d0a0 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b6400f040 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6400db50 con 0x7f7b70102270 2024-04-26T05:04:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.137+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7010d640 con 0x7f7b70102270 2024-04-26T05:04:36.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.138+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b700fd970 con 0x7f7b70102270 2024-04-26T05:04:36.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.138+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b6401c400 con 0x7f7b70102270 2024-04-26T05:04:36.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.139+0000 7f7b577fe640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 0x7f7b58070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.139+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7b6408d470 con 0x7f7b70102270 2024-04-26T05:04:36.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.139+0000 7f7b6effd640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 0x7f7b58070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.139+0000 7f7b6effd640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 0x7f7b58070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004a30 tx=0x7f7b60037040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.143+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7b640567a0 con 0x7f7b70102270 2024-04-26T05:04:36.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.359+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f7b70109370 con 0x7f7b70102270 2024-04-26T05:04:36.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.359+0000 7f7b577fe640 1 -- 172.21.15.77:0/778516166 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+4350 (secure 0 0 0) 0x7f7b64059e60 con 0x7f7b70102270 2024-04-26T05:04:36.360 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:36.360 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:13.973602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 33 2024-04-26T05:04:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 msgr2=0x7f7b58070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 0x7f7b58070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004a30 tx=0x7f7b60037040 comp rx=0 tx=0).stop 2024-04-26T05:04:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 msgr2=0x7f7b70113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b64002a50 tx=0x7f7b64004600 comp rx=0 tx=0).stop 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 shutdown_connections 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7b5806e0b0 0x7f7b58070560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b70102270 0x7f7b70113170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 --2- 172.21.15.77:0/778516166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b70064390 0x7f7b70112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.361+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 >> 172.21.15.77:0/778516166 conn(0x7f7b70071fd0 msgr2=0x7f7b700fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.362+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 shutdown_connections 2024-04-26T05:04:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.362+0000 7f7b754ae640 1 -- 172.21.15.77:0/778516166 wait complete. 2024-04-26T05:04:36.372 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:36 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/315179147' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-26T05:04:36.372 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:36 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1315021882' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-26T05:04:36.372 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:36 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/271117078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-26T05:04:36.373 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 -- 172.21.15.77:0/2401521008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 msgr2=0x7effc81027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 --2- 172.21.15.77:0/2401521008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc81027c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7effbc0048d0 tx=0x7effbc009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 -- 172.21.15.77:0/2401521008 shutdown_connections 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 --2- 172.21.15.77:0/2401521008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 0x7effc80faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 --2- 172.21.15.77:0/2401521008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc81027c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 -- 172.21.15.77:0/2401521008 >> 172.21.15.77:0/2401521008 conn(0x7effc8071990 msgr2=0x7effc8071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 -- 172.21.15.77:0/2401521008 shutdown_connections 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.467+0000 7effcecae640 1 -- 172.21.15.77:0/2401521008 wait complete. 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 Processor -- start 2024-04-26T05:04:36.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 -- start start 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 0x7effc8113410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc8113a10 con 0x7effc81023f0 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcecae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc810d060 con 0x7effc8102d90 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59650/0 (socket says 172.21.15.77:59650) 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 -- 172.21.15.77:0/1447191673 learned_addr learned my addr 172.21.15.77:0/1447191673 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effc7fff640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 0x7effc8113410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 -- 172.21.15.77:0/1447191673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 msgr2=0x7effc8113410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 0x7effc8113410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 -- 172.21.15.77:0/1447191673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc005040 con 0x7effc81023f0 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.468+0000 7effcca4a640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7effbc004930 tx=0x7effbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.469+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effbc0094c0 con 0x7effc81023f0 2024-04-26T05:04:36.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.469+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effbc032e50 con 0x7effc81023f0 2024-04-26T05:04:36.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.469+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc810d2e0 con 0x7effc81023f0 2024-04-26T05:04:36.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.469+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effbc0433b0 con 0x7effc81023f0 2024-04-26T05:04:36.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.469+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc810d7a0 con 0x7effc81023f0 2024-04-26T05:04:36.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.470+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc8065570 con 0x7effc81023f0 2024-04-26T05:04:36.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.470+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7effbc02f070 con 0x7effc81023f0 2024-04-26T05:04:36.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.470+0000 7effc57fa640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 0x7effb8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.471+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7effbc0afaf0 con 0x7effc81023f0 2024-04-26T05:04:36.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.471+0000 7effc7fff640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 0x7effb8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.471+0000 7effc7fff640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 0x7effb8070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effb4007fd0 tx=0x7effb4009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.475+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7effbc078d70 con 0x7effc81023f0 2024-04-26T05:04:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:36 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/315179147' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-26T05:04:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:36 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1315021882' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-26T05:04:36.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:36 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/271117078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-26T05:04:36.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.687+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7effc8064350 con 0x7effc81023f0 2024-04-26T05:04:36.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.687+0000 7effc57fa640 1 -- 172.21.15.77:0/1447191673 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+5201 (secure 0 0 0) 0x7effbc07c430 con 0x7effc81023f0 2024-04-26T05:04:36.688 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:36.688 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44341,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6827/1653368163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1653368163},{"type":"v1","addr":"172.21.15.78:6827","nonce":1653368163}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:23.420268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:36.689 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-04-26T05:04:36.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.689+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 msgr2=0x7effb8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.689+0000 7effcecae640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 0x7effb8070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effb4007fd0 tx=0x7effb4009e90 comp rx=0 tx=0).stop 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.689+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 msgr2=0x7effc8112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.689+0000 7effcecae640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7effbc004930 tx=0x7effbc009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 shutdown_connections 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc8102d90 0x7effc8113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7effb806e0b0 0x7effb8070560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 --2- 172.21.15.77:0/1447191673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc81023f0 0x7effc8112ed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 >> 172.21.15.77:0/1447191673 conn(0x7effc8071990 msgr2=0x7effc8105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 shutdown_connections 2024-04-26T05:04:36.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.690+0000 7effcecae640 1 -- 172.21.15.77:0/1447191673 wait complete. 2024-04-26T05:04:36.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-26T05:04:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- 172.21.15.77:0/3383523576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99940fb150 msgr2=0x7f99940fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 --2- 172.21.15.77:0/3383523576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99940fb150 0x7f99940fb5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99840048d0 tx=0x7f9984009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- 172.21.15.77:0/3383523576 shutdown_connections 2024-04-26T05:04:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 --2- 172.21.15.77:0/3383523576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99940fb150 0x7f99940fb5c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 --2- 172.21.15.77:0/3383523576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9994101170 0x7f9994101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- 172.21.15.77:0/3383523576 >> 172.21.15.77:0/3383523576 conn(0x7f9994071fd0 msgr2=0x7f99940723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- 172.21.15.77:0/3383523576 shutdown_connections 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- 172.21.15.77:0/3383523576 wait complete. 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 Processor -- start 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.795+0000 7f999c11d640 1 -- start start 2024-04-26T05:04:36.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99940fb150 0x7f9994111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941126a0 con 0x7f9994101170 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999410bcf0 con 0x7f99940fb150 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59672/0 (socket says 172.21.15.77:59672) 2024-04-26T05:04:36.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 -- 172.21.15.77:0/615813173 learned_addr learned my addr 172.21.15.77:0/615813173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 -- 172.21.15.77:0/615813173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99940fb150 msgr2=0x7f9994111b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99940fb150 0x7f9994111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 -- 172.21.15.77:0/615813173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984005040 con 0x7f9994101170 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f99996b8640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9984004930 tx=0x7f9984009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99840094c0 con 0x7f9994101170 2024-04-26T05:04:36.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9984032e50 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99840433b0 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999410bf70 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.796+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999410c430 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.797+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99940fc870 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.798+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f998402f070 con 0x7f9994101170 2024-04-26T05:04:36.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.798+0000 7f998a7fc640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 0x7f9980070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:36.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.798+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f99840af960 con 0x7f9994101170 2024-04-26T05:04:36.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.798+0000 7f9999eb9640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 0x7f9980070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:36.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.799+0000 7f9999eb9640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 0x7f9980070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9990009af0 tx=0x7f9990009cd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:36.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:36.803+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9984078be0 con 0x7f9994101170 2024-04-26T05:04:37.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.017+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f9994063f00 con 0x7f9994101170 2024-04-26T05:04:37.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.017+0000 7f998a7fc640 1 -- 172.21.15.77:0/615813173 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+5198 (secure 0 0 0) 0x7f998407c2a0 con 0x7f9994101170 2024-04-26T05:04:37.018 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:37.018 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi077.urxtgp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6829/2532102778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2532102778},{"type":"v1","addr":"172.21.15.77:6829","nonce":2532102778}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44323,"name":"cephfs.smithi078.wkrgkp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6825/4272329395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272329395},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272329395}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44341,"name":"cephfs.smithi078.nihewy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6827/1653368163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1653368163},{"type":"v1","addr":"172.21.15.78:6827","nonce":1653368163}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:49:55.003675+0000","modified":"2024-04-26T05:01:24.003176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi077.quaiyg","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.77:6827/1508073120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":1508073120},{"type":"v1","addr":"172.21.15.77:6827","nonce":1508073120}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T05:04:37.019 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 msgr2=0x7f9980070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 0x7f9980070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9990009af0 tx=0x7f9990009cd0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 msgr2=0x7f99941120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9984004930 tx=0x7f9984009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 shutdown_connections 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f998006e0b0 0x7f9980070560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9994101170 0x7f99941120a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 --2- 172.21.15.77:0/615813173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99940fb150 0x7f9994111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 >> 172.21.15.77:0/615813173 conn(0x7f9994071fd0 msgr2=0x7f99940fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 shutdown_connections 2024-04-26T05:04:37.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.020+0000 7f999c11d640 1 -- 172.21.15.77:0/615813173 wait complete. 2024-04-26T05:04:37.037 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T05:04:37.050 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T05:04:37.050 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T05:04:37.050 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T05:04:37.067 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T05:04:37.067 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T05:04:37.124 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T05:04:37.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- 172.21.15.77:0/3973301985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 msgr2=0x7f2080101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 --2- 172.21.15.77:0/3973301985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080101460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f20700048d0 tx=0x7f2070009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- 172.21.15.77:0/3973301985 shutdown_connections 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 --2- 172.21.15.77:0/3973301985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 0x7f20800fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 --2- 172.21.15.77:0/3973301985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080101460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- 172.21.15.77:0/3973301985 >> 172.21.15.77:0/3973301985 conn(0x7f20800f6c20 msgr2=0x7f20800f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- 172.21.15.77:0/3973301985 shutdown_connections 2024-04-26T05:04:37.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- 172.21.15.77:0/3973301985 wait complete. 2024-04-26T05:04:37.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 Processor -- start 2024-04-26T05:04:37.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.260+0000 7f208539d640 1 -- start start 2024-04-26T05:04:37.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f208539d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f208539d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 0x7f20801120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f208539d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801126b0 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f208539d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080112820 con 0x7f2080101a30 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59694/0 (socket says 172.21.15.77:59694) 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 -- 172.21.15.77:0/2619952063 learned_addr learned my addr 172.21.15.77:0/2619952063 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207e7fc640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 0x7f20801120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 -- 172.21.15.77:0/2619952063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 msgr2=0x7f20801120b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 0x7f20801120b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 -- 172.21.15.77:0/2619952063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070005040 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f207effd640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2070004930 tx=0x7f2070009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.261+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20700094c0 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.262+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208010be50 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.262+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2070032e50 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.262+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208010c310 con 0x7f2080101090 2024-04-26T05:04:37.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.262+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20700433b0 con 0x7f2080101090 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.263+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20800fc800 con 0x7f2080101090 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.263+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f207002f070 con 0x7f2080101090 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.263+0000 7f20677fe640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 0x7f2068070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.263+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f20700afaa0 con 0x7f2080101090 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.263+0000 7f207e7fc640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 0x7f2068070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.264+0000 7f207e7fc640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 0x7f2068070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f208010d470 tx=0x7f207400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.268+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2070078d70 con 0x7f2080101090 2024-04-26T05:04:37.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.479+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f208010d250 con 0x7f2080101090 2024-04-26T05:04:37.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.480+0000 7f20677fe640 1 -- 172.21.15.77:0/2619952063 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f207007c430 con 0x7f2080101090 2024-04-26T05:04:37.481 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:37.481 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T05:01:13.968714+0000","last_up_change":"2024-04-26T04:59:52.906609+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:49:53.533004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:49:54.229975+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6803","nonce":2493290312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6805","nonce":2493290312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6809","nonce":2493290312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6807","nonce":2493290312}]},"public_addr":"172.21.15.77:6803/2493290312","cluster_addr":"172.21.15.77:6805/2493290312","heartbeat_back_addr":"172.21.15.77:6809/2493290312","heartbeat_front_addr":"172.21.15.77:6807/2493290312","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6811","nonce":3696890435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6813","nonce":3696890435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6817","nonce":3696890435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6815","nonce":3696890435}]},"public_addr":"172.21.15.77:6811/3696890435","cluster_addr":"172.21.15.77:6813/3696890435","heartbeat_back_addr":"172.21.15.77:6817/3696890435","heartbeat_front_addr":"172.21.15.77:6815/3696890435","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6819","nonce":3580780377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6821","nonce":3580780377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6825","nonce":3580780377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6823","nonce":3580780377}]},"public_addr":"172.21.15.77:6819/3580780377","cluster_addr":"172.21.15.77:6821/3580780377","heartbeat_back_addr":"172.21.15.77:6825/3580780377","heartbeat_front_addr":"172.21.15.77:6823/3580780377","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6801","nonce":2853701677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6803","nonce":2853701677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6807","nonce":2853701677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6805","nonce":2853701677}]},"public_addr":"172.21.15.78:6801/2853701677","cluster_addr":"172.21.15.78:6803/2853701677","heartbeat_back_addr":"172.21.15.78:6807/2853701677","heartbeat_front_addr":"172.21.15.78:6805/2853701677","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6809","nonce":2142591175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6811","nonce":2142591175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6815","nonce":2142591175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6813","nonce":2142591175}]},"public_addr":"172.21.15.78:6809/2142591175","cluster_addr":"172.21.15.78:6811/2142591175","heartbeat_back_addr":"172.21.15.78:6815/2142591175","heartbeat_front_addr":"172.21.15.78:6813/2142591175","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6817","nonce":1951816516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6819","nonce":1951816516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6823","nonce":1951816516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6821","nonce":1951816516}]},"public_addr":"172.21.15.78:6817/1951816516","cluster_addr":"172.21.15.78:6819/1951816516","heartbeat_back_addr":"172.21.15.78:6823/1951816516","heartbeat_front_addr":"172.21.15.78:6821/1951816516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6826/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.78:6825/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.78:6824/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.77:6828/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:6827/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.78:0/4271908245":"2024-04-27T04:54:38.567168+0000","172.21.15.77:6801/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/528446640":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3379821906":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.78:0/3124669946":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/3441382659":"2024-04-27T04:52:26.462041+0000","172.21.15.78:6829/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/192705865":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6828/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/1835173819":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/2218529211":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/529966052":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6827/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.77:0/4033800957":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6829/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/2809920581":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1055621332":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6826/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:6801/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1531077053":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/716951029":"2024-04-27T04:55:30.843837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T05:04:37.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 msgr2=0x7f2068070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 0x7f2068070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f208010d470 tx=0x7f207400b040 comp rx=0 tx=0).stop 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 msgr2=0x7f2080111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2070004930 tx=0x7f2070009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 shutdown_connections 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080101a30 0x7f20801120b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f206806e0b0 0x7f2068070560 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 --2- 172.21.15.77:0/2619952063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2080101090 0x7f2080111b70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 >> 172.21.15.77:0/2619952063 conn(0x7f20800f6c20 msgr2=0x7f20800f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 shutdown_connections 2024-04-26T05:04:37.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.482+0000 7f208539d640 1 -- 172.21.15.77:0/2619952063 wait complete. 2024-04-26T05:04:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:37 smithi077 ceph-mon[91833]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:37 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2729341831' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-26T05:04:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:37 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/778516166' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-26T05:04:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:37 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1447191673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-26T05:04:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:37 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:04:37.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:47:22.739195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:49:53.533004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:49:54.229975+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T05:04:37.494 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T05:04:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:37 smithi078 ceph-mon[79689]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:37 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2729341831' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-26T05:04:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:37 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/778516166' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-26T05:04:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:37 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1447191673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-26T05:04:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:37 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:04:37.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 -- 172.21.15.77:0/3216128242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fd1a0 msgr2=0x7fe6e40fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/3216128242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e40fd570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40048d0 tx=0x7fe6d4009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 -- 172.21.15.77:0/3216128242 shutdown_connections 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/3216128242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/3216128242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e40fd570 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 -- 172.21.15.77:0/3216128242 >> 172.21.15.77:0/3216128242 conn(0x7fe6e40f6cb0 msgr2=0x7fe6e40f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 -- 172.21.15.77:0/3216128242 shutdown_connections 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.611+0000 7fe6ea75d640 1 -- 172.21.15.77:0/3216128242 wait complete. 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 Processor -- start 2024-04-26T05:04:37.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 -- start start 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e4112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4113860 con 0x7fe6e40fdb40 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6ea75d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e410ceb0 con 0x7fe6e40fd1a0 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47320/0 (socket says 172.21.15.77:47320) 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.612+0000 7fe6e37fe640 1 -- 172.21.15.77:0/2684654838 learned_addr learned my addr 172.21.15.77:0/2684654838 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e3fff640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e4112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e37fe640 1 -- 172.21.15.77:0/2684654838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fd1a0 msgr2=0x7fe6e4112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e37fe640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e4112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e37fe640 1 -- 172.21.15.77:0/2684654838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4005040 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e37fe640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800eb70 tx=0x7fe6d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d8007c30 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6d800f040 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e410d130 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d80095b0 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.613+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e410d650 con 0x7fe6e40fdb40 2024-04-26T05:04:37.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.614+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e4065860 con 0x7fe6e40fdb40 2024-04-26T05:04:37.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.615+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe6d8009730 con 0x7fe6e40fdb40 2024-04-26T05:04:37.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.615+0000 7fe6e0ff9640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 0x7fe6cc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.615+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe6d808d910 con 0x7fe6e40fdb40 2024-04-26T05:04:37.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.615+0000 7fe6e3fff640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 0x7fe6cc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.616+0000 7fe6e3fff640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 0x7fe6cc070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4003080 tx=0x7fe6d4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.620+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe6d8056c40 con 0x7fe6e40fdb40 2024-04-26T05:04:37.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.803+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6e4109300 con 0x7fe6e40fdb40 2024-04-26T05:04:37.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.803+0000 7fe6e0ff9640 1 -- 172.21.15.77:0/2684654838 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6d805a300 con 0x7fe6e40fdb40 2024-04-26T05:04:37.804 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-04-26T05:04:37.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 msgr2=0x7fe6cc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 0x7fe6cc070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4003080 tx=0x7fe6d4009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 msgr2=0x7fe6e4113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800eb70 tx=0x7fe6d800b040 comp rx=0 tx=0).stop 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 shutdown_connections 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fe6cc06e0b0 0x7fe6cc070560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6e40fdb40 0x7fe6e4113260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 --2- 172.21.15.77:0/2684654838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e40fd1a0 0x7fe6e4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.805+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 >> 172.21.15.77:0/2684654838 conn(0x7fe6e40f6cb0 msgr2=0x7fe6e40f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.806+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 shutdown_connections 2024-04-26T05:04:37.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.806+0000 7fe6ea75d640 1 -- 172.21.15.77:0/2684654838 wait complete. 2024-04-26T05:04:37.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 -- 172.21.15.77:0/4255406273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 msgr2=0x7fd6681025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4255406273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6681025b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd6640048d0 tx=0x7fd664009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 -- 172.21.15.77:0/4255406273 shutdown_connections 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4255406273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 0x7fd6680647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4255406273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6681025b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 -- 172.21.15.77:0/4255406273 >> 172.21.15.77:0/4255406273 conn(0x7fd6680710c0 msgr2=0x7fd6680714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 -- 172.21.15.77:0/4255406273 shutdown_connections 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.916+0000 7fd66fa25640 1 -- 172.21.15.77:0/4255406273 wait complete. 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 Processor -- start 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 -- start start 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 0x7fd6680fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680fd090 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66fa25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680fbc30 con 0x7fd668064380 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66d7c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 0x7fd6680fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47340/0 (socket says 172.21.15.77:47340) 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 -- 172.21.15.77:0/4157717212 learned_addr learned my addr 172.21.15.77:0/4157717212 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 -- 172.21.15.77:0/4157717212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 msgr2=0x7fd6680fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 0x7fd6680fb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.917+0000 7fd66cfc0640 1 -- 172.21.15.77:0/4157717212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664005040 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd66cfc0640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd65800eb70 tx=0x7fd65800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd658007c30 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd65800f040 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6580095b0 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680fbeb0 con 0x7fd6681021e0 2024-04-26T05:04:37.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.918+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd668119370 con 0x7fd6681021e0 2024-04-26T05:04:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.919+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd658009730 con 0x7fd6681021e0 2024-04-26T05:04:37.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.919+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6680fd8e0 con 0x7fd6681021e0 2024-04-26T05:04:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.920+0000 7fd65dffb640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 0x7fd654070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.920+0000 7fd66d7c1640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 0x7fd654070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.920+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd65808d8f0 con 0x7fd6681021e0 2024-04-26T05:04:37.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.920+0000 7fd66d7c1640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 0x7fd654070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd664003080 tx=0x7fd664009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:37.925+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd658056c20 con 0x7fd6681021e0 2024-04-26T05:04:38.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.111+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd66804bcc0 con 0x7fd6681021e0 2024-04-26T05:04:38.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.111+0000 7fd65dffb640 1 -- 172.21.15.77:0/4157717212 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd65805a2e0 con 0x7fd6681021e0 2024-04-26T05:04:38.112 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 msgr2=0x7fd654070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 0x7fd654070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd664003080 tx=0x7fd664009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 msgr2=0x7fd6680fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd65800eb70 tx=0x7fd65800b040 comp rx=0 tx=0).stop 2024-04-26T05:04:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 shutdown_connections 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7fd65406e0b0 0x7fd654070560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.114+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6681021e0 0x7fd6680fb6f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.115+0000 7fd66fa25640 1 --2- 172.21.15.77:0/4157717212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd668064380 0x7fd6680fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.115+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 >> 172.21.15.77:0/4157717212 conn(0x7fd6680710c0 msgr2=0x7fd6680f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.115+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 shutdown_connections 2024-04-26T05:04:38.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.115+0000 7fd66fa25640 1 -- 172.21.15.77:0/4157717212 wait complete. 2024-04-26T05:04:38.131 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:38 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/615813173' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-26T05:04:38.131 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:38 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2619952063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:38.131 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:38.131 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:38 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:38.131 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:38 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2684654838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:38.132 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T05:04:38.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.230+0000 7f6bc819d640 1 -- 172.21.15.77:0/4282851596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 msgr2=0x7f6bc0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 --2- 172.21.15.77:0/4282851596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc0101460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0048d0 tx=0x7f6bbc009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- 172.21.15.77:0/4282851596 shutdown_connections 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 --2- 172.21.15.77:0/4282851596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc00fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 --2- 172.21.15.77:0/4282851596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc0101460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- 172.21.15.77:0/4282851596 >> 172.21.15.77:0/4282851596 conn(0x7f6bc00f6c80 msgr2=0x7f6bc00f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- 172.21.15.77:0/4282851596 shutdown_connections 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- 172.21.15.77:0/4282851596 wait complete. 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 Processor -- start 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- start start 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc010e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00664f0 con 0x7f6bc0101090 2024-04-26T05:04:38.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc819d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc010ec30 con 0x7f6bc0101a30 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc5f39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc5f39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47350/0 (socket says 172.21.15.77:47350) 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.231+0000 7f6bc5f39640 1 -- 172.21.15.77:0/119639100 learned_addr learned my addr 172.21.15.77:0/119639100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5738640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc010e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5f39640 1 -- 172.21.15.77:0/119639100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 msgr2=0x7f6bc010e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5f39640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc010e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5f39640 1 -- 172.21.15.77:0/119639100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc005040 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5738640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc5f39640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004930 tx=0x7f6bbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bbc0094c0 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bbc032e50 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc010eeb0 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bbc0433b0 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.232+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc01173e0 con 0x7f6bc0101090 2024-04-26T05:04:38.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.233+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00fc800 con 0x7f6bc0101090 2024-04-26T05:04:38.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.233+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6bbc02f070 con 0x7f6bc0101090 2024-04-26T05:04:38.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.234+0000 7f6bb67fc640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 0x7f6bac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.234+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6bbc0afaf0 con 0x7f6bc0101090 2024-04-26T05:04:38.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.234+0000 7f6bc5738640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 0x7f6bac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.234+0000 7f6bc5738640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 0x7f6bac070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0007fd0 tx=0x7f6bb0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.238+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6bbc078d70 con 0x7f6bc0101090 2024-04-26T05:04:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.414+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6bc004bcc0 con 0x7f6bc0101090 2024-04-26T05:04:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.414+0000 7f6bb67fc640 1 -- 172.21.15.77:0/119639100 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6bbc07c430 con 0x7f6bc0101090 2024-04-26T05:04:38.415 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:38.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.416+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 msgr2=0x7f6bac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.416+0000 7f6bc819d640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 0x7f6bac070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0007fd0 tx=0x7f6bb0009e90 comp rx=0 tx=0).stop 2024-04-26T05:04:38.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.416+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 msgr2=0x7f6bc010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.416+0000 7f6bc819d640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004930 tx=0x7f6bbc009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 shutdown_connections 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc0101a30 0x7f6bc010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f6bac06e0b0 0x7f6bac070560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 --2- 172.21.15.77:0/119639100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bc0101090 0x7f6bc010e180 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 >> 172.21.15.77:0/119639100 conn(0x7f6bc00f6c80 msgr2=0x7f6bc00f7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 shutdown_connections 2024-04-26T05:04:38.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.417+0000 7f6bc819d640 1 -- 172.21.15.77:0/119639100 wait complete. 2024-04-26T05:04:38.428 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T05:04:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:38 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/615813173' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-26T05:04:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:38 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2619952063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:38 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:38.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:38 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2684654838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:38.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 -- 172.21.15.77:0/836636524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c102220 msgr2=0x7f8a3c1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 --2- 172.21.15.77:0/836636524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c102220 0x7f8a3c1025f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009c50 tx=0x7f8a38004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 -- 172.21.15.77:0/836636524 shutdown_connections 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 --2- 172.21.15.77:0/836636524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c064390 0x7f8a3c0647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 --2- 172.21.15.77:0/836636524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c102220 0x7f8a3c1025f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.523+0000 7f8a4399d640 1 -- 172.21.15.77:0/836636524 >> 172.21.15.77:0/836636524 conn(0x7f8a3c0710c0 msgr2=0x7f8a3c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 -- 172.21.15.77:0/836636524 shutdown_connections 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 -- 172.21.15.77:0/836636524 wait complete. 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 Processor -- start 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 -- start start 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 0x7f8a3c113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1138a0 con 0x7f8a3c064390 2024-04-26T05:04:38.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a4399d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c10ce60 con 0x7f8a3c102220 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47354/0 (socket says 172.21.15.77:47354) 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 -- 172.21.15.77:0/1596698586 learned_addr learned my addr 172.21.15.77:0/1596698586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a40f38640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 0x7f8a3c113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 -- 172.21.15.77:0/1596698586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 msgr2=0x7f8a3c113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 0x7f8a3c113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.524+0000 7f8a41739640 1 -- 172.21.15.77:0/1596698586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a380098b0 con 0x7f8a3c064390 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a40f38640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 0x7f8a3c113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a41739640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8a38004a30 tx=0x7f8a38004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3803a070 con 0x7f8a3c064390 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a38035b20 con 0x7f8a3c064390 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c10d0e0 con 0x7f8a3c064390 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3803eaa0 con 0x7f8a3c064390 2024-04-26T05:04:38.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.525+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c10d600 con 0x7f8a3c064390 2024-04-26T05:04:38.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.526+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c0fd920 con 0x7f8a3c064390 2024-04-26T05:04:38.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.526+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8a38032070 con 0x7f8a3c064390 2024-04-26T05:04:38.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.527+0000 7f8a31ffb640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 0x7f8a28070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.527+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f8a380b0480 con 0x7f8a3c064390 2024-04-26T05:04:38.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.527+0000 7f8a40f38640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 0x7f8a28070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.527+0000 7f8a40f38640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 0x7f8a28070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c006fd0 tx=0x7f8a2c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.532+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8a38079700 con 0x7f8a3c064390 2024-04-26T05:04:38.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.722+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a3c04bcc0 con 0x7f8a3c064390 2024-04-26T05:04:38.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.722+0000 7f8a31ffb640 1 -- 172.21.15.77:0/1596698586 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2544 (secure 0 0 0) 0x7f8a3c04bcc0 con 0x7f8a3c064390 2024-04-26T05:04:38.723 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6826/4041373129 2024-04-27T05:01:13.968688+0000 2024-04-26T05:04:38.723 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6825/3670372668 2024-04-27T05:00:50.181711+0000 2024-04-26T05:04:38.723 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6824/3670372668 2024-04-27T05:00:50.181711+0000 2024-04-26T05:04:38.723 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6828/2377141710 2024-04-27T05:00:25.565165+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6827/132419939 2024-04-27T04:59:59.294793+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3470497411 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/171166588 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/4271908245 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1184941417 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3186647962 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/4070229355 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3157890049 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1289451679 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3157890049 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1305748236 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1184941417 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/566106414 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:38.724 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/528446640 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/171166588 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3379821906 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1671997778 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/3124669946 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3441382659 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6829/1876826264 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/192705865 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/2855259383 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6828/1876826264 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/1835173819 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/2218529211 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/529966052 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2042851017 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/558029148 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:38.725 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/2855259383 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6827/4041373129 2024-04-27T05:01:13.968688+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/4033800957 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6829/2377141710 2024-04-27T05:00:25.565165+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1031953382 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2809920581 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1055621332 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6826/132419939 2024-04-27T04:59:59.294793+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3186647962 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1531077053 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.726 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/716951029 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:listed 43 entries 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 msgr2=0x7f8a28070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 0x7f8a28070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c006fd0 tx=0x7f8a2c008040 comp rx=0 tx=0).stop 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 msgr2=0x7f8a3c112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8a38004a30 tx=0x7f8a38004d10 comp rx=0 tx=0).stop 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 shutdown_connections 2024-04-26T05:04:38.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c102220 0x7f8a3c113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f8a2806e0b0 0x7f8a28070560 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.724+0000 7f8a4399d640 1 --2- 172.21.15.77:0/1596698586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a3c064390 0x7f8a3c112cd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.725+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 >> 172.21.15.77:0/1596698586 conn(0x7f8a3c0710c0 msgr2=0x7f8a3c0f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.725+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 shutdown_connections 2024-04-26T05:04:38.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.725+0000 7f8a4399d640 1 -- 172.21.15.77:0/1596698586 wait complete. 2024-04-26T05:04:38.736 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-04-26T05:04:38.736 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T05:04:38.753 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 -- 172.21.15.77:0/2212284711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 msgr2=0x7f394c0fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 --2- 172.21.15.77:0/2212284711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c0fb160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39340048d0 tx=0x7f3934009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 -- 172.21.15.77:0/2212284711 shutdown_connections 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 --2- 172.21.15.77:0/2212284711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c0fb160 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 --2- 172.21.15.77:0/2212284711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 -- 172.21.15.77:0/2212284711 >> 172.21.15.77:0/2212284711 conn(0x7f394c0f6cc0 msgr2=0x7f394c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 -- 172.21.15.77:0/2212284711 shutdown_connections 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.885+0000 7f395135d640 1 -- 172.21.15.77:0/2212284711 wait complete. 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 Processor -- start 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 -- start start 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c1151c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c115d00 con 0x7f394c101ad0 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f395135d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c10f350 con 0x7f394c101130 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47370/0 (socket says 172.21.15.77:47370) 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 -- 172.21.15.77:0/614712369 learned_addr learned my addr 172.21.15.77:0/614712369 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f394affd640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c1151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 -- 172.21.15.77:0/614712369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 msgr2=0x7f394c1151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c1151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 -- 172.21.15.77:0/614712369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934005040 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f394affd640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c1151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.886+0000 7f3943fff640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3934004930 tx=0x7f3934009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.887+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39340094c0 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.887+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c10f5d0 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.887+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3934032e50 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.887+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39340433b0 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.887+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c10fa90 con 0x7f394c101ad0 2024-04-26T05:04:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.888+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c0fc8a0 con 0x7f394c101ad0 2024-04-26T05:04:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.888+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f393402f070 con 0x7f394c101ad0 2024-04-26T05:04:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.888+0000 7f39437fe640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 0x7f3938070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.889+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f39340afaf0 con 0x7f394c101ad0 2024-04-26T05:04:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.889+0000 7f394affd640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 0x7f3938070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.889+0000 7f394affd640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 0x7f3938070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f393c004d80 tx=0x7f393c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:38.893+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3934078d70 con 0x7f394c101ad0 2024-04-26T05:04:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.071+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394c108290 con 0x7f394c101ad0 2024-04-26T05:04:39.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.072+0000 7f39437fe640 1 -- 172.21.15.77:0/614712369 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f393407c430 con 0x7f394c101ad0 2024-04-26T05:04:39.073 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-04-26T05:04:39.073 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"3c1ee652-0387-11ef-bc93-c7b262605968","created":"2024-04-26T04:43:00.281790+0000","modified":"2024-04-26T05:01:13.968714+0000","last_up_change":"2024-04-26T04:59:52.906609+0000","last_in_change":"2024-04-26T04:48:15.468516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:47:22.739195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:49:53.533004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:49:54.229975+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"074d417d-f631-4ee8-b43f-670fc24d4908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6803","nonce":2493290312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6805","nonce":2493290312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6809","nonce":2493290312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2493290312},{"type":"v1","addr":"172.21.15.77:6807","nonce":2493290312}]},"public_addr":"172.21.15.77:6803/2493290312","cluster_addr":"172.21.15.77:6805/2493290312","heartbeat_back_addr":"172.21.15.77:6809/2493290312","heartbeat_front_addr":"172.21.15.77:6807/2493290312","state":["exists","up"]},{"osd":1,"uuid":"beee2092-5b6a-45a9-93ae-0f368a2d613f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6811","nonce":3696890435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6813","nonce":3696890435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6817","nonce":3696890435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3696890435},{"type":"v1","addr":"172.21.15.77:6815","nonce":3696890435}]},"public_addr":"172.21.15.77:6811/3696890435","cluster_addr":"172.21.15.77:6813/3696890435","heartbeat_back_addr":"172.21.15.77:6817/3696890435","heartbeat_front_addr":"172.21.15.77:6815/3696890435","state":["exists","up"]},{"osd":2,"uuid":"61c0054c-1766-462b-9bc6-fe56deac78e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6819","nonce":3580780377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6821","nonce":3580780377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6825","nonce":3580780377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3580780377},{"type":"v1","addr":"172.21.15.77:6823","nonce":3580780377}]},"public_addr":"172.21.15.77:6819/3580780377","cluster_addr":"172.21.15.77:6821/3580780377","heartbeat_back_addr":"172.21.15.77:6825/3580780377","heartbeat_front_addr":"172.21.15.77:6823/3580780377","state":["exists","up"]},{"osd":3,"uuid":"be335c4c-983a-4c4e-84e8-d53e881c90b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6801","nonce":2853701677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6803","nonce":2853701677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6807","nonce":2853701677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2853701677},{"type":"v1","addr":"172.21.15.78:6805","nonce":2853701677}]},"public_addr":"172.21.15.78:6801/2853701677","cluster_addr":"172.21.15.78:6803/2853701677","heartbeat_back_addr":"172.21.15.78:6807/2853701677","heartbeat_front_addr":"172.21.15.78:6805/2853701677","state":["exists","up"]},{"osd":4,"uuid":"5de5516b-b1f0-405b-984d-1d9ce0900adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6809","nonce":2142591175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6811","nonce":2142591175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6815","nonce":2142591175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2142591175},{"type":"v1","addr":"172.21.15.78:6813","nonce":2142591175}]},"public_addr":"172.21.15.78:6809/2142591175","cluster_addr":"172.21.15.78:6811/2142591175","heartbeat_back_addr":"172.21.15.78:6815/2142591175","heartbeat_front_addr":"172.21.15.78:6813/2142591175","state":["exists","up"]},{"osd":5,"uuid":"bf968b1d-2324-4b14-9723-24ebb6b6c406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6817","nonce":1951816516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6819","nonce":1951816516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6823","nonce":1951816516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":1951816516},{"type":"v1","addr":"172.21.15.78:6821","nonce":1951816516}]},"public_addr":"172.21.15.78:6817/1951816516","cluster_addr":"172.21.15.78:6819/1951816516","heartbeat_back_addr":"172.21.15.78:6823/1951816516","heartbeat_front_addr":"172.21.15.78:6821/1951816516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:22.358456+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:46:51.215254+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:19.012777+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:47:42.810368+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:07.418230+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:48:32.538337+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6826/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.78:6825/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.78:6824/3670372668":"2024-04-27T05:00:50.181711+0000","172.21.15.77:6828/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:6827/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:0/3470497411":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.78:0/4271908245":"2024-04-27T04:54:38.567168+0000","172.21.15.77:6801/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/4070229355":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6801/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1289451679":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/3157890049":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/1305748236":"2024-04-27T04:45:08.425280+0000","172.21.15.77:6800/1184941417":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/566106414":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/528446640":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6800/171166588":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/3379821906":"2024-04-27T04:52:26.462041+0000","172.21.15.77:0/1671997778":"2024-04-27T04:44:03.833148+0000","172.21.15.78:0/3124669946":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/3441382659":"2024-04-27T04:52:26.462041+0000","172.21.15.78:6829/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/192705865":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6800/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6828/1876826264":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/1835173819":"2024-04-27T04:54:38.567168+0000","172.21.15.78:0/2218529211":"2024-04-27T04:54:38.567168+0000","172.21.15.77:0/529966052":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/2042851017":"2024-04-27T04:43:32.800118+0000","172.21.15.77:0/558029148":"2024-04-27T04:43:32.800118+0000","172.21.15.77:6801/2855259383":"2024-04-27T04:45:08.425280+0000","172.21.15.78:6827/4041373129":"2024-04-27T05:01:13.968688+0000","172.21.15.77:0/4033800957":"2024-04-27T04:52:26.462041+0000","172.21.15.77:6829/2377141710":"2024-04-27T05:00:25.565165+0000","172.21.15.77:0/1031953382":"2024-04-27T04:44:03.833148+0000","172.21.15.77:0/2809920581":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1055621332":"2024-04-27T04:55:30.843837+0000","172.21.15.77:6826/132419939":"2024-04-27T04:59:59.294793+0000","172.21.15.77:6801/3186647962":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/1531077053":"2024-04-27T04:55:30.843837+0000","172.21.15.77:0/716951029":"2024-04-27T04:55:30.843837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 msgr2=0x7f3938070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 0x7f3938070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f393c004d80 tx=0x7f393c00b040 comp rx=0 tx=0).stop 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 msgr2=0x7f394c115700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3934004930 tx=0x7f3934009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 shutdown_connections 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f393806e0b0 0x7f3938070560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f394c101ad0 0x7f394c115700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 --2- 172.21.15.77:0/614712369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c101130 0x7f394c1151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 >> 172.21.15.77:0/614712369 conn(0x7f394c0f6cc0 msgr2=0x7f394c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 shutdown_connections 2024-04-26T05:04:39.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.074+0000 7f395135d640 1 -- 172.21.15.77:0/614712369 wait complete. 2024-04-26T05:04:39.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:47:22.739195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:49:53.533004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:49:54.229975+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T05:04:39.087 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T05:04:39.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 -- 172.21.15.77:0/1997376777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 msgr2=0x7f7e641015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1997376777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e641015b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e600048d0 tx=0x7f7e60009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 -- 172.21.15.77:0/1997376777 shutdown_connections 2024-04-26T05:04:39.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1997376777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 0x7f7e640fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1997376777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e641015b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.182+0000 7f7e6b754640 1 -- 172.21.15.77:0/1997376777 >> 172.21.15.77:0/1997376777 conn(0x7f7e64071700 msgr2=0x7f7e64071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 -- 172.21.15.77:0/1997376777 shutdown_connections 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 -- 172.21.15.77:0/1997376777 wait complete. 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 Processor -- start 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 -- start start 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 0x7f7e64111c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64112750 con 0x7f7e641011e0 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e6b754640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6410bda0 con 0x7f7e640fb950 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e68cef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e68cef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47382/0 (socket says 172.21.15.77:47382) 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.183+0000 7f7e68cef640 1 -- 172.21.15.77:0/1709144016 learned_addr learned my addr 172.21.15.77:0/1709144016 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e68cef640 1 -- 172.21.15.77:0/1709144016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 msgr2=0x7f7e64111c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e694f0640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 0x7f7e64111c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e68cef640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 0x7f7e64111c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e68cef640 1 -- 172.21.15.77:0/1709144016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60005040 con 0x7f7e641011e0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e68cef640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e54002a80 tx=0x7f7e54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e54007c30 con 0x7f7e641011e0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e5400f040 con 0x7f7e641011e0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6410c080 con 0x7f7e641011e0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e540096b0 con 0x7f7e641011e0 2024-04-26T05:04:39.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.184+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6410c5a0 con 0x7f7e641011e0 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.185+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e640fc960 con 0x7f7e641011e0 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.185+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7e5400d6b0 con 0x7f7e641011e0 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.186+0000 7f7e59ffb640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 0x7f7e50070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.186+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7e5408d620 con 0x7f7e641011e0 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.186+0000 7f7e694f0640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 0x7f7e50070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.186+0000 7f7e694f0640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 0x7f7e50070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e60003080 tx=0x7f7e60009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.190+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7e540568a0 con 0x7f7e641011e0 2024-04-26T05:04:39.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.366+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e641082d0 con 0x7f7e641011e0 2024-04-26T05:04:39.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.367+0000 7f7e59ffb640 1 -- 172.21.15.77:0/1709144016 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e54056030 con 0x7f7e641011e0 2024-04-26T05:04:39.368 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 msgr2=0x7f7e50070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 0x7f7e50070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e60003080 tx=0x7f7e60009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 msgr2=0x7f7e64112150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e54002a80 tx=0x7f7e54004600 comp rx=0 tx=0).stop 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 shutdown_connections 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f7e5006e0b0 0x7f7e50070560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e641011e0 0x7f7e64112150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 --2- 172.21.15.77:0/1709144016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e640fb950 0x7f7e64111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 >> 172.21.15.77:0/1709144016 conn(0x7f7e64071700 msgr2=0x7f7e640f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 shutdown_connections 2024-04-26T05:04:39.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.369+0000 7f7e6b754640 1 -- 172.21.15.77:0/1709144016 wait complete. 2024-04-26T05:04:39.379 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/4157717212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/119639100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:39.380 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:39 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1596698586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T05:04:39.381 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 -- 172.21.15.77:0/2605610455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 msgr2=0x7f6398101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 --2- 172.21.15.77:0/2605610455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f6398101470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63880048d0 tx=0x7f6388009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 -- 172.21.15.77:0/2605610455 shutdown_connections 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 --2- 172.21.15.77:0/2605610455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 0x7f63980fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 --2- 172.21.15.77:0/2605610455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f6398101470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 -- 172.21.15.77:0/2605610455 >> 172.21.15.77:0/2605610455 conn(0x7f63980f6cb0 msgr2=0x7f63980f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 -- 172.21.15.77:0/2605610455 shutdown_connections 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.478+0000 7f639e0ae640 1 -- 172.21.15.77:0/2605610455 wait complete. 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 Processor -- start 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 -- start start 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 0x7f639810e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639810e9d0 con 0x7f63981010a0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f639e0ae640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639810eb40 con 0x7f6398101a40 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47410/0 (socket says 172.21.15.77:47410) 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 -- 172.21.15.77:0/331267252 learned_addr learned my addr 172.21.15.77:0/331267252 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f6396ffd640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 0x7f639810e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 -- 172.21.15.77:0/331267252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 msgr2=0x7f639810e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 0x7f639810e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 -- 172.21.15.77:0/331267252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f63981010a0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.479+0000 7f63977fe640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6388004930 tx=0x7f6388009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.480+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63880094c0 con 0x7f63981010a0 2024-04-26T05:04:39.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.480+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639810edc0 con 0x7f63981010a0 2024-04-26T05:04:39.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.480+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6388032e50 con 0x7f63981010a0 2024-04-26T05:04:39.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.480+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63880433b0 con 0x7f63981010a0 2024-04-26T05:04:39.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.480+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398117210 con 0x7f63981010a0 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.481+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63980fc810 con 0x7f63981010a0 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.481+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f638802f070 con 0x7f63981010a0 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.481+0000 7f637ffff640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 0x7f6380070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.481+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f63880afa20 con 0x7f63981010a0 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.481+0000 7f6396ffd640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 0x7f6380070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.482+0000 7f6396ffd640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 0x7f6380070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.486+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6388078ca0 con 0x7f63981010a0 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/4157717212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/119639100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:39 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1596698586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T05:04:39.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.661+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f639810edc0 con 0x7f63981010a0 2024-04-26T05:04:39.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.661+0000 7f637ffff640 1 -- 172.21.15.77:0/331267252 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f639810edc0 con 0x7f63981010a0 2024-04-26T05:04:39.662 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:39.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 msgr2=0x7f6380070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 0x7f6380070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c008040 comp rx=0 tx=0).stop 2024-04-26T05:04:39.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 msgr2=0x7f639810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6388004930 tx=0x7f6388009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 shutdown_connections 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6398101a40 0x7f639810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f638006e0b0 0x7f6380070560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 --2- 172.21.15.77:0/331267252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63981010a0 0x7f639810df50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.663+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 >> 172.21.15.77:0/331267252 conn(0x7f63980f6cb0 msgr2=0x7f63980f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.664+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 shutdown_connections 2024-04-26T05:04:39.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.664+0000 7f639e0ae640 1 -- 172.21.15.77:0/331267252 wait complete. 2024-04-26T05:04:39.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 -- 172.21.15.77:0/2888774311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102190 msgr2=0x7f6810102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2888774311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102190 0x7f6810102560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f68000048d0 tx=0x7f6800009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 -- 172.21.15.77:0/2888774311 shutdown_connections 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2888774311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102b30 0x7f68100fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2888774311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102190 0x7f6810102560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 -- 172.21.15.77:0/2888774311 >> 172.21.15.77:0/2888774311 conn(0x7f68100f7da0 msgr2=0x7f68100fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 -- 172.21.15.77:0/2888774311 shutdown_connections 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.770+0000 7f6816d9d640 1 -- 172.21.15.77:0/2888774311 wait complete. 2024-04-26T05:04:39.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 Processor -- start 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 -- start start 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 0x7f6810112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810113910 con 0x7f6810102b30 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f6816d9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681010ced0 con 0x7f6810102190 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f680ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f680ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47422/0 (socket says 172.21.15.77:47422) 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.771+0000 7f680ffff640 1 -- 172.21.15.77:0/2249797539 learned_addr learned my addr 172.21.15.77:0/2249797539 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f6814b39640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 0x7f6810112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680ffff640 1 -- 172.21.15.77:0/2249797539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 msgr2=0x7f6810112d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680ffff640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 0x7f6810112d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680ffff640 1 -- 172.21.15.77:0/2249797539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800005040 con 0x7f6810102b30 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f6814b39640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 0x7f6810112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680ffff640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f680400b790 tx=0x7f68040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680400c700 con 0x7f6810102b30 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681010d1b0 con 0x7f6810102b30 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f680400cd30 con 0x7f6810102b30 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6804013370 con 0x7f6810102b30 2024-04-26T05:04:39.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.772+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681010d6d0 con 0x7f6810102b30 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.773+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68100fd900 con 0x7f6810102b30 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.773+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6804012070 con 0x7f6810102b30 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.774+0000 7f680d7fa640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 0x7f67f8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.774+0000 7f6814b39640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 0x7f67f8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.774+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f680408d870 con 0x7f6810102b30 2024-04-26T05:04:39.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.774+0000 7f6814b39640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 0x7f67f8070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6800000c00 tx=0x7f6800009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:39.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.778+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6804056af0 con 0x7f6810102b30 2024-04-26T05:04:39.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.953+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f68101093b0 con 0x7f6810102b30 2024-04-26T05:04:39.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.954+0000 7f680d7fa640 1 -- 172.21.15.77:0/2249797539 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f680405a1b0 con 0x7f6810102b30 2024-04-26T05:04:39.955 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-04-26T05:04:39.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 msgr2=0x7f67f8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 0x7f67f8070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6800000c00 tx=0x7f6800009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 msgr2=0x7f6810113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f680400b790 tx=0x7f68040091d0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 shutdown_connections 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7f67f806e0b0 0x7f67f8070560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6810102b30 0x7f6810113280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 --2- 172.21.15.77:0/2249797539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6810102190 0x7f6810112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 >> 172.21.15.77:0/2249797539 conn(0x7f68100f7da0 msgr2=0x7f68100f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 shutdown_connections 2024-04-26T05:04:39.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:39.956+0000 7f6816d9d640 1 -- 172.21.15.77:0/2249797539 wait complete. 2024-04-26T05:04:39.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 -- 172.21.15.77:0/302253140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 msgr2=0x7ff8cc0fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 --2- 172.21.15.77:0/302253140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc0fc1e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00048d0 tx=0x7ff8c0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 -- 172.21.15.77:0/302253140 shutdown_connections 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 --2- 172.21.15.77:0/302253140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc0fc1e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 --2- 172.21.15.77:0/302253140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 -- 172.21.15.77:0/302253140 >> 172.21.15.77:0/302253140 conn(0x7ff8cc0f7dc0 msgr2=0x7ff8cc0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 -- 172.21.15.77:0/302253140 shutdown_connections 2024-04-26T05:04:40.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.063+0000 7ff8d3261640 1 -- 172.21.15.77:0/302253140 wait complete. 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 Processor -- start 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 -- start start 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc113800 con 0x7ff8cc102b50 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d3261640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc10ce50 con 0x7ff8cc1021b0 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47426/0 (socket says 172.21.15.77:47426) 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8cbfff640 1 -- 172.21.15.77:0/1292889290 learned_addr learned my addr 172.21.15.77:0/1292889290 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.064+0000 7ff8d0ffd640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8cbfff640 1 -- 172.21.15.77:0/1292889290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 msgr2=0x7ff8cc112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:40.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8cbfff640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8cbfff640 1 -- 172.21.15.77:0/1292889290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0005040 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8d0ffd640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8cbfff640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0004930 tx=0x7ff8c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c00094c0 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc10d0d0 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8c0032e50 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c00433b0 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.065+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc10d590 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.066+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc0fd920 con 0x7ff8cc102b50 2024-04-26T05:04:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.066+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 40) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8c002f070 con 0x7ff8cc102b50 2024-04-26T05:04:40.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.067+0000 7ff8c97fa640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 0x7ff8b4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T05:04:40.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.067+0000 7ff8d0ffd640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 0x7ff8b4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T05:04:40.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.067+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff8c00afaf0 con 0x7ff8cc102b50 2024-04-26T05:04:40.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.067+0000 7ff8d0ffd640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 0x7ff8b4070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T05:04:40.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.072+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8c0078d70 con 0x7ff8cc102b50 2024-04-26T05:04:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:40 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/614712369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:40 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1709144016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:40 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/331267252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:40 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/2249797539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.263+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8cc109310 con 0x7ff8cc102b50 2024-04-26T05:04:40.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.264+0000 7ff8c97fa640 1 -- 172.21.15.77:0/1292889290 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2544 (secure 0 0 0) 0x7ff8c007c430 con 0x7ff8cc102b50 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6826/4041373129 2024-04-27T05:01:13.968688+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6825/3670372668 2024-04-27T05:00:50.181711+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6824/3670372668 2024-04-27T05:00:50.181711+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6828/2377141710 2024-04-27T05:00:25.565165+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6827/132419939 2024-04-27T04:59:59.294793+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3470497411 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/171166588 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/4271908245 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1184941417 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3186647962 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/4070229355 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:40.265 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3157890049 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1289451679 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3157890049 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1305748236 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1184941417 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/566106414 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/528446640 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/171166588 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3379821906 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1671997778 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/3124669946 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3441382659 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6829/1876826264 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.266 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/192705865 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/2855259383 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6828/1876826264 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/1835173819 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:0/2218529211 2024-04-27T04:54:38.567168+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/529966052 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2042851017 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/558029148 2024-04-27T04:43:32.800118+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/2855259383 2024-04-27T04:45:08.425280+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.78:6827/4041373129 2024-04-27T05:01:13.968688+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/4033800957 2024-04-27T04:52:26.462041+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6829/2377141710 2024-04-27T05:00:25.565165+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1031953382 2024-04-27T04:44:03.833148+0000 2024-04-26T05:04:40.267 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2809920581 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.268 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1055621332 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.268 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6826/132419939 2024-04-27T04:59:59.294793+0000 2024-04-26T05:04:40.268 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3186647962 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.268 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1531077053 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.269 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/716951029 2024-04-27T04:55:30.843837+0000 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:listed 43 entries 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 msgr2=0x7ff8b4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 0x7ff8b4070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc009e90 comp rx=0 tx=0).stop 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 msgr2=0x7ff8cc113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0004930 tx=0x7ff8c0009da0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 shutdown_connections 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:6800/1604716213,v1:172.21.15.77:6801/1604716213] conn(0x7ff8b406e0b0 0x7ff8b4070560 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8cc102b50 0x7ff8cc113200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 --2- 172.21.15.77:0/1292889290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8cc1021b0 0x7ff8cc112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T05:04:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 >> 172.21.15.77:0/1292889290 conn(0x7ff8cc0f7dc0 msgr2=0x7ff8cc0f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T05:04:40.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 shutdown_connections 2024-04-26T05:04:40.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-04-26T05:04:40.266+0000 7ff8d3261640 1 -- 172.21.15.77:0/1292889290 wait complete. 2024-04-26T05:04:40.277 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi077.front.sepia.ceph.com... 2024-04-26T05:04:40.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T05:04:40.278 DEBUG:teuthology.orchestra.run.smithi077:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T05:04:40.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T05:04:40.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:40 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/614712369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T05:04:40.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:40 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1709144016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T05:04:40.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:40 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/331267252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T05:04:40.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:40 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/2249797539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T05:04:41.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:41 smithi077 ceph-mon[91833]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:41.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:41 smithi077 ceph-mon[91833]: from='client.? 172.21.15.77:0/1292889290' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T05:04:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:41 smithi078 ceph-mon[79689]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:41 smithi078 ceph-mon[79689]: from='client.? 172.21.15.77:0/1292889290' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T05:04:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:43 smithi077 ceph-mon[91833]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:43 smithi078 ceph-mon[79689]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:45.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:45 smithi077 ceph-mon[91833]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:45.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:45 smithi078 ceph-mon[79689]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:46.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:46.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:04:47.418 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:47 smithi078 ceph-mon[79689]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:47.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:47 smithi077 ceph-mon[91833]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:49.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:49 smithi077 ceph-mon[91833]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:49.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:49 smithi078 ceph-mon[79689]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:51.309 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:51 smithi077 ceph-mon[91833]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:51.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:51 smithi078 ceph-mon[79689]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:53.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:53 smithi077 ceph-mon[91833]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:53.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:53 smithi078 ceph-mon[79689]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:55.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:55 smithi077 ceph-mon[91833]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:55.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:55 smithi078 ceph-mon[79689]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:57.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:57 smithi077 ceph-mon[91833]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:57.519 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:57 smithi078 ceph-mon[79689]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:59.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:04:59 smithi077 ceph-mon[91833]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:04:59.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:04:59 smithi078 ceph-mon[79689]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:01.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:01 smithi077 ceph-mon[91833]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:01.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:01 smithi078 ceph-mon[79689]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:01.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:03.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:03 smithi077 ceph-mon[91833]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:03.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:03 smithi078 ceph-mon[79689]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:05.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:05 smithi077 ceph-mon[91833]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:05.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:05 smithi078 ceph-mon[79689]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:07.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:07 smithi077 ceph-mon[91833]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:07.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:07 smithi078 ceph-mon[79689]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:09.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:09 smithi077 ceph-mon[91833]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:09.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:09 smithi078 ceph-mon[79689]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:11.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:11 smithi077 ceph-mon[91833]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:11.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:11 smithi078 ceph-mon[79689]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:13.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:13 smithi077 ceph-mon[91833]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:13.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:13 smithi078 ceph-mon[79689]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:15 smithi077 ceph-mon[91833]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:15.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:15 smithi078 ceph-mon[79689]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:16.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:17.418 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:17 smithi078 ceph-mon[79689]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:17 smithi077 ceph-mon[91833]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:19.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:19 smithi077 ceph-mon[91833]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:19.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:19 smithi078 ceph-mon[79689]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:21.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:21 smithi077 ceph-mon[91833]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:21.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:21 smithi078 ceph-mon[79689]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:23.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:23 smithi077 ceph-mon[91833]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:23.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:23 smithi078 ceph-mon[79689]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:25.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:25 smithi077 ceph-mon[91833]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:25.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:25 smithi078 ceph-mon[79689]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:27.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:27 smithi077 ceph-mon[91833]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:27 smithi078 ceph-mon[79689]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:29.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:29 smithi077 ceph-mon[91833]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:29.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:29 smithi078 ceph-mon[79689]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:31.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:31 smithi077 ceph-mon[91833]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:31.494 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:31 smithi078 ceph-mon[79689]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:33.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:33 smithi077 ceph-mon[91833]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:33 smithi078 ceph-mon[79689]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:35.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:35 smithi077 ceph-mon[91833]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:35 smithi078 ceph-mon[79689]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:37.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:37 smithi077 ceph-mon[91833]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:37 smithi078 ceph-mon[79689]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:39.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:39 smithi077 ceph-mon[91833]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:39.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:05:39.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:39.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:39 smithi078 ceph-mon[79689]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:39.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:05:39.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:39.518 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:40.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:05:40.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:05:40.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:40.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:05:40.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:05:40.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:05:41.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:41 smithi077 ceph-mon[91833]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:41 smithi078 ceph-mon[79689]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:43.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:43 smithi077 ceph-mon[91833]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:43.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:43 smithi078 ceph-mon[79689]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:45.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:45 smithi077 ceph-mon[91833]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:45.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:45 smithi078 ceph-mon[79689]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:46.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:46 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:46.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:46 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:05:47.418 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:47 smithi078 ceph-mon[79689]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:47.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:47 smithi077 ceph-mon[91833]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:48.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:48 smithi077 ceph-mon[91833]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:48.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:48 smithi078 ceph-mon[79689]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:51.196 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:50 smithi077 ceph-mon[91833]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:50 smithi078 ceph-mon[79689]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:52 smithi077 ceph-mon[91833]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:52 smithi078 ceph-mon[79689]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:55.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:54 smithi077 ceph-mon[91833]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:54 smithi078 ceph-mon[79689]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:56 smithi077 ceph-mon[91833]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:56 smithi078 ceph-mon[79689]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:05:58 smithi077 ceph-mon[91833]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:05:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:05:58 smithi078 ceph-mon[79689]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:00 smithi077 ceph-mon[91833]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:00 smithi078 ceph-mon[79689]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:02.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:01 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:02.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:01 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:02 smithi077 ceph-mon[91833]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:02 smithi078 ceph-mon[79689]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:04 smithi077 ceph-mon[91833]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:04 smithi078 ceph-mon[79689]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:06 smithi077 ceph-mon[91833]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:06 smithi078 ceph-mon[79689]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:08 smithi077 ceph-mon[91833]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:08 smithi078 ceph-mon[79689]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:11.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:10 smithi077 ceph-mon[91833]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:10 smithi078 ceph-mon[79689]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:12 smithi077 ceph-mon[91833]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:12 smithi078 ceph-mon[79689]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:14 smithi077 ceph-mon[91833]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:14 smithi078 ceph-mon[79689]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:16 smithi077 ceph-mon[91833]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:17.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:16 smithi078 ceph-mon[79689]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:19.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:18 smithi077 ceph-mon[91833]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:18 smithi078 ceph-mon[79689]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:21.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:20 smithi077 ceph-mon[91833]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:20 smithi078 ceph-mon[79689]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:22 smithi077 ceph-mon[91833]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:22 smithi078 ceph-mon[79689]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:24 smithi077 ceph-mon[91833]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:24 smithi078 ceph-mon[79689]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:26 smithi077 ceph-mon[91833]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:26 smithi078 ceph-mon[79689]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:28 smithi077 ceph-mon[91833]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:28 smithi078 ceph-mon[79689]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:31.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:30 smithi078 ceph-mon[79689]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:30 smithi077 ceph-mon[91833]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:32 smithi077 ceph-mon[91833]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:32 smithi078 ceph-mon[79689]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:35.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:34 smithi077 ceph-mon[91833]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:34 smithi078 ceph-mon[79689]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:36 smithi077 ceph-mon[91833]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:36 smithi078 ceph-mon[79689]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:38 smithi077 ceph-mon[91833]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:38 smithi078 ceph-mon[79689]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:39 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:06:40.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:39 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:06:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:40 smithi077 ceph-mon[91833]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:06:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:06:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:06:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:40 smithi078 ceph-mon[79689]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:06:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:06:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:06:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:42 smithi077 ceph-mon[91833]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:42 smithi078 ceph-mon[79689]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:45.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:44 smithi077 ceph-mon[91833]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:44 smithi078 ceph-mon[79689]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:46.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:06:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:46 smithi077 ceph-mon[91833]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:46 smithi078 ceph-mon[79689]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:48 smithi077 ceph-mon[91833]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:48 smithi078 ceph-mon[79689]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:50 smithi077 ceph-mon[91833]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:50 smithi078 ceph-mon[79689]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:52 smithi077 ceph-mon[91833]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:52 smithi078 ceph-mon[79689]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:54 smithi077 ceph-mon[91833]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:54 smithi078 ceph-mon[79689]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:56 smithi077 ceph-mon[91833]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:56 smithi078 ceph-mon[79689]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:06:58 smithi077 ceph-mon[91833]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:06:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:06:58 smithi078 ceph-mon[79689]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:00 smithi077 ceph-mon[91833]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:00 smithi078 ceph-mon[79689]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:02 smithi077 ceph-mon[91833]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:02 smithi078 ceph-mon[79689]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:04 smithi077 ceph-mon[91833]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:04 smithi078 ceph-mon[79689]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:07.269 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:06 smithi078 ceph-mon[79689]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:07.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:06 smithi077 ceph-mon[91833]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:08.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:08 smithi077 ceph-mon[91833]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:09.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:08 smithi078 ceph-mon[79689]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:11.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:10 smithi077 ceph-mon[91833]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:10 smithi078 ceph-mon[79689]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:12 smithi077 ceph-mon[91833]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:12 smithi078 ceph-mon[79689]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:15.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:14 smithi077 ceph-mon[91833]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:14 smithi078 ceph-mon[79689]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:16 smithi077 ceph-mon[91833]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:17.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:16 smithi078 ceph-mon[79689]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:19.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:18 smithi077 ceph-mon[91833]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:18 smithi078 ceph-mon[79689]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:21.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:20 smithi077 ceph-mon[91833]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:20 smithi078 ceph-mon[79689]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:22 smithi077 ceph-mon[91833]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:22 smithi078 ceph-mon[79689]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:24 smithi077 ceph-mon[91833]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:24 smithi078 ceph-mon[79689]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:27.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:26 smithi077 ceph-mon[91833]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:26 smithi078 ceph-mon[79689]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:28 smithi077 ceph-mon[91833]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:28 smithi078 ceph-mon[79689]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:30 smithi077 ceph-mon[91833]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:30 smithi078 ceph-mon[79689]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:32 smithi077 ceph-mon[91833]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:32 smithi078 ceph-mon[79689]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:35.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:34 smithi077 ceph-mon[91833]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:34 smithi078 ceph-mon[79689]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:37.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:36 smithi077 ceph-mon[91833]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:36 smithi078 ceph-mon[79689]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:38 smithi077 ceph-mon[91833]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:39.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:38 smithi078 ceph-mon[79689]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:40 smithi077 ceph-mon[91833]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:41.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:40 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:07:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:40 smithi078 ceph-mon[79689]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:40 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:07:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:07:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:07:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:07:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:07:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:07:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:07:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:42 smithi077 ceph-mon[91833]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:42 smithi078 ceph-mon[79689]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:44 smithi077 ceph-mon[91833]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:44 smithi078 ceph-mon[79689]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:46.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:07:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:46 smithi077 ceph-mon[91833]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:46 smithi078 ceph-mon[79689]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:48 smithi077 ceph-mon[91833]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:48 smithi078 ceph-mon[79689]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:50 smithi077 ceph-mon[91833]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:50 smithi078 ceph-mon[79689]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:52 smithi077 ceph-mon[91833]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:52 smithi078 ceph-mon[79689]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:54 smithi077 ceph-mon[91833]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:54 smithi078 ceph-mon[79689]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:57.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:56 smithi077 ceph-mon[91833]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:56 smithi078 ceph-mon[79689]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:07:58 smithi077 ceph-mon[91833]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:07:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:07:58 smithi078 ceph-mon[79689]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:00 smithi077 ceph-mon[91833]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:00 smithi078 ceph-mon[79689]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:02 smithi077 ceph-mon[91833]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:02 smithi078 ceph-mon[79689]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:04 smithi077 ceph-mon[91833]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:04 smithi078 ceph-mon[79689]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:06 smithi077 ceph-mon[91833]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:06 smithi078 ceph-mon[79689]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:08 smithi077 ceph-mon[91833]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:08 smithi078 ceph-mon[79689]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:11.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:10 smithi077 ceph-mon[91833]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:10 smithi078 ceph-mon[79689]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:13.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:12 smithi077 ceph-mon[91833]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:12 smithi078 ceph-mon[79689]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:14 smithi078 ceph-mon[79689]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:14 smithi077 ceph-mon[91833]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:15 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:15 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:17 smithi078 ceph-mon[79689]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:17 smithi077 ceph-mon[91833]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:18.992 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:18 smithi077 ceph-mon[91833]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:19.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:18 smithi078 ceph-mon[79689]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:21.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:20 smithi077 ceph-mon[91833]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:20 smithi078 ceph-mon[79689]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:23.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:22 smithi077 ceph-mon[91833]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:22 smithi078 ceph-mon[79689]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:25.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:24 smithi077 ceph-mon[91833]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:25.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:24 smithi078 ceph-mon[79689]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:27.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:26 smithi077 ceph-mon[91833]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:27.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:26 smithi078 ceph-mon[79689]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:29.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:28 smithi077 ceph-mon[91833]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:29.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:28 smithi078 ceph-mon[79689]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:31.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:30 smithi077 ceph-mon[91833]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:31.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:30 smithi078 ceph-mon[79689]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:32.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:32.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:33.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:32 smithi077 ceph-mon[91833]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:33.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:32 smithi078 ceph-mon[79689]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:35.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:34 smithi077 ceph-mon[91833]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:35.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:34 smithi078 ceph-mon[79689]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:37.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:36 smithi077 ceph-mon[91833]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:37.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:36 smithi078 ceph-mon[79689]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:39.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:38 smithi077 ceph-mon[91833]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:39.270 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:38 smithi078 ceph-mon[79689]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:41.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:40 smithi077 ceph-mon[91833]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:41.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:40 smithi078 ceph-mon[79689]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:08:42.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:08:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:08:42.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:41 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:08:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T05:08:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T05:08:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T05:08:42.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:41 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' 2024-04-26T05:08:43.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:42 smithi077 ceph-mon[91833]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:43.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:42 smithi078 ceph-mon[79689]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:45.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:44 smithi077 ceph-mon[91833]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:45.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:44 smithi078 ceph-mon[79689]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:46.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:45 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:46.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:45 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:08:47.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:46 smithi077 ceph-mon[91833]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:47.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:46 smithi078 ceph-mon[79689]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:49.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:48 smithi077 ceph-mon[91833]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:49.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:48 smithi078 ceph-mon[79689]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:51.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:50 smithi077 ceph-mon[91833]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:51.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:50 smithi078 ceph-mon[79689]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:53.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:52 smithi077 ceph-mon[91833]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:53.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:52 smithi078 ceph-mon[79689]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:55.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:54 smithi077 ceph-mon[91833]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:55.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:54 smithi078 ceph-mon[79689]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:57.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:56 smithi077 ceph-mon[91833]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:57.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:56 smithi078 ceph-mon[79689]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:59.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:08:58 smithi077 ceph-mon[91833]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:08:59.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:08:58 smithi078 ceph-mon[79689]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:01.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:00 smithi077 ceph-mon[91833]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:01.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:00 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:00 smithi078 ceph-mon[79689]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:01.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:00 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:03.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:02 smithi077 ceph-mon[91833]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:03.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:02 smithi078 ceph-mon[79689]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:05.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:04 smithi077 ceph-mon[91833]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:05.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:04 smithi078 ceph-mon[79689]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:07.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:06 smithi077 ceph-mon[91833]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:07.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:06 smithi078 ceph-mon[79689]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:09.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:08 smithi077 ceph-mon[91833]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:09.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:08 smithi078 ceph-mon[79689]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:11.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:10 smithi078 ceph-mon[79689]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:11.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:10 smithi077 ceph-mon[91833]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:13.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:12 smithi078 ceph-mon[79689]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:13.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:12 smithi077 ceph-mon[91833]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:15.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:15 smithi078 ceph-mon[79689]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:15.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:15 smithi077 ceph-mon[91833]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:16.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:16 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:16.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:16 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:17.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:17 smithi078 ceph-mon[79689]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:17.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:17 smithi077 ceph-mon[91833]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:19.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:19 smithi078 ceph-mon[79689]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:19.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:19 smithi077 ceph-mon[91833]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:21.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:21 smithi078 ceph-mon[79689]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:21.295 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:21 smithi077 ceph-mon[91833]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:23.267 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:23 smithi078 ceph-mon[79689]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:23.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:23 smithi077 ceph-mon[91833]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:25.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:25 smithi077 ceph-mon[91833]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:25.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:25 smithi078 ceph-mon[79689]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:27.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:27 smithi077 ceph-mon[91833]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:27.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:27 smithi078 ceph-mon[79689]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:29.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:29 smithi077 ceph-mon[91833]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:29.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:29 smithi078 ceph-mon[79689]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:31.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:31 smithi077 ceph-mon[91833]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:31.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:31 smithi077 ceph-mon[91833]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:31 smithi078 ceph-mon[79689]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:31.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:31 smithi078 ceph-mon[79689]: from='mgr.34104 172.21.15.77:0/286655151' entity='mgr.smithi077.hkvsor' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T05:09:33.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:33 smithi077 ceph-mon[91833]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:33.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:33 smithi078 ceph-mon[79689]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:35.493 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:35 smithi077 ceph-mon[91833]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:35.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:35 smithi078 ceph-mon[79689]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:37.492 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:37 smithi077 ceph-mon[91833]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:37.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:37 smithi078 ceph-mon[79689]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:39.373 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-26T05:09:39.374 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-26T05:09:39.375 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T05:09:39.390 INFO:tasks.cephadm:Teardown begin 2024-04-26T05:09:39.390 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-26T05:09:39.391 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T05:09:39.417 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:39 smithi077 ceph-mon[91833]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:39.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T05:09:39.447 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:39 smithi078 ceph-mon[79689]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-26T05:09:39.453 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T05:09:39.454 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T05:09:39.478 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T05:09:39.511 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T05:09:39.511 INFO:tasks.cephadm.mon.smithi077:Stopping mon.smithi077... 2024-04-26T05:09:39.511 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077 2024-04-26T05:09:39.742 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:39 smithi077 systemd[1]: Stopping Ceph mon.smithi077 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:09:40.242 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:39 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T05:09:39.839+0000 7fe628993700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi077 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:09:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:39 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077[91826]: 2024-04-26T05:09:39.839+0000 7fe628993700 -1 mon.smithi077@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T05:09:40.243 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:39 smithi077 podman[149379]: 2024-04-26 05:09:39.931929004 +0000 UTC m=+0.223181760 container died d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T05:09:40.658 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:40 smithi077 podman[149379]: 2024-04-26 05:09:40.423722782 +0000 UTC m=+0.714975577 container cleanup d3426e141588dbf2d1c04393301ce34da5161fbdb94c7e285e2450bb4d96915f (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb) 2024-04-26T05:09:40.658 INFO:journalctl@ceph.mon.smithi077.smithi077.stdout:Apr 26 05:09:40 smithi077 bash[149379]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi077 2024-04-26T05:09:40.789 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi077.service' 2024-04-26T05:09:40.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:09:40.834 INFO:tasks.cephadm.mon.smithi077:Stopped mon.smithi077 2024-04-26T05:09:40.834 INFO:tasks.cephadm.mon.smithi078:Stopping mon.smithi078... 2024-04-26T05:09:40.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078 2024-04-26T05:09:41.181 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:40 smithi078 systemd[1]: Stopping Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:09:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:41 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078[79685]: 2024-04-26T05:09:41.179+0000 7ff705c27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi078 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:09:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:41 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078[79685]: 2024-04-26T05:09:41.179+0000 7ff705c27700 -1 mon.smithi078@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T05:09:41.517 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:41 smithi078 podman[116381]: 2024-04-26 05:09:41.405789221 +0000 UTC m=+0.380995075 container died ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-26T05:09:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:41 smithi078 podman[116381]: 2024-04-26 05:09:41.839526716 +0000 UTC m=+0.814732578 container cleanup ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T05:09:42.017 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:41 smithi078 bash[116381]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078 2024-04-26T05:09:42.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service' 2024-04-26T05:09:42.335 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:42 smithi078 podman[116396]: 2024-04-26 05:09:42.066024669 +0000 UTC m=+0.656977013 container remove ca912136bf31a3d4856433583b5c783a1a0949edbc3fe60527911f591d087484 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-mon-smithi078, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T05:09:42.335 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:42 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service: Deactivated successfully. 2024-04-26T05:09:42.335 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:42 smithi078 systemd[1]: Stopped Ceph mon.smithi078 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T05:09:42.335 INFO:journalctl@ceph.mon.smithi078.smithi078.stdout:Apr 26 05:09:42 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@mon.smithi078.service: Consumed 6.591s CPU time. 2024-04-26T05:09:42.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:09:42.360 INFO:tasks.cephadm.mon.smithi078:Stopped mon.smithi078 2024-04-26T05:09:42.360 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T05:09:42.360 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.0 2024-04-26T05:09:42.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:42 smithi077 systemd[1]: Stopping Ceph osd.0 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:09:43.242 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:42 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T05:09:42.903+0000 7f3bfbc9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:09:43.243 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:42 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T05:09:42.903+0000 7f3bfbc9a700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-26T05:09:43.243 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:42 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0[101128]: 2024-04-26T05:09:42.903+0000 7f3bfbc9a700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:09:48.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:48 smithi077 podman[149514]: 2024-04-26 05:09:48.435155266 +0000 UTC m=+5.889538817 container died ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T05:09:49.097 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:48 smithi077 podman[149514]: 2024-04-26 05:09:48.836856613 +0000 UTC m=+6.291240149 container cleanup ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T05:09:49.097 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:48 smithi077 bash[149514]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0 2024-04-26T05:09:49.431 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149532]: 2024-04-26 05:09:49.097034061 +0000 UTC m=+0.657798140 container remove ecb573fb9bdd6e35a71ca370c54d3d41e86a8e9cf9b0d6a4720911e48601efc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T05:09:49.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.334493016 +0000 UTC m=+0.018398935 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:09:49.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.431285011 +0000 UTC m=+0.115190910 container create b29c1383c5b30fe351ebd09464f6049fe681c34e4c4f9d54cbea561fa1d9e968 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, ceph=True) 2024-04-26T05:09:49.742 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.642161253 +0000 UTC m=+0.326067154 container init b29c1383c5b30fe351ebd09464f6049fe681c34e4c4f9d54cbea561fa1d9e968 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T05:09:49.743 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.645337531 +0000 UTC m=+0.329243427 container start b29c1383c5b30fe351ebd09464f6049fe681c34e4c4f9d54cbea561fa1d9e968 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T05:09:49.743 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.742142653 +0000 UTC m=+0.426048563 container attach b29c1383c5b30fe351ebd09464f6049fe681c34e4c4f9d54cbea561fa1d9e968 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T05:09:49.993 INFO:journalctl@ceph.osd.0.smithi077.stdout:Apr 26 05:09:49 smithi077 podman[149622]: 2024-04-26 05:09:49.947774828 +0000 UTC m=+0.631680737 container died b29c1383c5b30fe351ebd09464f6049fe681c34e4c4f9d54cbea561fa1d9e968 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T05:09:50.529 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T05:09:50.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:09:50.567 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T05:09:50.567 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T05:09:50.567 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.1 2024-04-26T05:09:50.804 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:50 smithi077 systemd[1]: Stopping Ceph osd.1 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:09:51.242 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:50 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T05:09:50.924+0000 7f4f5b88a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:09:51.242 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:50 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T05:09:50.924+0000 7f4f5b88a700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-26T05:09:51.243 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:50 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1[106985]: 2024-04-26T05:09:50.924+0000 7f4f5b88a700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:09:56.676 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:56 smithi077 podman[149730]: 2024-04-26 05:09:56.417518354 +0000 UTC m=+5.610074860 container died 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T05:09:56.994 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:56 smithi077 podman[149730]: 2024-04-26 05:09:56.774471272 +0000 UTC m=+5.967027769 container cleanup 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T05:09:56.995 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:56 smithi077 bash[149730]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1 2024-04-26T05:09:57.431 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149758]: 2024-04-26 05:09:57.134420786 +0000 UTC m=+0.712973984 container remove 60022e2ec48ebaf741c3721be8bb5dd107a61ba7f1a5f6f054d094aad7ff5d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T05:09:57.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.334519664 +0000 UTC m=+0.018261850 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:09:57.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.452110109 +0000 UTC m=+0.135852313 container create 47387357bddd025a95c86b492312cfddcc119cd21127395714851ac978e19f0a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T05:09:57.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.654407557 +0000 UTC m=+0.338149724 container init 47387357bddd025a95c86b492312cfddcc119cd21127395714851ac978e19f0a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T05:09:57.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.657610413 +0000 UTC m=+0.341352577 container start 47387357bddd025a95c86b492312cfddcc119cd21127395714851ac978e19f0a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T05:09:57.730 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.729646237 +0000 UTC m=+0.413388405 container attach 47387357bddd025a95c86b492312cfddcc119cd21127395714851ac978e19f0a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T05:09:57.993 INFO:journalctl@ceph.osd.1.smithi077.stdout:Apr 26 05:09:57 smithi077 podman[149850]: 2024-04-26 05:09:57.950533215 +0000 UTC m=+0.634275397 container died 47387357bddd025a95c86b492312cfddcc119cd21127395714851ac978e19f0a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T05:09:58.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T05:09:58.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:09:58.584 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T05:09:58.584 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T05:09:58.584 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.2 2024-04-26T05:09:58.912 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:09:58 smithi077 systemd[1]: Stopping Ceph osd.2 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:09:59.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:09:58 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T05:09:58.910+0000 7f4ec63d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:09:59.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:09:58 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T05:09:58.910+0000 7f4ec63d4700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-26T05:09:59.243 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:09:58 smithi077 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2[113104]: 2024-04-26T05:09:58.910+0000 7f4ec63d4700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:10:04.588 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:04 smithi077 podman[149955]: 2024-04-26 05:10:04.306348708 +0000 UTC m=+5.497577667 container died 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T05:10:04.954 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:04 smithi077 podman[149955]: 2024-04-26 05:10:04.661488481 +0000 UTC m=+5.852717447 container cleanup 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD) 2024-04-26T05:10:04.955 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:04 smithi077 bash[149955]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2 2024-04-26T05:10:05.242 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:04 smithi077 podman[149972]: 2024-04-26 05:10:04.954533233 +0000 UTC m=+0.646124842 container remove 9ad85529d167abcc8b45b81de0379d53bb4723e7729383c4b8c3d4af9743d8dd (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T05:10:05.591 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.272215917 +0000 UTC m=+0.103389750 container create 54d1ea9524e4eb4e58cc5ed1df95dd8d41371ff7be8164aac5f5b63b21255cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T05:10:05.592 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.186818871 +0000 UTC m=+0.017992682 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:10:05.592 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.507936165 +0000 UTC m=+0.339109956 container init 54d1ea9524e4eb4e58cc5ed1df95dd8d41371ff7be8164aac5f5b63b21255cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T05:10:05.592 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.511269661 +0000 UTC m=+0.342443453 container start 54d1ea9524e4eb4e58cc5ed1df95dd8d41371ff7be8164aac5f5b63b21255cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T05:10:05.592 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.591261618 +0000 UTC m=+0.422435426 container attach 54d1ea9524e4eb4e58cc5ed1df95dd8d41371ff7be8164aac5f5b63b21255cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T05:10:05.992 INFO:journalctl@ceph.osd.2.smithi077.stdout:Apr 26 05:10:05 smithi077 podman[150064]: 2024-04-26 05:10:05.799676362 +0000 UTC m=+0.630850147 container died 54d1ea9524e4eb4e58cc5ed1df95dd8d41371ff7be8164aac5f5b63b21255cb7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T05:10:06.435 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T05:10:06.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:10:06.473 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T05:10:06.473 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T05:10:06.473 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.3 2024-04-26T05:10:06.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:06 smithi078 systemd[1]: Stopping Ceph osd.3 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:10:07.026 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:06 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T05:10:06.818+0000 7fc29f97d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:10:07.027 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:06 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T05:10:06.818+0000 7fc29f97d700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-26T05:10:07.027 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:06 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T05:10:06.818+0000 7fc29f97d700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:10:09.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:09 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:09.469+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:10.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:10 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T05:10:10.307+0000 7fc29876a700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:44.830353+0000 front 2024-04-26T05:09:44.830400+0000 (oldest deadline 2024-04-26T05:10:10.130159+0000) 2024-04-26T05:10:10.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:10 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:10.469+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:11.767 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:11 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3[89091]: 2024-04-26T05:10:11.351+0000 7fc29876a700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:44.830353+0000 front 2024-04-26T05:09:44.830400+0000 (oldest deadline 2024-04-26T05:10:10.130159+0000) 2024-04-26T05:10:11.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:11 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:11.437+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:12.542 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:12 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:12.389+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:12.542 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:12 smithi078 podman[116505]: 2024-04-26 05:10:12.27659816 +0000 UTC m=+5.613517639 container died a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T05:10:12.953 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:12 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:12.541+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:12.954 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:12 smithi078 podman[116505]: 2024-04-26 05:10:12.660235404 +0000 UTC m=+5.997154884 container cleanup a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T05:10:12.954 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:12 smithi078 bash[116505]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3 2024-04-26T05:10:13.267 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:12 smithi078 podman[116535]: 2024-04-26 05:10:12.953403642 +0000 UTC m=+0.669860326 container remove a56a9fe0bde4a205ad1789de6df73a80b052e62b01842a76c8add0f22a9068b9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T05:10:13.547 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:13.399+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:13.548 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.270974331 +0000 UTC m=+0.109970858 container create 407ae8f80766c81edd0f2aa2374f0f1104aa02e00f96e4afcebce5aee8629536 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T05:10:13.548 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.178972877 +0000 UTC m=+0.017969379 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:10:13.548 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.473767875 +0000 UTC m=+0.312764361 container init 407ae8f80766c81edd0f2aa2374f0f1104aa02e00f96e4afcebce5aee8629536 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T05:10:13.548 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.477541495 +0000 UTC m=+0.316537992 container start 407ae8f80766c81edd0f2aa2374f0f1104aa02e00f96e4afcebce5aee8629536 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T05:10:14.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:13 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:13.546+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:14.017 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.548676293 +0000 UTC m=+0.387672795 container attach 407ae8f80766c81edd0f2aa2374f0f1104aa02e00f96e4afcebce5aee8629536 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T05:10:14.018 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 26 05:10:13 smithi078 podman[116626]: 2024-04-26 05:10:13.767127184 +0000 UTC m=+0.606123662 container died 407ae8f80766c81edd0f2aa2374f0f1104aa02e00f96e4afcebce5aee8629536 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T05:10:14.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T05:10:14.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:10:14.386 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T05:10:14.386 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T05:10:14.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.4 2024-04-26T05:10:14.723 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:14.382+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:14.723 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:14 smithi078 systemd[1]: Stopping Ceph osd.4 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:10:14.723 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:14.545+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:15.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:14.722+0000 7f95e4b27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:10:15.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:14.722+0000 7f95e4b27700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-26T05:10:15.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:14 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:14.722+0000 7f95e4b27700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:10:15.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:15 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:15.506+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:15.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:15 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:15.389+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:16.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:16 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:16.554+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:16.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:16 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:16.438+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:17.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:17 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:17.583+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:17.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:17 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:17.430+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:17.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:17 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:17.430+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:18.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:18 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:18.553+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:18.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:18 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:18.470+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:18.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:18 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:18.470+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:19.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:19 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:19.516+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:46.836944+0000 front 2024-04-26T05:09:46.836857+0000 (oldest deadline 2024-04-26T05:10:12.136713+0000) 2024-04-26T05:10:19.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:19 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4[94305]: 2024-04-26T05:10:19.516+0000 7f95e191c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:55.537728+0000 front 2024-04-26T05:09:55.537745+0000 (oldest deadline 2024-04-26T05:10:19.037673+0000) 2024-04-26T05:10:19.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:19 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:19.446+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:19.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:19 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:19.446+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:20.333 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:20 smithi078 podman[116732]: 2024-04-26 05:10:20.081960987 +0000 UTC m=+5.474449752 container died 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T05:10:20.641 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:20 smithi078 podman[116732]: 2024-04-26 05:10:20.431182774 +0000 UTC m=+5.823671541 container cleanup 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T05:10:20.641 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:20 smithi078 bash[116732]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4 2024-04-26T05:10:20.641 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:20 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:20.468+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:20.641 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:20 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:20.468+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:21.017 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:20 smithi078 podman[116748]: 2024-04-26 05:10:20.640916978 +0000 UTC m=+0.551739962 container remove 61aaf454ddae9a39e6d8e426c47d29d24612af4ed699cfdc65ba54894775387a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T05:10:21.319 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:20.950836822 +0000 UTC m=+0.017962625 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:10:21.321 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:21.050307032 +0000 UTC m=+0.117432877 container create 1a33548beac6f1ac1f41cf43624dc34f35d389901e34669568d4eddcc4d07342 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T05:10:21.321 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:21.244292906 +0000 UTC m=+0.311418722 container init 1a33548beac6f1ac1f41cf43624dc34f35d389901e34669568d4eddcc4d07342 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T05:10:21.321 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:21.247268634 +0000 UTC m=+0.314394444 container start 1a33548beac6f1ac1f41cf43624dc34f35d389901e34669568d4eddcc4d07342 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T05:10:21.321 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:21.31924009 +0000 UTC m=+0.386365895 container attach 1a33548beac6f1ac1f41cf43624dc34f35d389901e34669568d4eddcc4d07342 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T05:10:21.767 INFO:journalctl@ceph.osd.4.smithi078.stdout:Apr 26 05:10:21 smithi078 podman[116843]: 2024-04-26 05:10:21.530470908 +0000 UTC m=+0.597596711 container died 1a33548beac6f1ac1f41cf43624dc34f35d389901e34669568d4eddcc4d07342 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T05:10:21.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:21 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:21.493+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:21.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:21 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:21.493+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:22.144 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T05:10:22.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:10:22.184 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T05:10:22.185 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T05:10:22.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5 2024-04-26T05:10:22.514 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 systemd[1]: Stopping Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968... 2024-04-26T05:10:22.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:22.513+0000 7fdefa032700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T05:10:22.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:22.513+0000 7fdefa032700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-26T05:10:22.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:22.513+0000 7fdefa032700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T05:10:22.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:22.513+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:22.768 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:22 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:22.513+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:24.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:23 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:23.521+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:24.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:23 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:23.521+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:24.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:24 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:24.538+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:24.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:24 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:24.538+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:26.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:25 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:25.563+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:26.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:25 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:25.563+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:27.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:26 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:26.519+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:27.017 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:26 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:26.519+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:27.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:27 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:27.485+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-04-26T05:09:43.435267+0000 front 2024-04-26T05:09:43.435407+0000 (oldest deadline 2024-04-26T05:10:08.735321+0000) 2024-04-26T05:10:27.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:27 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:27.485+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6814 osd.1 since back 2024-04-26T05:09:52.136436+0000 front 2024-04-26T05:09:52.136509+0000 (oldest deadline 2024-04-26T05:10:16.836416+0000) 2024-04-26T05:10:27.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:27 smithi078 ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5[99327]: 2024-04-26T05:10:27.485+0000 7fdef6e27700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.77:6822 osd.2 since back 2024-04-26T05:10:03.738098+0000 front 2024-04-26T05:10:03.738076+0000 (oldest deadline 2024-04-26T05:10:26.638004+0000) 2024-04-26T05:10:28.273 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 podman[116950]: 2024-04-26 05:10:28.022391078 +0000 UTC m=+5.614701952 container died 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T05:10:28.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 podman[116950]: 2024-04-26 05:10:28.363013569 +0000 UTC m=+5.955324451 container cleanup 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T05:10:28.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 bash[116950]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5 2024-04-26T05:10:28.767 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 podman[116969]: 2024-04-26 05:10:28.522570954 +0000 UTC m=+0.495730573 container remove 2f3ab9eeb39b5be44a3f41ab83703668ff98a353853c32eb6c32d48332e5f475 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T05:10:29.218 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 podman[117058]: 2024-04-26 05:10:28.837260722 +0000 UTC m=+0.017888276 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T05:10:29.218 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:28 smithi078 podman[117058]: 2024-04-26 05:10:28.940107792 +0000 UTC m=+0.120735340 container create bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T05:10:29.219 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 podman[117058]: 2024-04-26 05:10:29.126252958 +0000 UTC m=+0.306880499 container init bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T05:10:29.219 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 podman[117058]: 2024-04-26 05:10:29.129228706 +0000 UTC m=+0.309856248 container start bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T05:10:29.517 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 podman[117058]: 2024-04-26 05:10:29.217926793 +0000 UTC m=+0.398554340 container attach bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T05:10:29.517 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 podman[117058]: 2024-04-26 05:10:29.418867326 +0000 UTC m=+0.599494866 container died bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T05:10:29.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T05:10:29.943 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 podman[117084]: 2024-04-26 05:10:29.896432468 +0000 UTC m=+0.469370691 container remove bd00327f9b4b230e83b93cd30d49ed569585f3cc27eafbfb42eea5eaa0f43ce2 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-3c1ee652-0387-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T05:10:29.944 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-26T05:10:29.944 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 systemd[1]: Stopped Ceph osd.5 for 3c1ee652-0387-11ef-bc93-c7b262605968. 2024-04-26T05:10:29.944 INFO:journalctl@ceph.osd.5.smithi078.stdout:Apr 26 05:10:29 smithi078 systemd[1]: ceph-3c1ee652-0387-11ef-bc93-c7b262605968@osd.5.service: Consumed 6.305s CPU time. 2024-04-26T05:10:29.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T05:10:29.958 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T05:10:29.958 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T05:10:33.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi077.stderr:ceph-fuse[73838]: fuse finished with error 0 and tester_r 0 2024-04-26T05:10:50.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T05:11:05.113 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T05:11:05.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T05:11:05.166 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T05:11:05.168 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi077/crash 2024-04-26T05:11:05.169 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash -- . 2024-04-26T05:11:05.202 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T05:11:05.202 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-04-26T05:11:05.204 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi078/crash 2024-04-26T05:11:05.205 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash -- . 2024-04-26T05:11:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T05:11:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-04-26T05:11:05.229 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T05:11:05.229 DEBUG:teuthology.orchestra.run.smithi077:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-26T05:11:05.272 INFO:teuthology.orchestra.run.smithi077.stdout:1714107000.0001655 mon.smithi077 (mon.0) 668 : cluster [ERR] fs cephfs is offline because no MDS is active for it. 2024-04-26T05:11:05.273 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-26T05:11:05.273 INFO:tasks.cephadm:Compressing logs... 2024-04-26T05:11:05.273 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T05:11:05.316 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T05:11:05.335 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-26T05:11:05.335 INFO:teuthology.orchestra.run.smithi077.stderr:: No such file or directory 2024-04-26T05:11:05.335 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.urxtgp.log 2024-04-26T05:11:05.336 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log 2024-04-26T05:11:05.336 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.urxtgp.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T05:11:05.337 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T05:11:05.338 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.quaiyg.log 2024-04-26T05:11:05.338 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T05:11:05.338 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.quaiyg.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi078.log 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi078.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T05:11:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T05:11:05.341 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T05:11:05.341 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.0.log: 88.0% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T05:11:05.342 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi077.log 2024-04-26T05:11:05.342 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T05:11:05.342 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi077.hkvsor.log 2024-04-26T05:11:05.342 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi077.log: 91.1% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T05:11:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi078.navvuw.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi078.navvuw.log: 88.1% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.nihewy.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.wkrgkp.log 2024-04-26T05:11:05.345 INFO:teuthology.orchestra.run.smithi078.stderr: 91.2% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T05:11:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T05:11:05.365 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.nihewy.log: gzip -5 --verbose -- /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T05:11:05.366 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi077.hkvsor.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T05:11:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.wkrgkp.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T05:11:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T05:11:05.416 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T05:11:05.425 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T05:11:05.441 INFO:teuthology.orchestra.run.smithi078.stderr: 89.9% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi078.navvuw.log.gz 2024-04-26T05:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T05:11:05.744 INFO:teuthology.orchestra.run.smithi077.stderr: 88.7% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mgr.smithi077.hkvsor.log.gz 2024-04-26T05:11:05.778 INFO:teuthology.orchestra.run.smithi078.stderr: 92.5% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi078.log.gz 2024-04-26T05:11:06.585 INFO:teuthology.orchestra.run.smithi077.stderr: 90.4% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mon.smithi077.log.gz 2024-04-26T05:11:09.651 INFO:teuthology.orchestra.run.smithi077.stderr: 93.6% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T05:11:10.294 INFO:teuthology.orchestra.run.smithi078.stderr: 93.5% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T05:11:10.488 INFO:teuthology.orchestra.run.smithi078.stderr: 93.7% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T05:11:10.657 INFO:teuthology.orchestra.run.smithi077.stderr: 93.7% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T05:11:11.070 INFO:teuthology.orchestra.run.smithi077.stderr: 93.7% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T05:11:11.235 INFO:teuthology.orchestra.run.smithi078.stderr: 93.7% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T05:11:11.362 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.nihewy.log.gz 2024-04-26T05:11:12.909 INFO:teuthology.orchestra.run.smithi077.stderr: 94.8% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.urxtgp.log.gz 2024-04-26T05:11:13.167 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi078.wkrgkp.log.gz 2024-04-26T05:11:19.820 INFO:teuthology.orchestra.run.smithi077.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-26T05:11:20.867 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-26T05:11:20.867 INFO:teuthology.orchestra.run.smithi078.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-26T05:11:20.869 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-04-26T05:11:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m15.539s 2024-04-26T05:11:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m44.561s 2024-04-26T05:11:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m1.256s 2024-04-26T05:12:36.834 INFO:teuthology.orchestra.run.smithi077.stderr: 92.8% -- replaced with /var/log/ceph/3c1ee652-0387-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi077.quaiyg.log.gz 2024-04-26T05:12:36.835 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-04-26T05:12:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:real 1m31.509s 2024-04-26T05:12:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:user 2m6.386s 2024-04-26T05:12:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m3.964s 2024-04-26T05:12:36.836 INFO:tasks.cephadm:Archiving logs... 2024-04-26T05:12:36.836 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi077/log 2024-04-26T05:12:36.837 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T05:12:55.285 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi078/log 2024-04-26T05:12:55.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T05:13:00.868 INFO:tasks.cephadm:Removing cluster... 2024-04-26T05:13:00.868 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 --force 2024-04-26T05:13:01.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c1ee652-0387-11ef-bc93-c7b262605968 --force 2024-04-26T05:13:02.165 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T05:13:02.165 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T05:13:02.182 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T05:13:02.197 INFO:tasks.cephadm:Teardown complete 2024-04-26T05:13:02.197 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T05:13:02.208 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-26T05:13:02.209 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T05:13:02.209 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T05:13:02.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T05:13:02.301 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T05:13:02.312 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T05:13:02.313 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T05:13:02.315 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T05:13:02.328 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-04-26T05:13:02.329 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-26T05:13:02.332 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T05:13:02.332 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-04-26T05:13:02.332 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 7 377 76 +1088us[+1153us] +/- 55ms 2024-04-26T05:13:02.332 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1153us[-1153us] +/- 76ms 2024-04-26T05:13:02.332 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +90us[ +90us] +/- 70ms 2024-04-26T05:13:02.333 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T05:13:02.334 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T05:13:02.334 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-26T05:13:02.334 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +934us[ +929us] +/- 56ms 2024-04-26T05:13:02.334 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -1327us[-1327us] +/- 76ms 2024-04-26T05:13:02.335 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -44us[ -44us] +/- 70ms 2024-04-26T05:13:02.335 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T05:13:02.335 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T05:13:02.346 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T05:13:02.347 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T05:13:02.377 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T05:13:02.458 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T05:13:02.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T05:13:02.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T05:13:02.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T05:13:02.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T05:13:02.583 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T05:13:02.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T05:13:02.629 INFO:teuthology.task.internal:Duration was 2235.956846 seconds 2024-04-26T05:13:02.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T05:13:02.679 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T05:13:02.679 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T05:13:02.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T05:13:02.716 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T05:13:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T05:13:03.044 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T05:13:03.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-04-26T05:13:03.045 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T05:13:03.104 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-04-26T05:13:03.105 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T05:13:03.135 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T05:13:03.135 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T05:13:03.148 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T05:13:03.199 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T05:13:03.199 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T05:13:03.212 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T05:13:03.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T05:13:03.884 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T05:13:03.884 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T05:13:03.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T05:13:03.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T05:13:03.949 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T05:13:03.952 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T05:13:03.971 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-04-26T05:13:03.995 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-04-26T05:13:04.015 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T05:13:04.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T05:13:04.040 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T05:13:04.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T05:13:04.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T05:13:04.085 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T05:13:04.086 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi077 2024-04-26T05:13:04.086 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T05:13:04.140 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581/remote/smithi078 2024-04-26T05:13:04.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T05:13:04.177 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T05:13:04.177 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T05:13:04.180 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T05:13:04.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T05:13:04.238 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T05:13:04.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T05:13:04.333 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T05:13:04.333 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T05:13:04.336 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T05:13:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 26 05:13 /home/ubuntu/cephtest 2024-04-26T05:13:04.348 INFO:teuthology.orchestra.run.smithi077.stdout: 550976 4 d--------- 2 ubuntu ubuntu 4096 Apr 26 04:50 /home/ubuntu/cephtest/mnt.0 2024-04-26T05:13:04.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T05:13:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-26T05:13:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-26T05:13:04.349 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi077 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-26T05:13:04.350 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T05:13:04.382 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T05:13:04.763 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-26T05:13:04.763 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T05:13:04.791 DEBUG:teuthology.nuke:shortname: smithi077 2024-04-26T05:13:04.792 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T05:13:04.817 DEBUG:teuthology.nuke:shortname: smithi078 2024-04-26T05:13:04.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T05:13:04.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:30:16.524234', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHokJW6KKOphAlUAn1Gx1/UP8XseML1VMycrSBv1nQQ1/dIpgnqyI3TpBiwQYhDdalWfw3FYL81jXKDCE99hi8g='} 2024-04-26T05:13:04.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:30:16.525782', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOp+Y3P28sQr8wXSrFkYhaXy48KVGlV8+JsgWS2VxVwXXWKbBMW84m8U+YDunFdWQ6WzgqHsTJJZvZojMKivmzA='} 2024-04-26T05:13:04.873 INFO:teuthology.orchestra.console.smithi077:Power off 2024-04-26T05:13:04.873 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T05:13:04.888 INFO:teuthology.orchestra.console.smithi078:Power off 2024-04-26T05:13:04.888 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T05:13:04.900 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2024-04-26T05:13:04.900 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:04.914 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2024-04-26T05:13:04.914 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:04.923 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2024-04-26T05:13:04.938 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-04-26T05:13:08.925 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:08.939 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:09.035 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2024-04-26T05:13:09.048 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-04-26T05:13:13.040 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:13.049 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:13.150 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2024-04-26T05:13:13.158 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-04-26T05:13:13.252 INFO:teuthology.orchestra.console.smithi077:Power off completed 2024-04-26T05:13:17.162 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T05:13:17.272 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-04-26T05:13:17.373 INFO:teuthology.orchestra.console.smithi078:Power off completed 2024-04-26T05:13:17.477 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2235.956845521927 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-26T05:13:17.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T05:13:17.583 INFO:teuthology.run:FAIL